- Anybody on social media has seen it.

Dueling posts arguing over a hot political issue.

The protagonists in these online debates are generally sincere.

But today's guest says that too many of us remain unaware of the media biases that shape our understanding of the world.

She's Vanessa Otero, this week on Story in the Public Square.

(Story in the Public Square theme music) Hello and welcome to Story in the Public Square, where storytelling meets public affairs.

I'm Jim Ludes from The Pell Center at Salve Regina University.

Joining me from his home in Rhode Island is my friend and co-host G. Wayne Miller of The Providence Journal.

Each week we talk about big issues with great guests, authors, journalists, artists and more, to make sense of the big stories that shape public life in the United States today.

This week we're joined by Vanessa Otero, founder and CEO of Ad Fontes Media.

But you probably have seen her work as the creator of The Media Bias Chart.

Vanessa, thank you so much for bein' with us.

- It's my pleasure to be here.

Thanks for having me.

- So I was telling you before we started that my stepson was actually the one who discovered The Media Bias Chart couple years ago on social media.

For those who haven't had the benefit of his exposure to the world, tell us what The Media Bias Chart is.

- Yeah, if you can just visualize a two dimensional graph, x-axis, y-axis, and on the vertical side it's reliability, news value and reliability.

So generally people recognize good at the top, okay in the middle, bad at the bottom, and all the things those mean.

And then left to right, your political bias.

And we plot logos of news sources on that chart.

So it's evolved over the years.

There have been several different iterations that have floated around the internet.

But it tends to spark a lot of discussion because everyone looks for their favorite news sources, and their least favorite news sources, and likes to talk about where they are and where they think they should be, and all the stuff that comes with that.

- So how did this come to be?

Because you're not a media analyst by trade and you're a patent attorney.

How did this come to be?

- Yes well, I'm a media analyst now.

But about four years ago, just over four years ago in the run-up to the 2016 election, you know, just workin' my day job as a patent attorney.

But I've always been, you know, a politics and news junky.

I just love it.

And it was bothering me how people would argue on social media about just things that they disagree with.

In particular, they would throw links at each other from different polarizing websites that did nothing to convince the people on the other side.

And I just thought it might facilitate conversation to kind of lay it out visually.

You know, the concept I had in my head that, you know, there are some that are better and worse for different reasons.

And some news sources that are more or less biased for different reasons.

And we don't talk about them in a very intelligent way.

So as a hobby, because I'm a huge nerd and I consider this a hobby.

(hosts laughing) - [G. Wayne] Look, you're in good company with us.

- Yeah, I spent a few weeks looking at various media sources that I was familiar with and then some new ones, and came up with a system for classifying them, and just threw it together like on a Microsoft Visio program, which is what I used for patent drawings in my job.

I'm not a graphic design kind of person at all.

So it's kinda ugly actually.

And I put it online to talk to my friends about it and it just went way viral beyond my wildest expectations.

- So the professed ambition of Ad Fontes is, and I'm gonna quote from your website, to make news consumers smarter, and news media better.

Those are lofty goals.

Break that down for us.

- Yeah, they're very lofty goals.

And I think we're at this point where everybody realizes that there is a problem.

And I think we've been there for a while.

There are problems with our news ecosystem.

There's toxicity in it, you know, misinformation, disinformation, polarization.

And they have resulted in really dire consequences especially this year.

And we just think of the death toll of COVID misinformation, and the death and destruction, violence, from election disinformation.

Just as a couple of examples, but there's so many more.

And you know, just putting up polls about, oh look, people don't trust the news.

Oh look, people are more polarized than ever.

I'm kind of tired of just seeing that information.

Like, we know there's a problem.

So what do we do about it?

And what I learned from putting this map of the media landscape out there, that there was this baseline need for people to understand where news sources fall in terms of reliability and bias.

Because various different stakeholders in good news media have a role to play.

It's not just, you know, one.

It's not just consumers.

There's not just educators teaching media literacy.

It's not just, you know, the social platforms.

Everyone has a role to play in making the news media ecosystem better.

But one thing that we can provide to all those stakeholders is these news content ratings, which haven't really existed before.

- So you also state who you consider to be the stakeholders.

And I think this is a very good summary.

And those include consumers, educators, journalism outlets, researchers, advertisers, and social media platforms.

Talk about that.

I mean that's really, that's probably all of us one way or another.

There's probably some overlap there too as well.

- Yeah absolutely.

I mean as consumers, just as a baseline, trying to determine the credibility and bias of news sources ourselves, be aware of it, and importantly, not spread misinformation.

And just educating yourself is part of that.

That's something that we do.

For educators, being able to teach media literacy more effectively and more widely.

I mean, there's a huge push especially in the subset of media literacy that is news literacy.

Because it's just become such a problem recently to integrate these kind of new curricula in schools.

But not everybody's in school.

Not everybody has a chance, or the time, the inclination, or the ability, to discern news sources for themselves.

So while that's the ideal, you know, folks that are in positions of power in the media and in distributing our media content need to take responsibility for what they put out there.

So for publishers, you know, the news business is a tough business.

And it changed a lot in the last few years.

For publishers to be able to pursue a business model and then avoid the temptation of, you know, putting up click bait, or lots of opinion sources in the place of news.

You know, punditry instead of reporting.

You know, that's a tricky balance that a lot of publishers have to grapple with.

- You know, I'm a historian by training.

And I had a reason not too long ago to look at some local newspapers from the 1930s.

I'm talkin' about small town local newspapers that were chock full of world news.

And that media environment is so different and so foreign to us today that I'm just curious sort of the reaction you've gotten from the publishers of the world.

Whether we're talkin' about a social media publisher, or a fine print book publisher, or sort of the broadcast and print media that are the mainstay of America's public forums these days.

- You know, honestly the folks that are interested in making the media better.

You know, journalists are among the most dedicated professionals to any profession you can find.

They do it for the love.

They do it for the passion.

It's hard, hard work.

And they want to be doing the right thing, the good thing, when they're working at these good publications, right.

So the response has been incredible.

I mean, major news organizations care about where they fall on The Media Bias Chart.

Now at the opposite end of the spectrum, you know, in the bottom corners, you can imagine they are like not a big fan of my company, and my work, right.

Almost to each one, they know what they're doing.

I mean, if you are putting out purposely polarizing, dehumanizing, vilifying, misleading and fake content, you're doing it because you want to attract certain audience.

You know there's money to be made.

- [Jim] It's a business model.

- Exactly, yeah so.

Different folks on different parts of the spectrum like us and don't like us for those reasons.

- How many news sources do you look at and analyze?

I mean, you see several of them obviously.

Many, many of them.

How many in total?

Do you have a number?

- Yeah, currently right now we have about 330 fully rated publications on our interactive media bias chart.

And in order to be fully rated, we need at least 13-15 news articles per news source, news or news-like source.

And we've got about 500 or so partially rated.

So we keep adding.

Right now we're adding about 80-100 new publishers to our interactive media bias chart and our database every month.

- So I'm curious.

How do you...

I guess it's two questions.

One is about the methodology.

So how do you actually rate these outlets?

And who actually is doing it?

Do you have a team of analysts that are looking at this stuff?

- Yep, absolutely.

So we're really lucky, our analysts are the best people.

They are, you know, a lot of 'em are journalists, teachers.

But others have, you know, a really wide background, different professional backgrounds.

And they are paid analysts.

They work pretty significant part-time hours for us.

And they rate news sources in panels of right-leaning, center-leaning, left-leaning analysts.

So three, like one of each, will be in a Zoom room together rating articles for each particular news source.

So our methodology is article by article.

It's a content analysis methodology.

So we look at a number of sub-factors for reliability.

We call 'em expression, voracity, headline and graphic are the most important ones.

And then for bias we have several sub-factors.

Which are political position, language, and comparison.

And there are, you know, different universes in each one of those things.

So each of our analysts has gone through several weeks of training before actually getting into analysis sessions.

And now they've got, you know, we've been doing this for about six months with a full analyst team consistently now.

Before that, we had some volunteers.

Before that, like back in the day, it was just me.

But that was just, you know... That was just bias 'cause it was just me.

(group laughing) - So these are analysts that come from the political spectrum from left to right.

Is that correct?

- Absolutely.

- That's such an important point.

So maybe you can elaborate on that.

This isn't, you know, just one group of people who are solely left, solely right, solely in the middle.

It sounds like there's a real balance here which can allow you to be objective in a way that, you know, just looking at page views or whatever, some other methodology would not allow.

Can you elaborate on that?

- Absolutely, yeah.

It's really important to gain trust and to have transparency that you have a broad coalition of folks that are looking at this through an objective as possible methodology.

All humans are biased.

Each one of us are biased on our team.

So the best we can do is try to mitigate bias.

So the scores for each article, and then rolled up to the scores for each source, are averages of the ratings.

But what we found is because of our methodology, folks from left, right, and center, generally come to the same neighborhood of scores on a particular article.

So if you look at a left-leaning opinion piece from the New York Times, the left person, and the right person, and the center person are all going to say this is left-leaning opinion, right.

I mean, it might be off by a few points here and there.

But you know, the more granular you get, the more you agree.

Your political opinion and viewpoint does factor in.

It does help you see things that maybe the other person doesn't see.

But then we can resolve discrepancies in-person, live, come to really high levels of agreement.

And so it's an average.

Each score's an average and The Media Bias Chart therefore doesn't represent the reality of any one person's view.

It's sort of a composite reality of these well-trained, politically astute analysts.

- So you talk about these Zoom meetings in which you hash this out.

Do they ever, for lack of a better term, get lively, or contentious?

I imagine they probably would at some point.

I mean, from time to time.

But a lot less than I originally expected, honestly.

And like I said, our analysts are the best.

And I really mean that.

So what helps is the format.

Because it's not like a discussion group like, you know, left, right, and center person are gonna try to convince each other that their political viewpoint on a thing is the correct one.

You know, that can get... That's really hard.

You're looking at an article.

And an article might be about a very difficult subject, a very polarizing subject.

I mean, we have so many to choose from.

You know, abortion, and guns, and race, right.

There's so many polarizing topics.

But what the analysts are saying is, I think this is more or less reliable because of this sentence, or this, you know, inclusion or exclusion to something.

Or, I think this is more left or right biased because of similar factors.

Like these words, these characterizations.

And so they're seeing things from their point of view politically, but they're talking about the article.

So it's like a level of abstraction.

Which does help, you know, reduce that kind of any discord.

I won't say there's no discord, right.

People certainly can disagree about these things.

But everyone's really great about handling those.

- How do you decide who gets rated?

- Well, it's been a process as we've grown.

So, we get a lot of requests as we've... - [Jim] People who want to be rated?

- Well sometimes from the publishers.

But mostly from consumers.

People come to our website.

That was the first when I first put out The Media Bias Chart.

They're like, well what about this one?

Well there's a lot of them out there.

There are thousands and thousands of news sources out there.

And you know, we started with just online web content.

But you know, there's a whole world there, right.

So some of the online content also has print content.

And then there's TV content.

And you can't really leave out that, especially your MSNBC, FOX, and CNN, right.

So we have to do that.

We've actually just recently gotten into podcast content.

And so our methodology allows us to rate any piece of content.

Some obviously take longer than others.

But you know, it's very similar methodology.

So to get back to your original question, you know, we would first start with the most requested.

But then as we, if we didn't have any particular requests, we just go by like Alexa rankings for web traffic.

That was a first way.

But we get so many requests now from the people who come to our site that, you know, we won't run out for a long time.

- So Vanessa, do you ever go back and look at a news source that you've previously rated and see if they've changed?

Because some news sources do change their political views over the months and years.

Not all do, but some certainly do change.

- Yes absolutely.

And that's a really important part of what we do.

We come up with updates all the time.

And not just to add new news sources, but to show any significant ones that have moved.

So we are in the process of, you know, making regular periodic updates.

We're at the point now where we have so many sources that we've got to consistently go back and every quarter add new ratings to existing sources.

One way we've done that over the years is by keeping track of current events, you know.

When people talk about what various news sources are doing, we turn and pay attention to those.

So you know, sources go in and out of existence all the time.

That's one reason we need to do it.

Like you said, some sources change.

Like IJR is one.

There's like fairly new news source in the last 10 years or so that shifted from right to center.

Now it's just like a little bit center-right.

I mean, it's substantially changed its character.

You know, the Blaze has changed a little bit over the years back and forth.

And notably during the, you know, this election, the post-election season especially, there was a real divergence you'd see on sources on the right.

There were some that really went in on, that were, you know, kind of a mix of reliability, and had like middling scores, particularly OAN, Newsmax, and Epoch Times, that really went in really hard on election disinformation, like as a choice.

So you know, going back and just adding a few samples of those really dropped their scores.

Whereas other sources that were sort of in the similar area were publishing fact checks about election disinformation.

You know, the conservative sources publishing fact checks informing their conservative audiences.

Like you know, the news over there, that disinformation is disinformation.

So you'd see this spread.

Those kind of things happen periodically from time to time.

It doesn't just happen on the right either.

- So you don't have ads.

So how do you support yourselves financially?

What is the funding?

'Cause this is a large enterprise, clearly.

- We're getting there.

So we're a public benefit corporation, which is a for-profit corporation with a stated public mission.

You know, that's our entity status.

Now why are we for-profit?

You know, a lot of folks kind of assume that because of the nature, the work we do in just educating the public, we have some education products, that you know, we might be a nonprofit or something like that.

But content analysis is time consuming, labor intensive and hard to do accurately.

And there's a lot of it out there.

We plan to address some of the issues of scale, the like sheer scale of the problem, through some automation.

But it's not this thing that you can just like wave a wand and like AI-ify it, and then like get these correct ratings.

If that was the case, the biggest tech companies in the world would have done it already.

So you know, how to solve this problem of making a business out of what we do?

Well it turns out there's a huge need for these news content ratings.

For consumers, for educators.

You know, we have a education media literacy product.

It's a SUMMA curriculum.

And it allows students and teachers to rate articles, learn how to rate them according to our methodology, and create their own versions of The Media Bias Chart for their classroom or their school.

And then we have brand safety products.

So I mentioned advertisers have a big role to play in fixing the news ecosystem.

Brands are what fund good journalism and polarizing junk news.

And they don't have a way to distinguish between the two.

So our data we've incorporated into products for websites and podcasts that advertisers can tell what their ad dollars are going to.

- You know, Vanessa, looking at the chart, I'm struck by the presence of Sputnik and RT, formerly Russia Today, on the right-hand side of that spectrum.

These are Russian government controlled media platforms that it's been well-documented have been active in everything from election interference to just the disinformation campaigns.

Not just the United States but around the world.

Do you see any comparable plays by government-owned media outlets on the left, or is this concentrated mostly in the right right now?

- You know, that's a fascinating thing to study, we found.

We made sure that we rated some extra samples from RT and Sputnik because, you know, it's sort of insidious how they go about it.

You know, the score is like on the right.

But like it's not the worst.

You know, they are certainly known for publishing disinformation.

However, if you go to the site regularly and read samples of their articles, many are just innocuous and they're like valid news reporting.

And I think that's a strategy.

It's a strategy to retain audiences.

I mean, they've hired like reputable journalists from the United States.

RT in particular is directed at US audiences.

And then when they throw in polarizing stories, they're not always necessarily right-leaning.

They tend to be more often than not.

But sometimes they're just sort of generally anti-American sentiments that folks on the left and right might both find appealing.

We haven't come across any foreign state media trying to do similar things that are more left-leaning.

You know, South China Morning Post is rated on our chart and it does skew left.

But it really has a character of a lot of our slightly left-leaning mainstream media if you were.

- So you have AP and UPI in the neutral or balanced bias category.

That was no surprise certainly to me.

But also there, this did surprise me.

You have Stars & Stripes and Military Times, right there in the middle, meaning they're not partial, not leaning strongly left or right.

Talk about that.

How do they wind up there?

- Well you know-- - [Jim] And we got about a minute left, Vanessa.

- Sure, so the middle is, we're trying to capture what the middle is in the United States.

Not what we'd like it to be or what it should be.

The middle is not necessarily morally right or correct by any means and it shifts over time, right.

So the left/right spectrum is anchored by contemporary, US political positions as expressed by our own elected officials.

And so that's straight fact reporting about things that are balanced or minimally biased.

That's the kind of thing that will land somebody like Military Times and Stars & Stripes in the middle.

- In about the 20 seconds we've got left, do you see any migration from those bottom dwellers up?

I mean we talked about the left/right migration.

You know, the folks who are in that less than accurate, less than credible category, do they ever percolate up towards more respectable news sources?

- Usually not.

I mean, junk food exists for a reason.

Some people like it.

(hosts laughing) - Well that's where we're gonna have to leave it.

Vanessa Otero, the company is Ad Fontes Media, and it's The Media Bias Chart.

Thank you so much for bein' with us.

That's all the time we have this week.

But if you want to know more about Story in the Public Square, you can find us on Facebook and Twitter.

Or visit pellcenter.org where you can always catch up on previous episodes.

For G. Wayne Miller, I'm Jim Ludes asking you to join us again next time for more Story in the Public Square.

(Story in the Public Square theme music) (upbeat guitar music)