Home » FactCheck.org

Tag: FactCheck.org

PolitiFact at 15: Lessons about innovation, the Truth-O-Meter and pirates

PolitiFact at 15: Lessons about innovation, the Truth-O-Meter and pirates

Published August 22, 2022 by stepht@duke.edu

Fifteen years ago, I worked with a small group of reporters and editors at the St. Petersburg Times (now the Tampa Bay Times) to start something bold: a fact-checking website that called out politicians for being liars.

That concept was too gutsy for the Times political editor. Sure, he liked the idea, he said at a meeting of editors, “but I want nothing to do with it.”

That was my first lesson that PolitiFact was going to disrupt the status quo, especially for political journalists. Back then, most of them were timid about calling out lies by politicians. They were afraid fact-checking would displease the elected officials they covered. I understood his reluctance because I had been a political reporter for many years. But after watching the lying grow in the early days of the internet, I felt it was time for us to change our approach.

Today, some political reporters have developed more courage, but many still won’t call out the falsehoods they hear. PolitiFact does. So I’m proud it’s going strong.

It’s now owned by the Poynter Institute, and it has evolved with the times. As a proud parent, allow me to brag: PolitiFact has published more than 22,000 fact-checks, won a Pulitzer Prize and sparked a global movement for political fact-checking. Pretty good for a journalism org that’s not even old enough to drive.

On PolitiFact’s 15th birthday, I thought it would be useful to share the lesson about disruption and a few others from my unusual journey through American political journalism. Among them:

Gimmicks are good

The Truth-O-Meter — loved by many and loathed by some — is at times derided as a mere gimmick. I used to bristle at that word. Now I’m fine with it.

My friend Brooks Jackson, the co-founder of FactCheck.org, often teased me about the meter. That teasing culminated in a farewell essay that criticized “inflexible rating systems” like our meter because they were too subjective.

I agree with Brooks to an extent. Summarizing a complex fact-check to a rating such as Half True is subjective. But it’s a tremendous service to readers who may not want to read a 1,000-word fact-check article. What’s more, while it relies on the judgment of the journalists, it’s not as subjective as some people think. Each fact-check is thoroughly researched and documented, and PolitiFact has a detailed methodology for its ratings.

Yes, the Truth-O-Meter is a gimmick! (I once got recognized in an airport by a lady who had seen me on TV and said, “You’re the Truth-O-Meter guy!”) But its ratings are the product of PolitiFact’s thorough and transparent journalism. It’s a gimmick with substance.

Empower the pirates

I was the founding editor, the guy with the initial ideas and some terrible sketches (my first design had an ugly rendering of the meter with “Kinda True” scribbled above). But the editors at the Tampa Bay Times believed in the idea enough to assign other staffers who had actual talent, including a spirited data journalist named Matt Waite and a marvelous designer named Martin Frobisher.

Times Executive Editor Neil Brown, now president of Poynter, gave us freedom. He cut me loose from my duties as Washington bureau chief so I could write sample fact-checks. Waite and Frobisher were allowed to build a website outside the infrastructure of the Times website so we had a fresh look and more flexibility to grow.

We were like a band of pirates, empowered to be creative. We were free of the gravitational pull of the Times, and not bound by its rules and conventions. That gave us a powerful spirit that infused everything we did.

Design is as important as content

We created PolitiFact at a time when political journalism, even on the web, was just words or pictures. But we spent as much energy on the design as on the journalism.

The PolitiFact home page from August 2007 had a simple design. Source: Wayback Machine – Internet Archive

It was 2007 and we were an American newspaper, so our team didn’t have a lot of experience or resources. But we realized that we could use the design to help explain our unique journalism. The main section of our homepage had a simple look — the face of the politician being checked (in the style of a campaign button), the statement the politician made and the Truth-O-Meter showing the rating they earned. We also created report cards so readers could see tallies that revealed how many True, Half True, False ratings, or whatever a politician had earned.

The design not only guided readers to our fact-check articles, it told the story as much as the words.

Twitter is not real life

My occasional bad days as editor always seemed more miserable because of Twitter. If we made an error or just got attacked by a partisan group, it showed up first and worst on Twitter.

I stewed over that. Twitter made it seem like the whole world hated us. The platform doesn’t foster a lot of nuance. You’re loved or hated. I got so caught up in it that when I left the office to go to lunch, I’d look around and have irrational thoughts about whether everyone had been reading the tweets and thought I was an idiot.

But then when I went out with friends or talked with my family, I realized that real people don’t use Twitter. It’s largely a platform for journalists and the most passionate (read: angry) political operatives. My friends and family never saw the attacks on us, nor would they care if they did.

So when the talk on Twitter turned nasty (which was often), I would remind our staff: Twitter is not real life.

People hate referees

My initial sketch of the website was called “The Campaign Referee” because I thought it was a good metaphor for our work: We were calling the fouls in a rough and tumble sport. But Times editors vetoed that name… and I soon saw why.

People hate referees! On many days, it seemed PolitiFact made everyone mad!

Bill Adair’s original sketch of “The Campaign Referee”

That phenomenon became clearer in 2013 when I stepped down as editor and came to Duke as a journalism professor. I became a Duke basketball fan and quickly noticed the shoddy work of the referees in the Atlantic Coast Conference. THEY ARE SO UNFAIR! Their calls always favor the University of North Carolina! What’s the deal? Did all the refs attend UNC.

Seek inspiration in unlikely places

When we expanded PolitiFact to the states (PolitiFact Wisconsin, PolitiFact Florida, etc.), our model was similar to fast-food franchises. We licensed our brand to local newspapers and TV and radio stations and let them do their own fact-checks using our Truth-O-Meter.

That was risky. We were allowing other news organizations to use our name and methods. If they did shoddy work, it would damage our brand. But how could we protect ourselves?

I got inspiration from McDonald’s and Subway. I assigned one of our interns to write a report about how those companies ensured quality as they franchised. The answers: training sessions, manuals that clearly described how to consistently make the Big Macs and sandwiches, and quality control inspectors.

We followed each recommendation. I conducted detailed training sessions for the new fact-checkers in each town and then checked the quality by taking part in the editing and ratings for several weeks.

I gave every fact-checker “The Truth-O-Meter Owner’s Manual,” a detailed guide to our journalism that reflected our lighthearted spirit (It began: “Congratulations on your purchase of a Truth-O-Meter! If operated and maintained properly, your Truth-O-Meter will give you years of enjoyment! But be careful because incorrect operation can cause an unsafe situation.”)

Adjust to complaints and dump the duds

We made adjustments. We had envisioned Pants on Fire as a joke rating (the first one was on a Joe Biden claim that President Bush was brain-dead), but readers liked the rating so much that we decided to use it on all claims that were ridiculously false. (There were a lot!)

In the meantime, though, we lost enthusiasm for the animated GIF for Pants on Fire. The burning Truth-O-Meter was amusing the first few times you saw it, but then … it was too much. Pants on Fire is now a static image.

As good as our design was, one section on the home page called the Attack File was too confusing. It showed the person making the attack as well as the individual being attacked. But readers didn’t grasp what we were doing. We 86’d the Attack File.

Initially, the rating between Half True and False was called Barely True, but many people didn’t understand it – and the National Republican Congressional Committee once distorted it. When the NRCC earned a Barely True, the group boasted in a news release, “POLITIFACT OHIO SAYS TRUE.”

Um, no. We changed the rating to Mostly False. We also rated the NRCC’s news release. This time: Pants on Fire!

Comments closed

At Global Fact 4: churros, courage and the need to expose propagandists

At Global Fact 4: churros, courage and the need to expose propagandists

Published July 6, 2017 by stepht@duke.edu

My opening remarks at Global Fact 4, the fourth annual meeting of the world’s fact-checkers, organized by the International Fact-Checking Network and the Reporters’ Lab, held July 5-7, 2017 in Madrid, Spain.

It’s wonderful to be here in Madrid. I’ve been enjoying the city the last two days, which has made me think of a giant warehouse store we have in the United States called Costco.

Costco where you go when you want to buy 10 pounds of American Cheese or a 6-pound tub of potato salad. Costco also makes a delicious fried pastry called a “churro.” And because everything in Costco is big, the churros are about three feet long.

When I got to Madrid I was really glad to see that you have churros here, too! It’s wonderful to see that Costco is spreading its great cuisine around the world!

I’m pleased to be here with my colleagues from the Duke Reporters’ Lab — Mark Stencel, Rebecca Iannucci and Riley Griffin. We also have our Share the Facts team here – Chris Guess and Erica Ryan. We’ll be sampling the churros throughout the week!

It’s been an amazing year for fact-checking. In the U.K., Full Fact and Channel 4 mobilized for Brexit and last month’s parliamentary elections. In France, the First Draft coalition showed the power of collaborations during the elections there. In the United States, the new president and his administration drove record traffic to sites such as FactCheck.org and PolitiFact and the Washington Post Fact Checker — and that has continued since the election, a time when sites typically have lower traffic. The impeachments and political scandals in Brazil and South Korea also meant big audiences for fact-checkers in those countries. And we expect the upcoming elections in Germany, Norway and elsewhere will generate many opportunities for fact-checkers in those countries as well, just as we’ve seen in Turkey and Iran. The popular demand for fact-checking has never been stronger.

Fact-checking is now so well known that it is part of pop culture. Comedians cite our work to give their jokes credibility. On Saturday Night Live last fall, Australian actress Margot Robbie “fact-checked” her opening monologue when she was the guest host.

Some news organizations not only have their own dedicated fact-checking teams, they’re also incorporating fact-checks in their news stories, calling out falsehoods at the moment they are uttered. This is a marvelous development because it helps to debunk falsehoods before they can take root.

We’ve also seen tremendous progress in automation to spread fact-checking to new audiences. There are promising projects underway at Full Fact in Britain and at the University of Texas in Arlington and in our own lab at Duke, among many others. We’ll be talking a lot about these projects this week.

Perhaps the most important development in the past year is one that we started at last year’s Global Fact conference in Buenos Aires – the Code of Principles. We came up with some excellent principles that set standards for transparency and non-partisan work. As Alexios noted, Facebook is using the code to determine which organizations qualify to debunk fake news. I hope your site will abide by the code and become a signatory.

At Duke, Mark just finished our annual summer count of fact-checking. Mark and Alexios like to tease me that I can’t stop repeating this mantra: “Fact-checking keeps growing.”

But it’s become my mantra because it’s true: When we held our first Global Fact meeting in 2014 in London, our Reporters’ Lab database listed 48 fact-checking sites around the world. Our latest count shows 126 active projects in 49 countries.

I’m thrilled to see fact-checking sprouting in countries such as South Korea and Germany and Brazil. And I continue to be amazed at the courage of our colleagues who check claims in Turkey and Iran, which are not very welcoming to our unique kind of journalism.

As our movement grows, we face new challenges. Now that our work is so well-known and an established form of journalism, governments and political actors are calling themselves fact-checkers, using our approach to produce propaganda. We need to speak out against this and make sure people know that government propagandists are not fact-checkers.

We also need to work harder to reach audiences that have been reluctant to accept our work. At Duke we published a study that showed a stark partisan divide in the United States. We found liberal publications loved fact-checking and often cited it; conservative sites criticized it and often belittled it. We need to focus on this problem and find new ways to reach reluctant audiences.

I’m confident we can accomplish these things. Individually and together we’ve overcome great hurdles in the past few years. I look forward to a productive meeting and a great year. And I’m confident:

Fact-checking will keep growing.

Comments closed

Heroes or hacks: The partisan divide over fact-checking

Heroes or hacks: The partisan divide over fact-checking

Published June 7, 2017 by stepht@duke.edu

Conservative writers aren’t fond of fact-checking. They belittle it and complain that it’s biased. They say it’s “left-leaning” and use sarcastic quotes (“fact-checking”) to suggest it’s not legitimate. One writer likens PolitiFact to a Bangkok prostitute.

Liberal writers admire fact-checking. They cite it favorably and use positive adjectives such as “independent” and “nonpartisan.” They refer to fact-checkers as “watchdogs” and “heroes.”

To examine partisan differences over fact-checking, we analyzed some of the most widely read conservative and liberal sites. Our students in the Duke Reporters’ Lab identified 792 statements that referred to fact-checkers or their work. We found a stark partisan divide in the tone, the type of references and even the adjectives the writers used.

Our report, Heroes or hacks: The partisan divide over fact-checking, reveals a serious problem for the growing number of fact-checkers, journalists who research and rate the accuracy of political statements. They emphasize their neutrality and nonpartisan approach, but they face relentless criticism from the political right that says they are biased and incompetent.

Our analysis found:

Liberal websites were far more likely to cite fact-checks to make their points than conservative sites were.
Conservative sites were much more likely to criticize fact-checks and to allege partisan bias.
When our student researchers categorized the tone of mentions, we found liberal sites made most of the positive references, while the negative references came primarily from the right.
Conservative sites made the most critical comments about fact-checking, occasionally using quotation marks (“fact-checking”) to imply it wasn’t legitimate.

(Read the full report.)

We found the most revealing differences in the words the writers used to describe fact-checkers and their work.

Liberals emphasize they are nonpartisan and call them “respected,” “reputable” and “independent.” Fact-checkers are “watchdogs” or “heroes.” PolitiFact is described as “Pulitzer Prize-winning.”

Conservatives use words such as “left-leaning,” “biased,” “hackiest” and “serial-lying.” They question the legitimacy of fact-checkers by calling them “self-proclaimed.”

The most wicked criticism came from Jonah Goldberg of the National Review, who called PolitiFact “the hackiest and most biased of the fact-checking outfits, which bends over like a Bangkok hooker to defend Democrats.”

Our findings indicate that fact-checkers have some work to do. They need to strengthen their outreach to conservative journalists and, particularly, to conservative audiences. The fact-checkers need to understand the reasons for the partisan divide and find ways to broaden the acceptance of their work.

Comments closed

Fact-checking comes to the Amazon Echo

Fact-checking comes to the Amazon Echo

Published October 21, 2016 by stepht@duke.edu

The Duke Reporters’ Lab has created a new fact-checking app for the Amazon Echo.

The app is a spin-off of Share the Facts, a project that has expanded the reach of fact-checking. The launch partners are PolitiFact, The Washington Post’s Fact Checker and FactCheck.org.

With the new Share the Facts skill, owners of the Echo and other Alexa-enabled devices, including the Tap and the Dot, can “ask the fact-checkers” about claims they hear from presidential candidates Hillary Clinton and Donald Trump, as well as other candidates and politicians who have been checked.

Share the Facts is now available in the Skills section of the Alexa app. (To find it, open the Alexa app on your smartphone, click on the left navigation panel, and then select “Skills.” From there, you can search for “Share the Facts” and select “Enable Skill.”)

We encourage you to try checking candidates’ claims from your couch after watching a campaign ad or during a discussion around the dinner table.

To begin a query, say: “Alexa, ask the fact-checkers.” (If you’re using the Tap, you’ll need to press the microphone button first and then say, “Ask the fact-checkers.”)

We have found it often works best if you wait for Alexa to reply, “Welcome to Share the Facts. We consolidate fact-checks from some of the most respected journalists in the U.S. Ask me to check a fact you’re wondering about” — and then ask your question, such as:

“Did Donald Trump oppose the war in Iraq?”
“Was Hillary Clinton right that her email practices were allowed?”
“Is it true that 300,000 Floridians have lost their health insurance because of Obamacare?”

Try to use the most important keywords in your question, following the examples above.

Share the Facts uses natural speech recognition to analyze and answer your questions from our database of roughly 2,000 professionally curated fact-checks. We scale our results so that they are timely and have the most consensus among our partners.

We welcome your feedback on our new Echo skill Share the Facts. Please send your thoughts to Share the Facts project manager Erica Ryan.

Comments closed

5 share-worthy fact-checks of Clinton’s acceptance speech

5 share-worthy fact-checks of Clinton’s acceptance speech

Published July 29, 2016 by stepht@duke.edu

As Hillary Clinton became the first female presidential candidate to accept the nomination of a major political party on Thursday night, fact-checkers dug into the talking points and attack lines that peppered her speech.

PolitiFact, The Washington Post and FactCheck.org were among those sorting the truth from the fiction. Here’s a roundup of five of their fact-checks that you can share on Facebook and Twitter using the Share the Facts widget, created by the Duke Reporters’ Lab and Jigsaw, a technology incubator within Alphabet, the parent company of Google. You can also embed them in articles and blog posts.

1. “Don’t believe anyone who says: ‘I alone can fix it.’ Those were actually Donald Trump’s words in Cleveland.”

Clinton used this line to contrast her style with that of her opponent, Republican Donald Trump. But FactCheck.org found it’s not so cut-and-dried: “In fact, Trump said that as a political outsider only he can fix a ‘rigged’ system. He has spoken about working with others many times, including in that same speech.”