How to Evaluate Any News Source With a Twelve-Point Checklist

Somewhere out there is a person who believes every word printed by one particular outlet, and another person who believes every word printed by its rival, and the two of them are equally easy to fool. The first has outsourced his judgment to a masthead. The second has outsourced it to a grudge. Neither has noticed that the thing they are reading was written by a human being on a deadline with a boss.

Conservatives have more reason than most to distrust the press, and I do not intend to talk you out of it. But distrust is a feeling, and a feeling is a poor filter. What you want is a method: the same questions, asked of every source, every time, regardless of whose side it claims to be on. A reader who tests an outlet this way is hard to fool, and he is also hard to insult, because the checklist treats his own team's sources the same way it treats the other team's.

Twelve checks follow, in four groups. Each one comes with the reason it matters. You do not need to run all twelve on every article. Run the first three on everything, and run the rest when a story is important, surprising or about to make you angry. (That last one is the most important rule. The stories engineered to make you furious are the ones to check first.)

Group one: who is talking

1. Who owns it and who pays for it?

Every outlet has a financial base: subscribers, advertisers, a parent company, a wealthy owner, a foundation, a political organization or the government. None of these makes a source dishonest. All of them shape what gets covered and how loudly. A paper owned by a family that sells the local car dealership's ads has a reason to go soft on the dealership. A nonprofit funded by a single donor has a reason to think about that donor. A broadcaster owned by a giant company has reasons connected to its parent's other businesses.

Where to look: the site's "About" page; a public company's filings with the Securities and Exchange Commission; and for a broadcast station, the ownership reports that stations file with the Federal Communications Commission. If an outlet hides who runs it, treat that as information.

2. Does it separate reporting from opinion?

A news story tells you what happened. A column or editorial tells you what the writer thinks about it. Good outlets label the two clearly and keep them apart. The trouble starts when opinion arrives dressed as news: loaded adjectives in the first paragraph, a verdict where a fact should be, "critics say" with no named critic.

Test: read the first three paragraphs and underline every word that tells you how to feel. If there are more of those than facts, you are reading opinion, whatever the label says. This works in both directions, and it applies to the friendly outlets too.

3. Who wrote it, and what is their record?

A byline is a minimum standard. Is there a name? Does the person cover this beat regularly, or are they a generalist writing about a field they do not know? Is the author a reporter, a commentator or an advocate? There is nothing wrong with advocates, but you should know when you are reading one.

Group two: what it says it knows

4. Are the sources named, or anonymous?

Named sources can be checked and can be held to account. Anonymous sources are sometimes necessary, since a person who could lose a job by speaking may only speak on condition of anonymity, and the reporting that exposed some real scandals depended on it. But "a person familiar with the matter" is a weaker foundation than a name. Ask three things. Does the outlet say why the source was granted anonymity? Is there more than one such source, independent of each other? Does the story contain anything that someone could check without relying on the unnamed person?

A story built entirely on unnamed sources, with no documents and no on-the-record confirmation, should be read as a claim, not a fact.

5. Does it link or name the primary document?

The primary source is the thing itself: the bill, the court opinion, the agency report, the transcript, the budget table, the press release from the company that is being described. A good story points you to it. A weak one paraphrases it and leaves you to trust the paraphrase.

You can often find the original in under five minutes. Bills and their text are on Congress.gov. Court opinions are on the courts' own websites. Federal spending and economic data come from agencies such as the Bureau of Labor Statistics and the Census Bureau. When a story says "a new law bans X," read the law. You will discover how often "bans" means "requires a form" and how often "slashes" means "increases more slowly than planned." That discovery is worth the five minutes.

6. Does the quote match the context?

Quotes are the easiest thing in journalism to bend. A sentence pulled from the middle of a speech can mean the opposite of the speech. Look for the full quote, and if the story is important, find the video or transcript. Be suspicious of a quote that is a fragment, of one that has no date or place, and of one repeated across many outlets with no original source. Ask: if I heard the whole conversation, would I agree with how it was described?

Group three: how it behaves

7. How does it handle corrections?

Every outlet makes mistakes, and the honest ones say so. Look for a corrections page or a note at the bottom of an article. Check whether corrections are visible or buried, and whether the fix appears in the same place the error did. An outlet that never corrects anything is not an outlet that never errs. It is one that does not tell you.

Notice, too, what it does after it is caught. A good newsroom explains what it got wrong and how. A weak one quietly edits the text and moves on, so the error lives on in the thousand copies already shared.

8. Does the headline match the body?

Headlines are usually written by editors, not by the reporter, and they are written to get clicks. The reporter may have carefully hedged the story, and the headline then promised something the story never says. Read past the headline. If the headline says "Senator Admits" and the article says the senator said something entirely different, you have found a headline problem and not a news story.

A quick test: cover the headline and write your own after reading the body. If yours is much milder, the headline was selling.

9. Does it show you the other side's best case, or only its worst?

Every outlet has a lean, and the useful question is not whether it has one but whether it is honest about the other side. Does the story quote the opposing view in words that the other side would recognize as fair? Or does it quote the least articulate person it could find and let him represent a million people?

This is the test I would run first on the outlets you love, because it is the one that reveals whether you are being informed or flattered.

Group four: what to do with a claim

10. Is it built on a single study, poll or number?

"A new study shows..." should make you slow down, not speed up. One study is a data point. Ask who funded it, how many people or cases it covered, whether it has been replicated, and whether the headline number is the finding or just the most dramatic corner of it. Ask whether the reporter gave you the sample size and the margin of error. If the story hands you a figure with no denominator, no time frame and no comparison, you cannot judge it. I give a whole set of tests for this in How to Spot a Misleading Statistic Before It Spots You, With Worked Examples.

11. Can a second, independent outlet confirm it?

Here is the cheapest and strongest check on the list. Take a story that matters and search for the same event in at least two other places, ideally from different sides. Here is the method.

1. Read the original.
2. Find an outlet with a different lean, and another with a different business model, such as a wire service that sells to many clients, which has less reason to slant.
3. Mark what they all agree on: the dates, the names, the quotes, the documents. Those are probably the facts.
4. Mark what differs: the adjectives, the emphasis, what was left out. That is the framing.
5. Ask what you would need to see to be convinced, and then go look for it.

When three outlets with three different outlooks agree on a fact, you can trust it a great deal more. When only the outlets that benefit from a claim are reporting it, you are in a closed loop.

12. How do I feel, and is the feeling the product?

Last on the list, and sometimes the first thing to check. If a story leaves you angry, ask whether it gave you anything to do with the anger, such as a bill number, a meeting date, a person to call or a document to read. Or was the point of the piece the anger itself? An outlet that profits from your outrage has every incentive to keep you outraged, and if a story's entire purpose is to make you feel contempt for a group of people, it is selling you something.

A worked example, with invented details

Here is the checklist applied to a hypothetical story, to show how fast it can go. Suppose a headline reads, "New Report: Local Schools Failing Half Their Students." This is invented, so do not look for it.

• Check 8 (headline vs. body): The body says half of students in one grade in one subject scored below a proficiency line the state set. That is narrower than "failing half their students."
• Check 5 (primary document): The story links nothing. A search of the state's education department site turns up the table. It shows the figure for that grade, and lists a different, higher figure for the next grade.
• Check 10 (single study): The "report" is from an advocacy group's survey. The story does not say who paid for it.
• Check 11 (second outlet): A local paper and a wire story use the state table and describe the same data with different framing.

Result: the claim is not false, but it was oversold. You can still argue that the school district needs reform. You just argue it from the actual numbers, which is a stronger position and one nobody can knock out with a fact-check.

The one-line version to keep by your keyboard

If you want the twelve checks on a sticky note, here they are in the order you will use them.

1. Who owns it and pays for it?
2. Is this news or opinion?
3. Who wrote it?
4. Are the sources named?
5. Where is the primary document?
6. What did the full quote say?
7. Does it correct its mistakes in the open?
8. Does the headline match the body?
9. Does it state the other side fairly?
10. Is it resting on one study or number?
11. Does an independent outlet confirm it?
12. Is the point to inform me, or to inflame me?

Three or more bad answers on a story that matters means you should hold your opinion until you have read something else. That is not cynicism. It is the ordinary caution you would use before signing a contract somebody handed you at a county fair.

Using the checklist on your own side

The test only works if you apply it to the sources you like. The conservative press has a long and honorable history, and I tell some of it in From a Small Magazine in 1955 to a Movement Press That Reshaped the Debate. Partisan newspapers are as old as the Republic, and the Founders' papers were nastier than any feed you read today, as The Founders' Partisan Press Was Far Nastier Than Anything on Your Feed shows. The point is not that the press is bad. The point is that every outlet is a human institution with incentives, and the reader's job is to know them. For the argument over why many on the right distrust the established media, the background is in Cable News Timeline From the First 24-Hour Channel to the Three-Network Race and in The Fairness Doctrine From 1949 to 1987 and Why Its Repeal Still Matters.

One more thing. When you share an article, you become a publisher of a sort, with a small audience that trusts you. Spend ten seconds on checks 2, 5 and 8 before you hit the button. If every reader did that, half of what is wrong with the information diet of this country would clear up by dinner.

The bottom line

• Ask the same questions of every source. Ownership, labelling, byline, sourcing, primary documents, corrections, headline, single studies and independent confirmation work on any outlet from any side.
• Click through to the primary document. If a story describes a law, a ruling or a report, find the real text on Congress.gov, a court's site or an agency's page before you repeat it.
• Compare three outlets on one story. What they agree on is probably fact. What they disagree on is probably framing.
• Check hardest the stories that make you angriest. Outrage is the product, and the cure is a five-minute check.
• Apply it to your own side first. A habit that only catches the other team's errors is not a method. It is a mood.