Research 101

Evidence supports a claim only through a link someone can inspect

Toulmin's model breaks an argument into a claim, its grounds (the evidence), and a warrant, which is the assumption connecting them. Warrants usually go unstated, and that's where weak arguments hide. A study of 40 students can be solid grounds, but "they represent all students" is a separate claim that needs its own support. On Critically, the warrant goes in a citation's "why." Write it and readers can check it. Leave it out and they have to guess.

A claim is checkable only if some evidence could prove it wrong

Popper proposed that a scientific theory must be falsifiable: it must be possible, in principle, to show it false. The same test works for any note. "Coffee is overrated" survives every observation, so no source can support or refute it. "Most adults in this survey reported drinking coffee daily" names something a source either shows or doesn't. Before writing a note, ask what finding would make you take it down. If nothing would, you've written an opinion.

A percentage change can't be checked without its baseline

"Doubles the risk" can mean 1 in 10,000 becoming 2 in 10,000, or 10 in 100 becoming 20. Relative figures sound large and say nothing about scale until you know the starting rate. Gigerenzer and colleagues found that this framing misleads patients, journalists and physicians, and they recommend stating absolute risks. In their example, a 25% drop in breast cancer deaths from mammography meant about 1 fewer death per 1,000 women. When a note uses a percentage change, give the base rate or cite a source that does.

Two things moving together doesn't mean one causes the other

Many sources report an association. Calling that association a cause is a separate step, and the source may not have taken it. In 1965 Bradford Hill listed questions to ask before treating an association as causal. They include how strong it is, whether it repeats across different studies, and which came first. One of his examples was diet: does a diet cause a disease, or do people with the disease choose that diet? When a note says "causes", check whether its source says so or only reports that the two go together.

A paper's conclusion can claim more than its results show

Boutron and colleagues looked at 72 trials published in December 2006 whose primary outcome showed no statistically significant effect. Half had conclusions that still played up a benefit or steered readers away from the null result, and 18% had this spin in the title. In a later experiment led by Boutron, clinicians who read an abstract rewritten with spin rated the treatment as more beneficial. An abstract or conclusion is the author's claim about the results. When you can, cite the results.

A cited source often doesn't support what it's cited for

Citing a source is not the same as the source backing you. A 2015 meta-analysis of 28 studies estimated that about 25% of quotations in medical journal articles contained an error, and about 12% were major errors. A 2025 update covering 46 studies found no improvement over time. These are peer-reviewed papers written by specialists. Before you cite something, open it and find the passage that supports your claim. Then say what that passage shows in the citation's "why".

A high search ranking is not evidence of reliability

In the same Stanford study, historians and students who did leave a page often searched poorly. They clicked the top results without realizing that search engine optimization shapes the ranking, and they skipped quotation marks when searching for exact phrases. The fact checkers showed what the researchers called click restraint: they read the results page first and then chose where to go. Read a results list before you work through it.

Fact checkers judge a site by leaving it

Wineburg and McGrew gave 10 historians, 10 professional fact checkers and 25 Stanford undergraduates the same websites to evaluate. The fact checkers were the fastest and the most accurate. They skimmed a page, then opened new tabs to see what other sources said about it. The historians and students read vertically, staying on the page, and were more often swayed by logos and academic-looking citations. The historians' skill at evaluating documents did not carry over to the open web. Knowing where to look did.

Lateral reading can be learned in a short course

In a 2021 study, an online college nutrition course added short modules on fact checkers' strategies. At pretest, 3 of 87 students left a website to check it against another source. After the modules, students read laterally far more often. Lateral reading is a habit that can be taught. Before reading an unfamiliar source closely, search for who is behind it and what others say about it.

A firsthand account is still a reconstruction

Loftus and Palmer showed people films of car accidents, then asked how fast the cars were going. People asked about cars that "smashed" gave higher speed estimates than people asked about cars that "hit". A week later, the "smashed" group was more likely to report broken glass that was never in the film. Eyewitnesses are close to the event, but questions asked afterwards can reshape what they remember. Treat testimony as evidence of what someone recalls, and look for records that don't depend on memory.

Whether a source is primary depends on your question

A primary source is firsthand evidence from the time or subject under study. A secondary source interprets that evidence. The label describes a relationship to a question, not the document itself. An article written in 2020 about the 1918 influenza is secondary if you're studying the pandemic, and primary if you're studying how people wrote about it in 2020. Before calling a source primary in a note, say which question it's primary for.

Check one primary source against others

Being close to an event doesn't make a source complete. The Library of Congress notes that primary sources are often incomplete and short on context. It tells readers to find other sources that support or contradict them. Wineburg's 1991 study of historians named this habit corroboration, meaning comparing documents with one another. He described it alongside sourcing (who made the document and why) and contextualization (when and where it was made). When a note rests on a primary source, cite what you checked it against.

Web pages disappear at a measurable rate

Pew Research Center checked a sample of web pages collected between 2013 and 2023. By October 2023, 25% were no longer accessible, including 38% of the pages from 2013. Scholarly references fare no better. A 2014 study found that one in five science, technology and medicine articles cited web resources that could no longer be revisited. A link points to a place. It is not a copy of what was there.

Archive a source when you cite it, not later

An archive can only preserve what someone captured in time. The Hiberlink project found that about 75% of links in scholarly articles weren't archived anywhere within two weeks of the article's publication. The Internet Archive later checked the 2013 pages that Pew found dead. The Wayback Machine had copies of roughly 38% of them, so most of what was lost can't be recovered. That's why a citation on Critically is captured the moment it's made.

A link that still works may not show what was cited

Link rot is the visible failure. Content drift is the quiet one: the address still loads, but the page has changed. A 2016 study compared archived snapshots with the live pages. For over 75% of web references in scholarly articles, the content had drifted from what it was when cited. A 2014 study found that more than 70% of URLs in several law journals, and 50% in U.S. Supreme Court opinions, no longer led to the information originally cited.

Using self-control makes the next act of self-control harder

This note is here because its sources disagree. The claim, known as ego depletion, holds that self-control draws on a limited resource. A 2010 meta-analysis of 83 studies found a significant effect. In 2016, 23 laboratories ran the same preregistered protocol with 2,141 participants and found an effect close to zero (d = 0.04, with a confidence interval that included zero). Both are serious work. They differ in what they measured: one pooled published studies, the other fixed its method in advance and ran it everywhere. Neither settles the question alone, which is why the refuting citation sits in the same list as the supporting one.

A failed replication tests one method, not every version of an idea

The 2016 replication used a single standardized task, based on the procedure from one earlier study. Across 23 labs the overall effect was near zero, although two labs found a significant effect and one found a significant reverse effect. Baumeister and Vohs, who originated the theory, published a response calling the project misguided. A null result shows that this method, run this way, didn't produce the effect. Write the note at that level: name the method, not just the conclusion.

A meta-analysis of published studies can overstate an effect

Pooling many studies only helps if the studies pooled are representative. Carter and McCullough reanalyzed the 2010 ego-depletion meta-analysis. They found strong signs of publication bias and small-study effects, which suggest that studies finding the effect were more likely to be published. The original authors replied that this kind of bias affects other meta-analyses too, and that the reanalysis didn't identify where it came from. When you cite a meta-analysis, check whether it included unpublished data.

How you phrase a search shapes the answer you find

White studied yes-or-no health questions using a survey, labeled search results and a large log of real searches. Searchers favored positive answers more than physicians' consensus answers would justify. The search engine also favored one perspective, usually the positive one, regardless of the truth. About half the answers searchers settled on were wrong. Search for the claim and for its opposite, and read both result pages before you decide.

People tend to seek evidence that fits what they already think

Nickerson's 1998 review defines confirmation bias as looking for or interpreting evidence in ways that favor existing beliefs or the hypothesis you started with. He documents it across many settings. An early demonstration came from Wason in 1960. People trying to discover a hidden rule mostly tested cases they expected to fit, rather than cases that could prove their guess wrong. On Critically, this shows up as a note with many supporting citations and an empty Refutes tab, because nobody looked.

Many agreeing sources can trace back to a few

Greenberg mapped every paper making one claim in muscle disease research: 242 papers and 675 citations. Support for the claim grew through three distortions. Papers that weakened the claim were under-cited. Papers cited it without adding any data of their own. And hypotheses turned into facts through citation alone. Counting sources that agree isn't the same as counting evidence. Follow the citations back to the papers that hold the data.

State the claim fairly before you refute it

The Rogerian method asks a writer to describe the opposing view fairly and accurately before giving their own, and to say when that view holds. A refutation that misstates the note it targets refutes nothing. On a refuting citation, start the "why" by stating what the note claims in terms its author would accept. The method has limits: it assumes good faith, and it is less common in academic writing, which puts evidence ahead of compromise.

A refutation works better when it says what's true instead

People keep relying on misinformation after it has been retracted, which Lewandowsky and colleagues call the continued influence effect. Their review found that an alternative explanation helps, because it fills the gap the retraction leaves. The 2020 Debunking Handbook, written by 22 researchers, recommends leading with the fact and then explaining how the false claim misleads. When you add a refuting citation, use its "why" to say what the source shows is true, not only that the note is wrong.

Answer the claim, not the person who made it

Graham ranks forms of disagreement from lowest to highest. At the bottom are name-calling, attacks on the author, responses to tone and bare contradiction. Above them come counterargument, refutation, and finally refuting the central point. The higher forms engage with what was actually said. A citation in the Refutes tab should do the same: point to the part of the note you dispute and show the evidence against it.

Some duplication is deliberate

Dividing the work doesn't mean every item is read only once. Cochrane recommends that at least two people extract key data independently. It cites a study in which independent double extraction produced fewer errors than one person extracting and a second checking. Decide in advance which items get read twice, starting with the claims the notebook depends on most. A 2017 methods review notes that the evidence behind these standards is weak, even though extraction errors are common.

Agree on the question and the method before splitting the work

In health research, duplicated reviews were common enough that a registry was built to prevent them. PROSPERO, launched in 2011, lets teams register a systematic review's protocol before they start. Others can see what's underway, and the finished review can be compared with its plan. For a group notebook, the equivalent is to write the question in the notebook's description, then give each section one angle and one owner before anyone starts citing.

People put in less effort when their own contribution can't be seen

Social loafing is the drop in individual effort that happens when people work as a group. Research has found that it shrinks when each person's output can be identified. It isn't limited to physical tasks. One study found loafing on a thought-provoking task where each person could make a unique contribution. On Critically, every note and citation carries its author's name, so each person's share of a shared notebook is visible.