Reading the Methods Section Like a Peer Reviewer

A peer reviewer decides whether to believe a result by reading the methods section first, before the abstract's conclusion sets expectations. The methods tell you what kind of study it was, what the authors committed to measuring before they saw the data, who was actually enrolled, what the treatment was compared against, and how each outcome was defined and captured. If those five things hold together, the result has a fair chance of being real. If any one of them is vague or arrived at after the fact, the conclusion sits on softer ground than it looks. This piece is educational and not medical advice, so use it to read more sharply and then talk with your own clinician about what a study means for you.

I have read methods sections from a few seats. Co-authoring a systematic review and meta-analysis in Diabetes Care meant judging other people's methods closely enough to decide whether their numbers could be pooled at all. Running a randomized controlled trial of our own, EASY-1, meant writing the methods before we collected a single data point. Reviewing grants, including for the Pivotal Philanthropies Action for Women's Health initiative, sharpened the same habit. Decisions made before any result exists usually decide whether the result can be trusted.

What is the methods section actually for?

The methods section tells you how the study was built, so that a stranger could in principle repeat it and check whether the conclusion follows. It is the load-bearing wall of the paper, where the claim is either earned or quietly given away. Read it as if you do not yet know what the study found, then ask what question this design can actually answer. Sometimes the question quietly grows between the methods and the headline, and catching that drift is most of the job.

Does the design match the claim?

Start with the architecture, because it caps how strong any conclusion can be. A randomized trial can support a causal claim. An observational cohort, however large and careful, can describe associations and adjust for what was measured, but it cannot fully account for the differences it never recorded. A case series describes what happened to a group of patients with no comparison at all. None of these is wrong, since they answer different questions, but a common trap is reading a cause-and-effect sentence on top of a design built only to show correlation.

So the first reviewer question is plain. Does the design support the verb in the conclusion? "Improves," "prevents," and "causes" want randomization. "Is associated with" is the honest verb for observational data. When the verb outruns the design, the methods have not done their work, however clean the figures look.

Was the primary outcome pre-specified?

This is the question I weight most heavily, because it best separates a finding that will replicate from one that will not. Pre-specification means the authors named their primary outcome, their main comparison, and their analysis plan before they saw the results, ideally in a public registry with a date attached. A pre-registered primary outcome that succeeds is strong evidence. A result that surfaced only after the data were in hand, then got promoted to the headline, is a hypothesis worth testing again.

The reason is statistical and human at once. If you measure many outcomes and report the one that reached significance, you have found the corner of the data that looked best, not a truth. So reviewers compare the registered primary outcome against the one the abstract celebrates. When the registered primary outcome is missing from the abstract and a former secondary outcome has taken its place, that swap is the most informative thing in the paper. The practical move is to open the registration entry and read its version history, since a primary outcome edited after enrollment began deserves an explanation that good authors provide.

Who was actually enrolled?

A study answers a question only about the people it let in, so the inclusion and exclusion criteria are not fine print, they are the boundary of the claim. Tight criteria buy a clean signal and a narrow population. Broad criteria buy generalizability and a noisier result. The reader has to know which trade was made before deciding whether the finding transfers to the patient in the room.

Read the enrollment numbers as carefully as the criteria. How many were screened, how many randomized, how many finished, and did more drop out of one arm than the other. Differential dropout can manufacture a difference the treatment never produced. Findings drawn from a narrow group can travel further than the data support, which is part of why some of my own work looked at ethnic differences in the relationship between insulin sensitivity and insulin response. Metabolic physiology does not behave identically across populations, so the table describing who was enrolled is often more revealing than the one reporting what happened to them.

What was the comparator, and how were outcomes defined?

A result has no meaning without the thing it was measured against, and the comparator is where a surprising amount of mischief hides. Compared with nothing, almost any attentive intervention looks good, because people in trials get structure and follow-up that ordinary care lacks. Compared with a weak or under-dosed alternative, a treatment can win on the design rather than the merits. In EASY-1 we compared our system against standard of care, because "better than no support" would have been the wrong and far easier bar. Ask whether the comparator was a fair, current version of what the new approach hopes to replace.

Then read how each outcome was defined and captured, because the same word can mean very different things. "Hypoglycemia" can mean a number on a meter, a symptom the patient reported, or an event that needed help from another person. Ask who assessed the outcome and whether they knew which arm the patient was in, since an unblinded assessor judging a subjective endpoint is a known source of drift. A precise definition, measured the same way in both arms by a blinded assessor, is the quiet mark of a study built to find the truth.

Reading like a reviewer, in practice

Put together, the method is almost mechanical, and that is its strength. A paper that answers all five questions without flinching has earned your attention. One that blurs any of them deserves a second look, however confident the final sentence sounds. The field is genuinely hard and most researchers work in good faith, so reading their methods closely is the most respectful thing a reader can do with that work.