Guides / research design
Study designs compared
Cross-sectional, cohort, case-control, experimental and quasi-experimental. Each answers a different question, each supports a different effect measure, and choosing the wrong one is the error no analysis can undo. This guide works through the arithmetic that distinguishes them and shows exactly when an odds ratio stops resembling a risk ratio.
They divide first into experimental and observational. In an experimental design the researcher assigns the exposure, usually at random, which is what permits a causal claim. In an observational design the exposure is whatever the participants already have. Observational designs then divide by timing: cross-sectional measures exposure and outcome at the same moment, cohort studies follow people forward from exposure to outcome, and case-control studies start from the outcome and look back at exposure. Quasi-experimental designs sit between, where an intervention is assigned but not at random.
Start here
The decision that comes first
Design follows the question, not the other way round. Three questions settle almost every case.
A cross-sectional study yields a prevalence ratio. A cohort study yields a risk ratio and a rate. A case-control study yields an odds ratio and nothing else, because the number of people at risk was never observed. Reporting a risk ratio from a case-control study is a category error, not a rounding problem, and it is one of the most commonly flagged faults in submitted work.
No assignment
The observational designs
| Design | Direction | Measures | Strength | Weakness |
|---|---|---|---|---|
| Cross-sectional | Snapshot | Prevalence, prevalence ratio | Fast, cheap, good for burden | Cannot establish temporal order |
| Cohort, prospective | Exposure → outcome | Incidence, risk ratio, rate ratio | Temporal order is clear | Slow, costly, loses people |
| Cohort, retrospective | Exposure → outcome, in records | Same, from existing data | Fast version of the above | Limited to what was recorded |
| Case-control | Outcome → exposure | Odds ratio only | Efficient for rare outcomes | Recall and control selection |
| Case series | Cases only | Description | Signals something new | No comparison group |
| Ecological | Groups, not people | Group-level association | Uses routine data | The ecological fallacy |
Why cross-sectional cannot show causation
Not because of sample size or statistical power. Because of order. If you measure physical activity and depression in the same questionnaire and find them inversely associated, the data is equally consistent with activity protecting against depression, depression reducing activity, or a third factor driving both. The design contains no information about which came first.
This is reverse causation, and it is a design property rather than an analytical one. No regression model repairs it. The only honest options are to report association and say so plainly, or to change the design.
Repeated cross-sectional is not longitudinal
Surveying 500 people in 2024 and a different 500 in 2026 tells you how the population changed. It cannot tell you how any individual changed, because you never observed the same person twice. A panel study, which follows the same people, can separate change within a person from differences between people. The two designs answer genuinely different questions and are frequently confused in write-ups.
The ecological fallacy
Regions with higher average income may have higher rates of a disease without any richer individual being at higher risk. The association exists at the group level and need not exist, or may even reverse, at the individual level. Ecological studies are useful for hypothesis generation and for policy questions that are genuinely about places. They do not support claims about people.
Assignment
Experimental and quasi-experimental
Randomisation does one thing that no amount of statistical adjustment can replicate: it balances the variables you did not think to measure. Adjustment can only handle confounders you identified and recorded.
| Design | Assignment | Causal claim |
|---|---|---|
| Randomised controlled trial | Random | Strongest available |
| Cluster randomised trial | Random, by group | Strong, but needs design-effect handling |
| Non-equivalent groups | Researcher or circumstance | Weak without strong controls |
| Interrupted time series | By date of a policy change | Reasonable where the trend is stable |
| Difference in differences | Natural, with a comparison group | Reasonable if trends were parallel beforehand |
| Regression discontinuity | By a cutoff on a continuous score | Strong near the cutoff only |
| Single group pre-post | None | None, and often mistakenly made |
The single-group pre-post problem
Measure, intervene, measure again, and the improvement has at least four explanations besides the intervention: regression to the mean, maturation, a secular trend, and testing effects. Regression to the mean alone is enough to produce a clean-looking improvement when participants were selected for having extreme scores, which is how most interventions recruit.
Adding any comparison group, even a non-randomised one, changes the design class entirely. It is usually the single cheapest improvement available to a struggling project.
What difference in differences actually assumes
Not that the two groups are similar. That their outcome trends would have stayed parallel without the intervention. Groups at very different levels are fine; groups moving at different speeds beforehand are not. Plot several pre-intervention periods for both groups, and if the lines are not parallel, the design does not hold and the estimate is measuring the divergence rather than the intervention.
Worked example one
A cohort study
2,000 people followed for five years. 1,000 exposed to a workplace hazard, 1,000 not.
| Developed outcome | Did not | Total | |
|---|---|---|---|
| Exposed | 90 | 910 | 1,000 |
| Unexposed | 45 | 955 | 1,000 |
Step 1. Risk in each group
Because you know who was at risk, you can compute actual risks.
risk if exposed = 90 / 1,000 = 0.0900, or 9.0%
risk if unexposed = 45 / 1,000 = 0.0450, or 4.5%
Step 2. The risk ratio
RR = 0.0900 / 0.0450 = 2.00
Exposed workers were twice as likely to develop the outcome.
Step 3. The confidence interval
Computed on the log scale, because the ratio's sampling distribution is skewed.
SE(ln RR) = √(1/90 − 1/1000 + 1/45 − 1/1000) = 0.1816
ln(2.00) ± 1.96 × 0.1816 = 0.3470 to 1.0398
exponentiate: 1.41 to 2.83
Note the interval is not symmetric around 2.00. That is correct and expected; an interval printed as symmetric around a ratio has been computed wrongly.
Step 4. The absolute measures, which matter more
A doubling sounds alarming until you ask how much risk that is in people.
risk difference = 0.0900 − 0.0450 = 0.0450, or 45 extra cases per 1,000 over five years
number needed to harm = 1 / 0.0450 = 22
One extra case for every 22 workers exposed over five years. This is the figure a safety committee can act on, and the one the ratio hides. The same relative risk of 2.00 on a baseline of 0.1% would mean one extra case per 1,000 workers, which is the same ratio and an entirely different decision.
Step 5. Attributable fraction
AF in the exposed = (0.0900 − 0.0450) / 0.0900 = 50%
Half the cases among exposed workers are attributable to the exposure, assuming it is causal and unconfounded. State that assumption when you report it.
The absolute and relative risk guide takes this contrast further, including the number needed to treat, which is the beneficial version of the same calculation.
Worked example two
The same question, case-control
Now suppose the outcome is rare and following 2,000 people for five years is not affordable. You instead identify 135 cases and recruit 270 controls, and ask both about past exposure.
| Exposed | Unexposed | Total | |
|---|---|---|---|
| Cases | 90 | 45 | 135 |
| Controls | 120 | 150 | 270 |
Step 1. Why you cannot compute risk here
Ninety of 210 exposed people are cases, which looks like a risk of 43%. It is not. You chose how many cases and how many controls to recruit, so that proportion reflects your recruitment ratio, not the population. Change the design to three controls per case and the number moves, while nothing about the disease has changed.
Step 2. The odds ratio
Odds of exposure can be compared, because that comparison does not depend on how many of each group you recruited.
exposure odds among cases = 90 / 45 = 2.00
exposure odds among controls = 120 / 150 = 0.80
OR = 2.00 / 0.80 = 2.50
Equivalently, the cross-product: (90 × 150) / (45 × 120) = 13,500 / 5,400 = 2.50
Step 3. The interval
SE(ln OR) = √(1/90 + 1/45 + 1/120 + 1/150) = 0.2199
exp(ln 2.50 ± 1.96 × 0.2199) = 1.62 to 3.85
Step 4. The useful property
The odds ratio is symmetric. The odds of exposure given disease equals the odds of disease given exposure, which is why a design that samples on outcome can still estimate the association with exposure. It is the mathematical fact that makes case-control studies work at all.
Efficiency, bought with three things. You cannot estimate absolute risk, so no risk difference and no number needed to harm. Exposure is usually reported retrospectively, so recall is a live concern and cases often remember harder. And control selection is a genuine design problem: controls must come from the same population that would have produced the cases, which is easy to state and difficult to achieve.
The conversion trap
When the odds ratio stops behaving
Odds ratios are routinely read as if they were risk ratios. That reading is defensible when the outcome is rare and badly wrong when it is not.
Hold the true risk ratio fixed at 1.5 and vary the baseline risk.
| Baseline risk | True risk ratio | Odds ratio | Overstatement |
|---|---|---|---|
| 1% | 1.50 | 1.51 | negligible |
| 5% | 1.50 | 1.54 | 3% |
| 10% | 1.50 | 1.59 | 6% |
| 30% | 1.50 | 1.91 | 27% |
| 50% | 1.50 | 3.00 | 100% |
At a 50% baseline, an effect that raises risk by half produces an odds ratio of 3.00. Anyone reading that as "three times the risk" has overstated the effect by a factor of two.
This matters because logistic regression reports odds ratios by default, and logistic regression is used on outcomes of every frequency. A cross-sectional survey where 40% of respondents report the outcome will produce odds ratios that should not be described in risk language at all.
If the outcome is common and you want a risk ratio, fit a model that estimates one: log-binomial regression, or Poisson regression with robust standard errors where the log-binomial fails to converge. If you report odds ratios, call them odds ratios throughout and give the baseline prevalence so a reader can judge the gap themselves.
Doing it
In R, SPSS and Stata
R
Risk ratio and risk difference from a cohort table, then a model for a common outcome.
library(epitools)
tab <- matrix(c(90, 910, 45, 955), nrow = 2, byrow = TRUE)
riskratio(tab, rev = "both")
oddsratio(tab)
# risk ratio directly, for a common outcome
glm(outcome ~ exposure + age, family = binomial(link = "log"), data = d)
# fallback when the above will not converge
library(sandwich); m <- glm(outcome ~ exposure + age, family = poisson, data = d)
coeftest(m, vcov = sandwich)
The link = "log" is the whole point: coefficients exponentiate to risk ratios rather than odds ratios.
SPSS
CROSSTABS TABLES = exposure BY outcome / STATISTICS = RISK.
This prints the odds ratio and both relative risks, labelled "For cohort". Read the row labels carefully, because the one you want depends on which outcome category SPSS treated as the event.
LOGISTIC REGRESSION outcome WITH exposure age / PRINT = CI(95).
SPSS reports Exp(B), which is an odds ratio. For a risk ratio on a common outcome use GENLIN with DISTRIBUTION=BINOMIAL LINK=LOG.
Stata
Stata's epidemiological commands are the most direct of the three.
csi 90 45 910 955
cci 90 45 120 150
cs outcome exposure, by(age_group)
csi gives the risk ratio, risk difference and attributable fraction in one table; cci gives the case-control odds ratio. For adjusted estimates:
glm outcome exposure age, family(binomial) link(log) eform
poisson outcome exposure age, vce(robust) irr
The eform and irr options exponentiate, so you read ratios rather than log coefficients.
Avoid these
Six mistakes that cost marks
| Mistake | Why it matters |
|---|---|
| Causal language from cross-sectional data | "Leads to", "results in", "the effect of". The design cannot establish which came first. |
| A risk ratio from a case-control study | The number at risk was never observed. Only the odds ratio is available. |
| Reading a large odds ratio as risk | At a 50% baseline an odds ratio of 3.00 corresponds to a risk ratio of 1.50. |
| Calling a repeated survey longitudinal | Different people each wave means no within-person change can be estimated. |
| Single-group pre-post as evidence of effect | Regression to the mean produces the same improvement with no intervention at all. |
| Group-level findings applied to individuals | The ecological fallacy. The association can reverse at the individual level. |
The first is the most common and the easiest to fix, because it is a writing problem rather than an analysis problem. Replace "was associated with" for every instance of causal phrasing and the chapter usually survives review intact.
The third is the most consequential, because it changes the size of the reported effect rather than its interpretation, and it passes unnoticed through peer review more often than it should.
Cross-sectional data cannot establish which came first, and no model repairs that afterwards. It is far cheaper to resolve at the protocol stage than in a response to reviewers.
See statistical consultancy
Sidra KhanLead BiostatisticianWriting it up
How to report it
A sentence that does all of this at once: "In this prospective cohort of 2,000 workers followed for five years, exposure was associated with a higher incidence of the outcome (9.0% versus 4.5%; risk ratio 2.00, 95% CI 1.41 to 2.83; 45 additional cases per 1,000 over five years)."
Common questions
Frequently asked questions
What is the difference between a cohort and a case-control study?
A cohort study starts from exposure and follows people forward to see who develops the outcome, which lets you calculate actual risks and a risk ratio. A case-control study starts from the outcome and looks back at exposure, which is far more efficient for rare outcomes but yields only an odds ratio, because the number of people at risk was never observed.
Can a cross-sectional study show causation?
No, and the reason is structural rather than statistical. Exposure and outcome are measured at the same moment, so the data cannot distinguish the exposure causing the outcome from the outcome causing the exposure. No sample size or regression model repairs this. Report association and say explicitly that temporal order could not be established.
Is a repeated cross-sectional survey longitudinal?
No. Surveying different people at each wave tells you how the population changed but nothing about how any individual changed. A panel study follows the same people and can separate within-person change from between-person differences. The distinction is frequently blurred in write-ups and examiners look for it.
When can I treat an odds ratio as a risk ratio?
When the outcome is rare, roughly below 10%, the two are close: at a 5% baseline an odds ratio of 1.54 corresponds to a risk ratio of 1.50. Above that the gap grows quickly, and at a 50% baseline an odds ratio of 3.00 corresponds to a risk ratio of only 1.50. For common outcomes, estimate the risk ratio directly with a log-binomial or robust Poisson model.
What makes a study quasi-experimental?
An intervention is assigned, but not at random. Interrupted time series, difference in differences and regression discontinuity are the main forms. Each rests on an explicit assumption, such as parallel pre-intervention trends for difference in differences, and the write-up should state that assumption and present evidence it holds rather than leaving it implicit.
Why is single-group pre-post weak?
Because at least four alternative explanations survive: regression to the mean, maturation, background trends and testing effects. Regression to the mean is the serious one, since participants are usually recruited for having extreme scores and extreme scores move toward the average on their own. Adding any comparison group, even a non-randomised one, improves the design substantially.
Which design is best?
The one that matches the question and is feasible. A randomised trial supports the strongest causal claim but cannot be run for most exposures, for ethical or practical reasons. A well-conducted cohort study often tells you more than a poorly conducted trial. The hierarchy ranks designs for causal inference, not for usefulness.
Do I need to follow a reporting guideline?
Most journals require it and most examiners expect it. STROBE covers observational designs, CONSORT covers randomised trials, and TREND covers non-randomised evaluations. Each is a checklist; working through it before you write usually surfaces two or three items you had not planned to report.
Related services
If the design has to survive review
Power analysis & sample size
A priori power analysis with the justification an ethics committee will accept.
From £180Statistical consultancy
Design, analysis and reporting for charities, universities, the NHS and business.
From £950Independent statistical review
A fixed-price check of analysis already done, before it reaches a journal or a board.
Fixed quoteEvery piece of work is done and signed by a named statistician. Send the data or describe the project and you get a price and a deadline, usually within one working day.