- Create a design rationale and claim/data/assumption matrix.
Different questions require different evidence
Describing a situation, predicting an outcome and estimating an intervention effect are different aims. A highly accurate predictor does not automatically identify causal effects. Write down the purpose, target population and outcome before choosing a statistical package.
Specify the unit and dependencies
Cross-sectional, longitudinal, experimental, qualitative and mixed-method designs have distinct requirements. State whether the unit is a person, class, organization, interview or repeated observation. Repeated measurements and clustered samples cannot simply be treated as independent observations; collection and analysis should respect these dependencies.
Connect concepts to measurements
Separate a theoretical construct from the instrument used to measure it. Document definitions, timing, units, ranges and model roles. Reliability alone does not establish construct validity. A translated instrument also needs appropriate linguistic and cultural assessment.
Check feasibility and bias
Review actual sample access, recruitment time, resources, permissions, computing needs and attrition. Describe differences between accessible participants and the target population. Maintain a risk table with the issue, consequence, mitigation and remaining limitation.
Create a defensible design record
Map each question to the required data, collection procedure, analysis, key assumption and output. Revise questions lacking suitable evidence or methods. A pilot can expose recording and workflow problems, but does not replace the main design. Include unresolved decisions so that a supervisor can review them before collection begins.
Practical research checklist
- Distinguish descriptive, predictive and causal goals.
- Define the unit and dependencies.
- Check construct and measurement alignment.
- Document permissions, resources and limitations.
Worked case and implementation decisions
The following is a fictional teaching case. Do not use its numbers or wording as actual study findings.
Similar questions need different designs. “Who uses support?” is descriptive; “Does support increase engagement?” needs an intervention or defensible causal strategy. Cross-sectional data leave temporal order and self-selection unresolved. If a causal design is infeasible, narrow the question and title to an estimable association. A mixed-methods project specifies where and why the components integrate; adding a few interviews alone does not create a coherent mixed-methods design.
| Stage | Teaching example | Verification question |
|---|---|---|
| Question | Description, association or effect? | Is the claim type explicit? |
| Time | Before and after exposure | Is temporal order known? |
| Selection | Volunteers or probability sample | Is selection bias assessed? |
| Measurement | Comparable definitions across groups | Is comparison valid? |
| Integration | Defined quantitative and qualitative roles | How do outputs answer one question? |
Exercise output: Create a design rationale and claim/data/assumption matrix.
Sources and further reading
Official sources for verification and further reading