Quantitative Research Skill

Complete workflow for designing, executing, analyzing, and reporting quantitative research. Produces four separate deliverable files.

Workflow Overview

Phase 1: Research Design      -> RQs, hypotheses, variables, design
Phase 2: Sampling Plan        -> Target user profiles, sample size, power
Phase 3: Instrument Design    -> Questionnaire (Deliverable 1)
Phase 4: Data Collection      -> Dataset (Deliverable 2)
Phase 5: Statistical Analysis -> Analysis Results (Deliverable 3)
Phase 6: Report Writing       -> Research Report (Deliverable 4)

Four Deliverables (each exported as a separate file):

1. questionnaire.md — Complete structured questionnaire with scales
2. data.csv — Clean dataset with all variables
3. analysis_results.md — Full statistical analysis output
4. report.md — APA-style research report

Execute phases sequentially. Each phase feeds into the next.

---

Phase 1: Research Design

Goal: Transform a research topic into a testable experimental/survey design.

Steps

1. Define research questions: Convert broad topic into 1-3 specific, testable RQs
2. Develop hypotheses: For each RQ, state H1 (alternative) and H0 (null)
   • Include: main effects, interaction effects, mediation, moderation
3. Operationalize variables:
   • IV(s): name, levels, how manipulated/measured
   • DV(s): name, how measured, expected range
   • Mediators: name, measurement approach
   • Moderators: name, measurement approach
   • Controls: covariates to hold constant
4. Choose design:
   • Between-subjects (independent groups)
   • Within-subjects (repeated measures)
   • Mixed factorial
   • Survey/correlational
5. Plan analysis: Map each hypothesis to a specific statistical test

Design Selection Guide

| Research Aim | Design | Analysis | |-------------|--------|----------| | Compare 2+ groups | Between-subjects ANOVA | One-way ANOVA / t-test | | Factorial effects | Factorial design | Multi-way ANOVA | | Mechanism (why) | Mediation design | Bootstrap indirect effect | | Boundary condition | Moderation design | Interaction regression | | Complex model | SEM design | Path analysis / SEM |

Output

• Research design document (RQs, hypotheses, variables, design matrix)
• Analysis plan table (each hypothesis -> test -> criterion)

---

Phase 2: Sampling Plan

Goal: Define who to recruit and how many.

Steps

1. Define target population: Who has the characteristics needed?
2. Create user personas: 6-10 detailed profiles representing variation
3. Determine sample size:
   • Rule of thumb: 30+ per cell for between-subjects
   • Power analysis: G*Power for precise calculation (power=0.80, alpha=0.05)
   • Minimum total: n=200 for survey, n=6/cell for experiment
4. Plan randomization: How to assign participants to conditions?
5. Plan compensation: Incentive structure if applicable

Sample Size Guidelines

| Design | Minimum per cell | Recommended per cell | |--------|-----------------|---------------------| | 2-group comparison | 30 | 50-100 | | 2×2 factorial | 25 | 40-60 | | 2×3 factorial | 20 | 30-50 | | 2×3×2 factorial | 15 | 25-40 | | Mediation | 200 total | 300-500 | | SEM/Path analysis | 200 total | 300-500 |

Output

• 6-10 participant personas
• Sample size justification (with power analysis)
• Randomization plan

---

Phase 3: Instrument Design

Goal: Create valid, reliable measurement tools. → Deliverable 1: questionnaire.md

Steps

1. Select scales: Use established scales when available (check Cronbach's alpha > 0.70)
2. Adapt scales: Modify existing scales to fit context, preserve core items
3. Create manipulation materials: Design stimuli for each experimental condition
4. Build questionnaire structure:
   • Section 1: Manipulation/instructions
   • Section 2: Manipulation check items
   • Section 3: DV measurement items
   • Section 4: Mediator items
   • Section 5: Moderator items
   • Section 6: Control variable items
   • Section 7: Demographics
5. Use Likert scales: 7-point strongly recommended (1-7), 5-point acceptable
6. Pilot test: Run with 20-30 participants, check for floor/ceiling effects

Scale Quality Checklist

• [ ] Each construct has 3-6 items
• [ ] Mix of positively and negatively worded items
• [ ] Reverse-coded items clearly marked
• [ ] Clear instructions for each section
• [ ] Estimated completion time < 15 minutes

Deliverable 1: questionnaire.md

Export a standalone questionnaire file containing:

• Study title and estimated completion time
• All experimental conditions with full text
• All scale items with source attribution
• All manipulation check items
• All demographic questions
• Randomization instructions

See references/templates.md for questionnaire template.

---

Phase 4: Data Collection

Goal: Obtain clean, complete data. → Deliverable 2: data.csv

Steps

1. Generate/simulate data: If conducting simulated research:
   • Set population means based on theory/prior research
   • Add realistic variance (SD 0.7-1.2 for 7-point scales)
   • Ensure manipulation checks show large effects (d > 2.0)
   • Build in hypothesized effects at expected magnitude
2. Real data collection: If fielding survey:
   • Use Qualtrics/SurveyMonkey/Google Forms
   • Set attention checks (e.g., "Select strongly agree")
   • Set completeness requirements
3. Clean data: Remove speeders, attention check failures, incomplete responses
4. Document exclusions: Report how many and why

Data Structure Standards

The CSV must include these columns:

• participant_id: Unique identifier (1 to N)
• Condition indicators: e.g., ai_use, transparency, originality
• Raw item scores: One column per item (e.g., CP1, CP2, CW1, AS1)
• Composite scores: Mean scores for each construct
• Demographics: gender, age, education, work_exp, industry, position, ai_freq

Data Simulation Standards

• Total n matches sample size plan
• Equal or near-equal cell sizes
• Manipulation checks show large effects (Cohen's d > 2.0)
• Realistic inter-item correlations (r = 0.30-0.70)
• Some missing data (5-10%) if realistic
• Demographics match target population distributions

Deliverable 2: data.csv

Export a clean CSV file with:

• Header row with variable names
• One row per participant
• All raw item scores and composite variables
• Condition assignment variables
• Demographic variables
• Missing values coded as NA

---

Phase 5: Statistical Analysis

Goal: Test all hypotheses rigorously. → Deliverable 3: analysis_results.md

Analysis Pipeline

Execute in this order:

Step 1: Descriptive Statistics

• Sample characteristics (demographics)
• Variable means, SDs, ranges
• Correlation matrix

Step 2: Manipulation Checks

• t-tests or ANOVA on manipulation check items
• Cohen's d for effect size
• Must be significant (p < .001) and large (d > 2.0)

Step 3: Reliability Analysis

• Cronbach's alpha for all multi-item scales
• Alpha > 0.70 acceptable, > 0.80 good, > 0.90 excellent

Step 4: Main Effect Tests

• Independent samples t-test (2 groups)
• One-way ANOVA (3+ groups)
• Report: M, SD, t/F, p, Cohen's d / eta-squared

Step 5: Interaction Tests

• Multi-way ANOVA for factorial designs
• Moderated regression for continuous moderators
• Report: F, p, eta-squared, simple effects

Step 6: Mediation Analysis

• Baron & Kenny 4-step method
• Bootstrap indirect effect (5000 samples)
• Report: indirect effect, 95% CI, proportion mediated

Step 7: Moderation Analysis

• Center predictor and moderator
• Compute interaction term
• Run moderated regression
• Report: beta, p, interaction plot

Step 8: Follow-up Tests

• Post-hoc comparisons (Tukey HSD, Bonferroni)
• Simple effects analysis for interactions
• Contrast analysis for planned comparisons

Deliverable 3: analysis_results.md

Export a standalone analysis file containing ALL statistical output:

• Section 1: Descriptive statistics tables
• Section 2: Manipulation check results (with effect sizes)
• Section 3: Reliability analysis (Cronbach's alpha table)
• Section 4: Main effects (t-tests/ANOVA tables)
• Section 5: Interaction effects (ANOVA tables + simple effects)
• Section 6: Mediation analysis (path coefficients + bootstrap CIs)
• Section 7: Moderation analysis (regression tables + interaction plots description)
• Section 8: Hypothesis testing summary (H1-Hn: supported/not supported)
• Section 9: Figures (bar charts, interaction plots, mediation diagrams)

See references/templates.md for analysis output table templates.

---

Phase 6: Report Writing

Goal: Produce APA-style report. → Deliverable 4: report.md

Report Structure

1. Abstract (150-250 words)
2. Introduction (2-3 pages) — literature review + hypotheses
3. Method (1-2 pages) — participants, design, materials, procedure
4. Results (2-4 pages) — key findings (refer to full analysis in Deliverable 3)
5. Discussion (2-3 pages) — contributions, implications, limitations
6. References
7. Appendix — note that full questionnaire, data, and analysis are separate files

Deliverable 4: report.md

Export a standalone APA-style report that:

• Presents key findings (not all raw output — refer to Deliverable 3)
• Interprets results in context of hypotheses
• Discusses theoretical and practical implications
• Acknowledges limitations
• Cites all four deliverable files in appendix

---

Four Deliverables Summary

| # | File | Content | When to export | |---|------|---------|---------------| | 1 | questionnaire.md | Complete questionnaire with all scales and items | After Phase 3 | | 2 | data.csv | Clean dataset with all variables | After Phase 4 | | 3 | analysis_results.md | Full statistical output (tables + figures) | After Phase 5 | | 4 | report.md | APA-style research report | After Phase 6 |

All four files are independent deliverables. The report references but does not duplicate the questionnaire, data, or analysis results.

---

Key Principles

1. Design first, analyze second: Never change hypotheses after seeing data
2. Report everything: Report non-significant results too
3. Effect size is king: Always report effect sizes, not just p-values
4. Check assumptions: Normality, homogeneity, independence before parametric tests
5. Four files, four purposes: Questionnaire for replication, data for transparency, analysis for rigor, report for communication
6. Separate concerns: Each file has one job — do not duplicate content across files