"Does X affect Y?" is a simple question. Most survey analyses stop there. Run a correlation, report a p-value, done.
But the questions that actually matter usually go one level deeper. Here's a hypothetical example I'll use throughout this article: does education improve wellbeing? A correlation can tell you the two move together. It can't tell you how. Maybe education raises a person's sense of control over their own life, and that sense of control is what actually lifts wellbeing. In diagram terms:
education (X) → sense of control (M) → wellbeing (Y)
plus whatever direct effect of education remains on top of that pathway.
Testing that whole chain at once, including whether fuzzy concepts like "sense of control" and "wellbeing" are even measured properly, is what structural equation modeling (SEM) is built for. This article walks through a real project where I used it. The education example is just for illustration, by the way. It's not the client's study, and since client work at DS Statistics is confidential, I'll describe the real project by method and process only.
The project
A graduate researcher came to me with survey data from a wellbeing study. Five validated questionnaires plus a demographic form, 85 items in total, and one core question their thesis depended on: was the relationship between a stressor and wellbeing carried, at least in part, by an intermediate psychological process? In statistical terms, a mediation question.
A mediation question with multiple validated scales is a textbook fit for SEM. But before I could build any model, the data had to earn its place.
Step 1: Data cleaning, where the project is won or lost
The raw dataset had 355 responses. The final sample had 214.
Losing 40% of the data sounds alarming. It wasn't a problem, it was the job. Online surveys collect a predictable amount of noise, and every noisy response left in quietly distorts everything built on top of it. I screened for four things:
- Missingness. Anyone missing more than 20% of items on the key measures was removed. A scale score built from half-answered questions isn't a measurement.
- Straight-lining. Answering "4" to every single question, including the reverse-worded ones, is a response pattern, not an opinion.
- Implausible completion times. You cannot read and answer an 85-item survey in two minutes.
- Failed attention checks. Items placed in the survey specifically to catch inattentive responding did their job.
The uncomfortable truth is that a beautiful model fit to contaminated data is worse than no model at all. It produces confident, publishable, wrong conclusions. If a consultant hands you results without telling you how the data was screened, ask.
Step 2: Check that each scale actually measures something
Before a scale goes into a model, it should show internal consistency. In plain terms: the items that are supposed to measure one thing should behave like they measure one thing. The standard check is Cronbach's alpha.
Every scale in this project cleared the conventional threshold comfortably. All alphas were above .83, several above .90. That mattered for two reasons. It confirmed the validated instruments were performing as designed in this sample, and it meant the latent constructs in the model (more on those in a moment) would sit on solid measurement.
When alpha comes back low, that's not a formality to bury in a footnote. It's a stop sign. Results downstream of a bad scale can't be trusted until you understand the measurement problem.
Step 3: Model latent constructs, not sum scores
The usual shortcut in survey analysis is to add up a scale's items into a single score and correlate the scores. It works, roughly. But it makes a strong hidden assumption: that every item measures the construct equally well, with no error.
SEM does something better. Each concept in the model is treated as a latent variable, an underlying quantity that the individual survey items each imperfectly reflect. The model estimates how strongly each item loads on its construct and separates true construct variance from measurement noise.
Back to the running example. "Wellbeing" isn't something you can read off a single question. Ask people to rate their wellbeing from 1 to 10 and every respondent will interpret it differently. A validated scale instead asks several items ("I am satisfied with my life," "my mood is generally positive," "my life has a sense of purpose"), and wellbeing enters the model as the latent construct those items jointly reflect. Same logic for sense of control: items about daily control, having choices, confidence in handling problems. Even education can work this way, reflected by years of schooling, highest degree, and further training.
The practical payoff: the relationships you actually care about (does X affect Y through M?) are estimated between the constructs themselves, the ovals in the diagram, not between error-contaminated sum scores. The effects are cleaner and the whole measurement structure is testable instead of assumed.
Step 4: Test the model, and let it fail if it should
A hypothesized model is a claim about how the data should be structured, and SEM makes that claim testable. I evaluated fit against the full set of standard indices: chi-square, RMSEA, CFI and TLI, with AIC and BIC used to compare competing model structures against each other.
Two things are worth underlining here.
First, competing models. The project tested more than one theoretically plausible structure and compared them formally, rather than fitting a single model and declaring victory. When a committee member or reviewer asks "why this model and not that one?", the answer exists in the analysis, with numbers attached.
Second, no cherry-picking. I judged fit across the full set of indices, not whichever one happened to look friendly. Any model can be made to look good if you only report its best statistic.
Step 5: Separate direct from indirect effects
The heart of a mediation analysis is splitting the total relationship between the predictor and the outcome into two parts:
- an indirect effect, the pathway that runs through the mediator, and
- a direct effect, whatever relationship remains after accounting for that pathway.
In the running example, the indirect effect is education → sense of control → wellbeing (the a × b path in the diagram). The direct effect (c′) is whatever link between education and wellbeing remains after that pathway is accounted for. Maybe it operates through income, or social status, or something the model doesn't include. If the indirect effect is significant and the direct effect shrinks but survives, sense of control partially mediates the relationship. If the direct effect disappears entirely, the mediation is complete and education's benefit runs wholly through control.
Estimating both, with confidence intervals on the indirect effect, is what lets a researcher say precisely which mechanisms carry a relationship. And just as importantly, which hypothesized mechanisms the data does not support.
That last part deserves its own paragraph. In this project, as in most honest analyses, some hypothesized pathways held up and others didn't. That's a good outcome. The point of the analysis is to find out what the data actually says, and a thesis defense goes far better when the candidate can explain a null result than when every result is suspiciously convenient.
The deliverable
The final output was a write-up-ready quantitative analysis the researcher could take straight into their thesis: cleaned data with documented exclusion criteria, reliability results for every scale, model fit statistics with comparisons, and effect estimates with the reasoning behind every decision. Defensible in front of a committee.
That last phrase is the real deliverable. Numbers are easy to produce. Numbers you can defend under questioning are the product.
Is SEM the right tool for your data?
SEM tends to be the right tool when three things are true:
- You measure concepts with multiple survey items (validated scales, questionnaires)
- You have a "why" or "through what" question, not just a "whether" question
- You want the whole chain tested at once, rather than one correlation at a time
That describes a lot of real-world work: program evaluations that need to show funders how an intervention works, workplace health studies untangling workload, burnout, and turnover, clinical research separating a treatment's direct effects from its side benefits, and graduate research of exactly the kind described here.
DS Statistics provides survey design and statistical analysis for researchers, nonprofits, and health organizations across BC. If your survey data has a "why" question buried in it, get in touch.