Guides  /  employee surveys

Employee engagement survey analysis

Most engagement reports quote a single percentage, rank the departments underneath it, and act on the bottom three. Every step of that is shakier than it looks. This guide covers what the response rate does to your headline number, how to compare teams without manufacturing false findings, what driver analysis can and cannot tell you, and how to report a result a board can act on.

Saad Tariq Written and reviewed by Saad Tariq, Principal Statistician
Updated 4 November 202613 min read
How do you analyse an employee engagement survey?

Start with who did not answer, because that decides what the headline number means. Report the engagement score as a proportion with a confidence interval rather than a bare percentage. Compare teams using differences with intervals, not a ranked table, and account for how many comparisons you are making. Use regression to identify drivers, while being clear that it shows association and not cause. Then report the uncertainty alongside the number, because a board acting on a two point movement needs to know whether two points means anything.

Definitions

What the score actually measures

Engagement is not a thing you can weigh. It is a construct, inferred from how people answer a set of questions, and the number you report depends entirely on which questions you asked and how you combined them.

Most instruments use a five point agreement scale and summarise in one of two ways.

MethodWhat it doesTrade-off
Percent favourableCounts the share choosing agree or strongly agreeEasy to explain. Throws away the difference between a 3 and a 1, so a workforce drifting from neutral to hostile can look unchanged.
Mean scoreAverages the numeric responsesUses the whole scale. Treats a Likert scale as though the gap from 1 to 2 equals the gap from 4 to 5, which is an assumption rather than a fact.

Neither is wrong. What matters is picking one, saying which, and never switching between them across waves, because a change of method can produce a movement larger than anything your organisation actually did.

Before anything else, check the index holds together

If you are combining several items into one engagement score, they should be measuring the same underlying thing. Cronbach's alpha is the usual check, and anything below about 0.7 means the items are not cohering and the composite is hiding more than it shows. An alpha above 0.95 is the opposite problem: the questions are near-duplicates and you are paying people to answer the same thing five times.

The part that decides everything

The response rate, and the number nobody reports

An engagement survey is voluntary. People who feel strongly, in either direction, are more likely to answer than people who feel nothing much. That makes it a textbook case of self-selection, and it is why the response rate is not an administrative footnote but the single most important figure in the report.

Typical rates for a whole-organisation survey run between 40% and 70%. Below 40% the result describes a subgroup you cannot characterise. Above 85%, check nobody felt obliged.

✓Report the rate with its denominator. 684 of 1,200 is informative; "a strong response" is not.
✓Report it by department too. An overall 57% hiding a department at 22% means that department's result should not be quoted at all.
✓Compare responders with the workforce on whatever you hold: grade, tenure, site, full or part time.
✓If the rate moved between waves, say so. A rise from 45% to 65% changes who is in the sample, and some of your year-on-year movement is that.

Worked example one

The score and its interval

1,200 staff invited. 684 responded. Of those, 431 answered favourably on the engagement index.

Step 1. The response rate

684 / 1,200 = 0.570, so 57.0%. Respectable, and it also means 516 people said nothing.

Step 2. The headline score

431 / 684 = 0.630, so 63.0% favourable.

Step 3. The interval around it

Use a Wilson interval rather than the ordinary normal approximation, because it behaves properly near the ends of the scale where department-level figures often sit.

95% CI: 59.3% to 66.5%, a width of 7.2 points.

That width is the useful part. If last year's figure was 61%, this year's 63% sits comfortably inside the range of ordinary sampling variation. Reporting "engagement rose two points" would be describing noise.

Step 4. The number nobody reports

The 63% describes respondents. The board will hear it as a statement about the workforce. Those are different claims, and you can bound the gap.

if every non-responder were unfavourable: 431 / 1,200 = 35.9%
if every non-responder were favourable: (431 + 516) / 1,200 = 78.9%

The true workforce figure lies somewhere between 36% and 79%. That is a 43 point span, and it is not a trick: it is the honest consequence of 516 people declining to answer.

What to do with that span

Not report it as the finding, which would be useless. Report the 63% as what it is, a figure describing respondents, and use the bounds once in the methodology to establish why the response rate matters. It stops anyone treating the headline as a census, and it makes the case for improving the response rate next wave far better than any amount of encouragement.

Worked example two

Comparing two departments

Operations and Finance are reported side by side. Operations looks poor.

DepartmentFavourableRespondedScore95% CI
Operations11821056.2%49.4% to 62.7%
Finance476275.8%63.8% to 84.8%

Step 1. Notice the interval widths

Operations spans 13 points, Finance spans 21. Finance looks better and is measured far less precisely, because 62 responses cannot pin a percentage down. Any ranking that ignores this will put small teams at both the top and the bottom.

Step 2. Test the difference properly

Do not eyeball whether the intervals overlap, which is a conservative and slightly wrong test. Compute the interval around the difference itself.

difference = 75.8% − 56.2% = 19.6 points
SE = √[ (0.562 × 0.438 / 210) + (0.758 × 0.242 / 62) ] = 0.0643
95% CI = 19.6 ± 1.96 × 6.43 = 7.0 to 32.2 points

Step 3. Read it honestly

The interval excludes zero, so there is a real difference here. But it runs from 7 points to 32, which is the difference between a mild gap worth monitoring and a serious problem worth intervening in. The point estimate of 19.6 is the middle of a wide range, not a measurement.

That is the sentence to put in the report: there is a difference, and its size is uncertain.

Worked example three

Comparing with last year, and with other organisations

Every engagement report answers two comparison questions. Both are harder than they look.

Year on year, worked through

Last year: 61% favourable from 652 responses. This year: 63% from 684. The slide deck says engagement is up two points.

change = 63.0% − 61.0% = +2.0 points
SE = √[ (0.61 × 0.39 / 652) + (0.63 × 0.37 / 684) ] = 2.66 points
95% CI = 2.0 ± 1.96 × 2.66 = −3.2 to +7.2 points

The interval includes zero. The honest reading is that engagement is not distinguishably different from last year. It may have risen seven points. It may have fallen three.

The smallest change this survey can actually detect

At these sample sizes anything under about 5.2 points is inside the noise. That single number is worth knowing before the survey goes out, because it tells you what movements are worth building a narrative around.

And the uncomfortable arithmetic behind it

To detect a two point change reliably, with 80% power, you would need roughly 9,242 responses in each wave. At a 57% response rate that means inviting about 16,214 people, every year.

An organisation of 1,200 staff cannot detect a two point change. Not with a better survey, not with more reminders, not ever. The sample is the organisation. That is a limit of the method rather than a failure of execution, and saying so plainly in the report protects everyone from chasing movements that are not there.

What to do instead of chasing points

Set a threshold in advance. Agree with the board that movements under five points will be reported as "no detectable change", and that anything you act on has to clear it. Done before the data arrives, that is good governance. Done afterwards it looks like excuse-making.

External benchmarks

Comparing against an industry benchmark is the weakest comparison in the report, and it is usually given the most prominence.

✓The question wording differs. Benchmark providers rarely use your exact items, and small wording changes move favourability by several points.
✓The benchmark pool is self-selected. It comprises organisations that bought the product and chose to contribute, which is not a random sample of employers.
✓The response rates differ. If their contributors average 80% and you achieved 57%, you are comparing different kinds of sample.
✓There is rarely an interval. A benchmark quoted as a single number invites a precision it does not have.

Use benchmarks for orientation, never as a target. Your own trend, measured consistently with the same instrument and reported with its uncertainty, is far more informative than a comparison with organisations you cannot inspect.

The league table problem

Why ranking departments manufactures findings

Comparing two departments is fine. Comparing all of them against each other and then acting on whichever gaps look significant is not, and it is what almost every engagement report does.

DepartmentsPairwise comparisonsChance of at least one false positive at p<.05
61554%
126697%
20190>99%

With twelve departments you are almost certain to find at least one "significant" gap that is pure chance. Act on it and you will restructure a team that was performing normally.

✓Decide in advance which comparisons matter. Two or three pre-specified ones need no correction.
✓If you are scanning everything, apply a multiplicity correction and say which.
✓Suppress any department below a minimum response count. Twenty is a common floor, as much for anonymity as for statistics.
✓Plot every department with its interval instead of ranking them. A chart of overlapping intervals communicates the truth that a ranked list conceals.

There is a second, quieter problem. Employees sit within teams, teams within departments, and people in the same team share a manager and a workload. Their answers are not independent, which is the clustering problem, and it means the effective sample size is smaller than the response count suggests.

The analysis boards ask for

Driver analysis, and what it cannot tell you

Driver analysis regresses the overall engagement score on the individual survey items to see which ones move with it. It is genuinely useful and routinely over-interpreted.

What it does

It ranks items by how strongly they relate to overall engagement, usually with standardised coefficients or relative weights. Items high on that list are candidates for attention.

Three reasons it is not a to-do list

✓It is cross-sectional. Everything was measured at one moment on one questionnaire. "My manager supports me" predicting engagement does not establish that improving managers raises engagement. Engaged people may simply rate everything more warmly.
✓The predictors overlap heavily. Survey items correlate with each other, often above 0.6. That makes individual coefficients unstable, and the ranking can reshuffle between waves without anything real changing. Relative weights or dominance analysis handle this better than raw coefficients.
✓Common method variance inflates everything. One person, one questionnaire, one sitting. Part of every correlation is the act of answering, not the thing being measured.

Report drivers as "areas most associated with engagement in this survey" and never as "what to fix to raise engagement". The second claim needs a change over time, or better still something that was actually changed in one area and not another.

Doing it

In R, SPSS and Stata

R

Score, interval and comparison in four lines.

library(DescTools)
BinomCI(431, 684, method = "wilson")
BinomDiffCI(47, 62, 118, 210, method = "scorecc")
psych::alpha(items)

For driver analysis use relative weights rather than plain coefficients, because the items overlap: relaimpo::calc.relimp(model, type = "lmg"). If you need to respect team clustering, lme4::lmer(engagement ~ driver + (1 | team)).

SPSS

Reliability first, then the proportions.

RELIABILITY VARIABLES = q1 TO q12 /MODEL = ALPHA /STATISTICS = SCALE CORR.
FREQUENCIES VARIABLES = favourable /STATISTICS = MEAN.
CROSSTABS TABLES = department BY favourable /STATISTICS = CHISQ /CELLS = COUNT ROW.

SPSS gives a Wald interval for a proportion by default, which misbehaves near 0% and 100%. For department-level figures, compute Wilson intervals yourself or use the bootstrap module.

Stata

ci proportions favourable, wilson
prtest favourable, by(department)
alpha q1-q12, item

If the survey was weighted to match the workforce profile, declare it once with svyset [pweight = wt] and prefix everything with svy:, otherwise the intervals ignore the weighting and come out too narrow.

Check the denominator in every table

Survey platforms vary in whether "percent favourable" excludes neutral responses, excludes non-answers to that specific item, or uses the full respondent count. Three defensible choices, three different numbers. Establish which your platform uses before comparing anything with last year.

Avoid these

Six mistakes that cost you credibility

MistakeWhy it matters
A headline percentage with no interval63% has a range of 59% to 67% here. Without it, a two point movement looks like a result.
Treating the score as the workforce viewIt describes respondents. With 57% responding, the workforce figure could sit anywhere from 36% to 79%.
Ranking departmentsWith twelve departments there is a 97% chance of at least one false positive. Small teams fill both ends of the ranking.
Quoting tiny teamsA department with 8 responses has an interval roughly 35 points wide. Suppress below a stated floor.
Changing the method between wavesSwitching from mean score to percent favourable can move the number more than anything the organisation did.
Presenting drivers as causesCross-sectional, overlapping predictors, single method. It identifies associations, not levers.
Who did not answer your staff survey?

A 54% response rate is not 54% of a view, it is the view of the people who chose to reply, and that group is rarely neutral. Send the export and you get the index, the drivers that actually move it, and what the non-response pattern does to the result.

See survey and insight analysis
Saad TariqSaad TariqPrincipal Statistician

Writing it up

How to report it

✓Response rate with its denominator, overall and by department.
✓How responders compare with the workforce on everything you hold.
✓The engagement score with a confidence interval, and a note of which method produced it.
✓Differences with intervals, not a ranked table. State which comparisons were planned.
✓A minimum reporting threshold, stated, and applied without exception.
✓Drivers labelled as associations, with the limitations in the same paragraph rather than an appendix.

One sentence is worth drafting carefully, because it is the one a chief executive repeats: "Engagement among respondents was 63% favourable (95% CI 59% to 67%), based on a 57% response rate. The two point movement from last year is within the range expected from sampling alone."

Common questions

Frequently asked questions

What is a good response rate for an employee survey?

Between 40% and 70% is typical for a whole-organisation survey. Below 40% the result describes a subgroup you cannot characterise, and department-level figures become unreliable well before the overall one does. Report the rate with its denominator and by department, because an overall 57% can hide a department at 22%.

Should I report percent favourable or the mean score?

Either, provided you pick one and keep it. Percent favourable is easier to explain but discards the difference between neutral and hostile. The mean uses the whole scale but assumes equal spacing between points. Switching between the two across waves can produce a movement larger than anything your organisation actually did.

Is a two point rise in engagement meaningful?

Usually not. With 684 responses a score of 63% carries an interval of roughly 59% to 67%, so a two point movement sits well inside ordinary sampling variation. Report the interval alongside the score and the question answers itself.

Can I rank departments by engagement?

You can, but it manufactures findings. Twelve departments produce 66 pairwise comparisons and a 97% chance of at least one false positive at the conventional threshold. Small teams also have wide intervals and so fill both ends of any ranking. Plot each department with its interval instead.

What minimum team size should I report?

Twenty responses is a common floor, for anonymity as much as for precision. Below that the interval is so wide the number carries almost no information, and individuals can start to be identifiable from the breakdown.

Does driver analysis tell me what to fix?

No. It identifies items most strongly associated with overall engagement in a single cross-sectional survey, where the predictors overlap heavily and everything was measured by one person on one questionnaire. It generates candidates worth investigating, not a list of levers.

Do I need to weight the results?

If responders differ from the workforce on something you can measure, such as grade or site, weighting brings the estimate closer. It corrects only on the variables you weight by, so describe the result as adjusted for those variables rather than corrected for non-response. Declare the weights in the software or the intervals will be too narrow.