Subgroup = a slice of a survey small enough that one person can move the headline.
In April, UX Tools published State of Prototyping. 1,478 designers, 18 regions, an open dataset and a free API. It is generous work and I recommend it. But one finding travelled further than the rest: design engineers feel more valuable, researchers feel most at risk. 50.0% against 17.4%.
So I called their API and asked how many researchers had answered. Twenty three.
"39.1% of researchers feel less secure" is nine people.
How a stat gets away from its own survey
1. Check the denominator before you quote the percentage
What got quoted: Researchers are the most anxious role in design. Nearly 40% feel less secure.
What the data says: The researcher row is 23 responses. 39.1% is 9 people. 17.4% is 4 people. The percentages are arithmetically correct, and that is the whole problem. They are formatted exactly like the trustworthy numbers in the same report.
Do this: Ask for the row count before you repeat the row.
2. The survey flagged it, the retelling did not
What got quoted: A clean two-role comparison, researcher against design engineer.
What the data says: The methodology says the researcher role should be treated as directional only, because of small sample size. Credit to them for printing that. It just did not travel. The summary section, the role-by-role career advice and the newsletter write-up all repeat the figure without it.
Do this: Read the methodology note. It is usually the most interesting paragraph on the page.
3. One person is worth four points
What the data says: At 23 responses, a single answer is 4.3 points. The margin of error is roughly 20 points either way, so the real figure sits somewhere between a fifth and three fifths of researchers. Pew puts it plainly: subgroup estimates have fewer cases, so their margins are larger, in some cases much larger.
Do this: Treat a decimal place on a small sample as decoration.
4. The solid finding never got a headline
What got quoted: The researcher panic.
What the data says: 862 IC designers answered. 32.4% of them feel less secure and 24.8% feel more valuable. That is the largest group in most design teams, it is the second most anxious row in the table, and it rests on enough people to believe. Nobody made a chart of it.
Do this: Look for the biggest row, not the widest gap.
5. Do not build a career plan on 23 people
What got quoted: Researchers should own the evaluation layer for AI output or get left behind.
What the data says: That may well be good advice. It is not this survey's advice. This survey asked 23 researchers how they felt, and four of them were cheerful. The design engineer row holds 94 people, which is firmer, and still thin next to 862.
Do this: Keep the advice if you like it. Drop the citation.
What each row can actually carry
| Role | People who answered | Feel less secure | That is |
|---|---|---|---|
| IC Designer | 862 | 32.4% | about 279 people |
| Lead / Principal | 280 | 20.0% | 56 people |
| Manager / Director | 161 | 23.6% | about 38 people |
| Design Engineer | 94 | 10.6% | 10 people |
| Non-designer | 57 | 24.6% | about 14 people |
| Researcher | 23 | 39.1% | 9 people |
The audit
Next time a design stat lands in your feed, ask three questions. How many people are in that specific slice, not in the whole survey? Does the published methodology carry a warning the post dropped? Would one person changing their answer move the number by more than a point?
If the answer to the last one is yes, you are looking at an anecdote with a decimal place. The UX Tools team did the honest thing and published the row counts, the caveat and the API. Checking it took me one request.
Get the PDF