Simulated feedback is a hypothesis, not a user study
A generated complaint can reveal a phrase worth testing. By itself, it does not establish how people behave, what a population prefers, or whether a product works for its users.
No persona model, dataset, product, participant, or user-study workflow was used. The example below is fictional and hand-authored. It demonstrates how to keep different kinds of evidence in different columns.
Four rungs that should not collapse into one
- 01
Input coverage
A phrase or interaction path exists in the test set. That proves only that the case was included.
- 02
System behavior
A deterministic observation records what the system did: parse, route, refuse, fail, or ask for clarification.
- 03
Simulated interpretation
A model may generate a reaction or explanation. Treat it as a prompt- and model-dependent hypothesis, not a person's testimony.
- 04
Real-user evidence
An appropriately consented study can record what actual participants did or reported, within its sampling and measurement limits.
A tiny synthetic example
Imagine a fictional reminder parser whose documented command is “remind me tomorrow at nine.” A hand-authored coverage set adds “tomorrow morning, ping me at nine.”
Phrase included
tomorrow morning, ping me at nine
Illustrated expected output · not executed
Under the fictional rule, the parser would reject ping as an unknown action. No parser was run.
Not collected
Empty. No model was asked to describe frustration, preference, intent, or identity.
Not collected
Empty. No participant used the parser and no user claim can be made.
The illustrated conclusion is narrow: under the stated fictional rule, this phrasing would produce a parser gap. Because no parser ran, this is a teaching example rather than an observed defect. It does not establish that people use this phrase, that the gap is common, or that fixing it improves usability.
Use simulated feedback to make questions
- Turn generated wording into a frozen test input, then observe the system separately.
- Keep generated interpretation out of fields reserved for participant behavior or quotations.
- Record model, prompt, persona definition, sampling rule, and run identity if simulation is ever performed.
- Do not convert frequency inside a synthetic sample into population prevalence.
- Escalate important usability, accessibility, preference, and safety claims to appropriately designed real-user research.
A blank record for the separation
The downloadable worksheet leaves simulator and real-user fields empty. It contains no modeled person, demographic profile, dataset row, generated opinion, participant record, or result.
Download the blank evidence worksheet →
Why mention simulated-persona systems at all?
The MatrAIx project repository and its arXiv preprint describe infrastructure for simulated-user evaluation. The repository explicitly positions the approach as useful for exploration, stress testing, and hypothesis generation—not as a replacement for evidence from real people. This article uses that narrow methodological distinction. It does not reproduce their dataset, run their software, repeat their reported scale or validation results, or make a dataset-license claim.
Primary sources checked. The evidence ladder, fictional parser example, and worksheet are original synthesis.