Understanding Variation in AI-Generated Answers
Where you'll see this
Feature | What it does | Can the result vary between runs? |
|---|---|---|
Instant Insights Engine | Answers a question you type, built from your records on the Instand Insights page, a Collection details page, or a Stakeholder details page | Yes, for open-ended or subjective questions |
Insight Generator | Writes a summary of a Collection or Stakeholder | Yes, for open-ended or subjective summaries |
This applies to features where you ask an open-ended question or request a summary. It does not apply to objective facts, which pull directly from your data rather than interpreting it.
Why answers vary
When you ask an open-ended question, such as "what are the top 5 insights" or "what are the most important themes," the AI has to interpret and weigh the information in your records, not just retrieve a fixed answer. There is more than one reasonable way to do that.
Think of it like asking a few colleagues to read the same stack of reports and summarize the main themes. You'd get a few different but equally reasonable summaries. Nobody did it wrong. They just weighed the information differently. The AI works the same way: it can read your records and reach a slightly different, still accurate, conclusion each time.
The larger or more open-ended the request, the more room there is for this kind of variation.
If you need an exact number or to find specific data, use Records or Stakeholders search and filters instead of asking the AI to count or search for you.
Getting more consistent results
Narrowing your question and adding more specficity reduces the room for variation, even on subjective topics.
Instead of | Ask |
|---|---|
"What are the top themes?" | "List the top 3 themes from oncologist interactions in Q2 2026, as a bulleted list" |
"Summarize this Collection" | "Summarize efficacy discussion of [drug/asset] in this Collection in 3 bullet points" |
"What's important here?" | "What adverse events were expressed most commonly in this Collection?" "What adverse events seem to have the most severe concern from stakeholders in this Collection?" |
Asking for a specific number of items and a specific format gives the AI less room to interpret the request differently each time.
What to expect
- Subjective or open-ended questions can return different, equally valid answers across separate attempts.
- The same question asked twice within a Collection or Stakeholder page can still vary because the scope is set but the interpretation is not.
- If two answers to the same question look contradictory rather than just differently worded, double-check against the reviewed records directly.
Troubleshooting
Problem | What to do |
|---|---|
Same question returns different themes or summaries | Expected for open-ended questions. Narrow the question or specify a format to reduce variation |
A count looks different between two answers | Confirm the number using Records search or filters, not the AI response |
Results feel inconsistent for a factual question | Rephrase using exact terminology from your records, such as an exact tag or keyword |
You need the same result every time | Use record filters and exact search instead of an open-ended AI question |