Skip to main content
The coherence score is an AI-driven metric that evaluates the plausibility and logical consistency of responses throughout an entire interview, from screening questions to socio-demographics. This highly robust quality check detects contradictions and inconsistencies, making manipulation nearly impossible. Unlike traditional trap questions—which fraudsters and AI bots can easily identify—the Coherence Score is a game-changer in fraud detection. It works seamlessly within regular questionnaires, identifying inconsistencies without relying on special questions. For example, it can flag contradictions, like a respondent claiming regular subway use while living in a city without one. Additionally, the Coherence Score minimizes false positives by avoiding the disqualification of respondents who may have misclicked or momentarily lost focus but otherwise performed acceptably. This approach not only enhances fraud detection but also ensures more accurate and fair assessments.

​
High-level checks performed

This check looks at, for example:
  • Whether answers that should logically align (e.g., travel mode, days stayed, budget) are consistent with one another.
  • Whether the pattern and combination of responses across different questions fit a plausible scenario.
  • Whether any answers appear contradictory (for example: “I never use the app” in one section and “I use it weekly” in another).
  • These checks are intended to flag inconsistent or implausible response sets.

​
What the Coherence Score Includes:

  • Explanation of the Rating: Provides a summary of the overall quality of the interview.
  • Describes how consistent or contradictory the respondent’s answers are across questions.
  • Highlights key factors that influenced the rating (e.g. logical flow, alignment of answers).
  • List of Incoherent Questions: Identifies specific questions where the respondent’s answers were inconsistent or contradictory.
  • Points out answers that conflict with earlier responses or are implausible in the context of the interview.
  • Offers valuable insights into areas of potential concern, helping to flag data reliability issues.

​
Survey Description

The Coherence Score can be enhanced by providing a survey description that gives additional context about the purpose and intent of the survey. This optional field helps improve the quality and accuracy of the Coherence Score evaluation by giving the AI model better context about the survey’s objectives. When submitting CHS data points via the API, you can include a surveyDescription field that provides supplementary information about what the survey aims to measure or understand.

​
When Survey Description is Especially Useful

The survey description is particularly valuable when your survey asks about current events that occurred after the training cutoff date of the large language models. Examples include:
  • Recent openings or events: Newly opened facilities (e.g., a shopping mall that opened two months ago)
  • Current political events: Recent political developments, elections, or policy changes
  • Product launches: Newly released products (e.g., a new smartphone model)
  • Recent market changes: Current economic conditions or industry developments
In these cases, provide as much detail as possible in the survey description to help the AI model understand the context and make more accurate coherence assessments.

​
Example: Survey About a Recently Opened Mall

This survey evaluates shopping experiences at the Grand Central Mall, a new shopping center that opened in downtown Springfield opened in November 2025. The mall features over 150 stores including major retailers, restaurants, a cinema complex, and an entertainment area. The survey aims to understand visitor experiences, shopping patterns, and satisfaction with the new facility.

​
Use of GPT for Coherence-Score

ReDem uses GPT hosted on Microsoft Azure, deployed in the EU region to ensure full GDPR compliance when calculating the Coherence Score.

​
Individual Responses & Anonymity

Each interview is sent to Azure with a fully anonymized ID. Only the individual interview is transmitted per API request—entire survey datasets are never shared.

​
Exclusive Interaction with Azure

Only ReDem communicates with Azure. No information about the origin or source of the data is disclosed to Microsoft.

​
Data Residency

All data related to the Coherence Score is transmitted, stored, and processed exclusively within the EU.