The coherence score is an AI-driven metric that evaluates the plausibility and logical consistency of responses throughout an entire interview, from screening questions to socio-demographics. This highly robust quality check detects contradictions and inconsistencies, making manipulation nearly impossible. Unlike traditional trap questions—which fraudsters and AI bots can easily identify—the Coherence Score is a game-changer in fraud detection. It works seamlessly within regular questionnaires, identifying inconsistencies without relying on special questions. For example, it can flag contradictions, like a respondent claiming regular subway use while living in a city without one.

Additionally, the Coherence Score minimizes false positives by avoiding the disqualification of respondents who may have misclicked or momentarily lost focus but otherwise performed acceptably. This approach not only enhances fraud detection but also ensures more accurate and fair assessments.

​
How is the Coherence-Score calculated?

The Coherence Score evaluates the entire interview to assess its internal consistency:

  • Score 20 is assigned to interviews with multiple major contradictions, indicating very low quality and potential signs of fraudulent behavior.
  • Score 50 reflects interviews with several coherence issues, suggesting that participants may have been inattentive at times, leading to noticeable contradictions.
  • Score 80 is given to interviews with only minor inconsistencies, likely caused by brief lapses in attention.
  • Score 100 represents interviews that are fully consistent, logically sound, and free of contradictions.

What the Coherence Score Includes:

  • Explanation of the Rating: Provides a summary of the overall quality of the interview.
  • Describes how consistent or contradictory the respondent’s answers are across questions.
  • Highlights key factors that influenced the rating (e.g. logical flow, alignment of answers).
  • List of Incoherent Questions: Identifies specific questions where the respondent’s answers were inconsistent or contradictory.
  • Points out answers that conflict with earlier responses or are implausible in the context of the interview.
  • Offers valuable insights into areas of potential concern, helping to flag data reliability issues.

​
Use of GPT for Coherence-Score

ReDem uses GPT hosted on Microsoft Azure, deployed in the EU region to ensure full GDPR compliance when calculating the Coherence Score.

​
Individual Responses & Anonymity

Each interview is sent to Azure with a fully anonymized ID. Only the individual interview is transmitted per API request—entire survey datasets are never shared.

​
Exclusive Interaction with Azure

Only ReDem communicates with Azure. No information about the origin or source of the data is disclosed to Microsoft.

​
Data Residency

All data related to the Coherence Score is transmitted, stored, and processed exclusively within the EU.