Overview

This article describes specific fraud detection features available in our Brown Qualtrics license as of March 2022. For more ways to deter fraud, you may also be interested in this other document with generally available ways to prevent fraud in Qualtrics, available to members of the Brown community.

These Qualtrics features allow you to collect embedded data along with survey responses; the embedded data marks whether the response is likely a duplicate, fraudulent, or a bot. You can either look at these values when you are cleaning up data or you can use them to branch within the survey, for example, immediately end a survey for someone who has already responded. 

Instructions: Survey Setup

1. Enable the features in survey settings

It's important to do this before people fill out your survey!

1. Open your survey and go to Settings > Security.
   

2. Enable Bot Detection
   
   

3. Enable Email Security Scan Monitor if desired
   
   

4. Enable RelevantID
   Important! If your survey is meant to be used more than once by the same participant, the relevantID will detect them as a duplicate. It’s still OK to collect it as long as you are not branching based on it. The setting does include a duration to reset the score. If you expect participants to take a survey periodically, you can choose a reset duration less than the frequency of survey taking.

5. Publish your survey

2. Look at the embedded data as you get responses

You can either view response details one by one, or add embedded data to your data & analysis view. It will automatically be recorded with every response (no need to add an embedded data element to your survey).

Here is what the embedded data means:

• Q_RecaptchaScore    Fraud Detection field that records when bot detection is enabled. Score returned using reCaptcha v3 technology. This is “invisible” recaptcha - it doesn’t require a recaptcha question (though if you already have one, you can leave it).
  • A score of 0.0 means the respondent is likely a bot.
  • Score of 1.0 means they are likely human. 
• Relevant ID improves fraud detection by assessing respondent metadata to determine the likelihood that the same respondent is answering over and over; Relevant ID does not necessarily check the content of the responses for duplicates, since respondent can answer multiple times while giving different answers. This feature has three fields associated with it that you can report on: Q_RelevantIDDuplicate, Q_RelevantIDDuplicateScore, and Q_RelevantIDFraudScore
  • Q_RelevantIDDuplicate Fraud Detection field that records when RelevantID is enabled. “True” means the response is likely a duplicate, no value means it is unlikely to be a duplicate.
  • Q_RelevantIDDuplicateScore Fraud Detection field that records when RelevantID is enabled. Number 0-100. How confident RelevantID is that the respondent is a duplicate. Higher is more confident.
  • Q_RelevantIDFraudScore Fraud Detection field that records when RelevantID is enabled. Number 0-130. How likely the respondent is fraudulent. A score of 130 means the respondent is extremely likely to be fraudulent.
  • Q_RelevantIDLastStartDate     - time the respondent started their last survey

3. Use Qualtrics’ Expert Review

1. Go to Data & Analysis

2. On the top right, look for Response Quality
   

3. Click it to view details