Rating Response¶

A rating response allows participants to respond using a numeric scale.

The default setup is a 5-point likert scale, as in the below image.

If you preview or have launched a study before August 15, 2022, you'll see something like this:

Since August 15, 2022, the rating response has a brand new look and supports multi-row questions via row labels:

screen_shot_2022-08-12_at_11.19.15_am.png

Required Properties¶

type¶

UsageExample code

Definition: The type of response
Possible values: For a text response, type must be specified as "rating".

{
  "type": "rating"
}

This piece of code sets up a rating response in the form of a five-point likert scale. If this is what you want, there is no need to add additional code to this response!

Optional Properties¶

auto_lock_in¶

UsageExample code

Definition: Whether or not to automatically lock in a participant's response. If set to false, then participants need to click a seperate Confirm button to lock in their answer(s). We recommend leaving it at its default value true especially if you are sequentially presenting multiple rows of individual rating scales.
Possible values: true or false
Default value: true
See also: row_labels, sequential

{
  "type": "rating",
  "auto_lock_in": false // this is the default; you don't need to specify this
}

delay¶

UsageExample code

Definition: Delay the displaying of a response (relative to the latest onset of all stimuli on a given trial) by a certain number of seconds. This setting can be useful for forcing participants to read some text before making a response.
Possible values: Any numeric value. Fractions are supported.
Default value: 0

{
  "type": "rating",
  "delay": 1
}

duration_timer¶

UsageExample code

Definition: Whether to show a duration timer that indicates the time elapsed since the reponse becomes visible.
Possible values: true,false
Default: false

{
  "type": "rating",
  "duration_timer": true
}

duration_timer_onset¶

Dependencies

Requires duration_timer to be set to "true".

UsageExample code

Definition: When to show the duration timer. When set to "start", the timer will appear as soon as the response is displayed and animate a stopwatch effect. When set to "end", the duration timer will only appear once the participant has made a reponse with no stopwatch effect.
Possible values: "start", "end"
Default: start

{
  "type": "rating",
  "duration_timer": true,
  "duration_timer_onset": "start"
}

instruction¶

UsageExample code

Definition: A string of text providing instructions for participants about the current response.
Possible values: A string of text
Default value: "How confident are you in your response?"
To remove the instruction entirely, set instruction to "" (empty quotes).

{
  "type": "rating",
  "instruction": "How much do you agree with the above statement?"
}

In this example code, participants will see the instruction "How much do you agree with the above statement?" above the rating scale.

{
  "type": "rating",
  "instruction": ""
}

In this example code, participants will see no instruction above the text box.

labels¶

UsageExample code

Definition: Optional column labels that are associated with each step of the rating response. If not specified, a numeric scale will be displayed.
Possible values:
- A list of text labels to cosmetically replace numeric values on the scale. Note that participant responses are still recorded in numeric terms in your output data. The number of labels must be the same as the number steps in the rating scale - for example, if there are 5 steps, the list must contain five quoted text labels, such as ["Strongly Disagree", "Disagree", "Neutral", "Agree", "Strongly Agree"].
- A dictionary that maps text labels to numeric values on the scale (e.g., {"Label1": 1, "Label5": 5}). This option allow researchers to specify only the column labels that are relevant (e.g., "Strongly Disagree", "Neutral", "Strongly Agree") while omitting labels for intermediate steps on the scale. See example code for details.
Default value: no default

{
  "type": "rating",
  "instruction": "How much do you agree with the above statement?",
  "labels": [
    "Strongly Disagree",
    "Disagree",
    "Neutral",
    "Agree",
    "Strongly Agree"
  ]
}

Because labels is assigned a list of values, participants will see the names "Strongly Disagree", "Disagree" Neutral", "Agree", "Strongly Agree" in place of the numbers 1-2-3-4-5. In your data, a response of "Strongly Disagree" will be coded as 1, "Disagree" as 2, etc.

Using the dictionary format, we can omit "Disagree" and "Agree" and just present 3 labels:

{
  "type": "rating",
  "instruction": "How much do you agree with the above statement?",
  "labels": {
    "Strongly Disagree": 1
    "Neutral": 3,
    "Strongly Agree": 5
  ]
}

In this case, participants will see only the three labels defined here, positioned at the corresponding numeric values on the scale (i.e., 1, 3, 5) as defined.

max¶

UsageExample code

Definition: The endpoint of the numeric rating scale.
Possible values: Any number (decimals not supported)
Default value: 5
See also: min, reverse_scoring, zero_centered

{
  "type": "rating",
  "min": 3,
  "max": 6
}

Because min is set to 3, participants will see a rating scale whose lowest value is 3. Because max is set to 6, the scale's highest value will be 6.

min¶

UsageExample code

Definition: The starting point of the numeric rating scale.
Possible values: Any number (decimals not supported)
Default value: 1
See also: max, reverse_scoring, zero_centered

{
  "type": "rating",
  "min": 3,
  "max": 6
}

Because min is set to 3, participants will see a rating scale whose lowest value is 3. Because max is set to 6, the scale's highest value will be 6.

parent¶

UsageExample code

Definition: The name of another response. If specified, then the properties of the other response will be inherited by this response (unless a property of the same name is defined in this response).
Possible values: The name of another response

Given a response named "response1" as follows:

{
  "type": "rating",
  "max": "10",
  "min": "5"
}

You can avoid repetitive typing by defining a child response, "response2", that inherits all settings of its parent:

{
  "type": "rating",
  "parent": "response1"
}

reverse_scoring¶

UsageExample code

Definition: Whether to use reverse scoring in your output data. If set to true, a response of 1 on a 1-5 scale will appear as 5 in your output data, a response of 2 will appear as 4, and a response of 5 will appear as 1.

Note

The reverse_scoring property does not change the order of the scale that participants see. For example, even if reverse_scoring is set to true, a scale with a minimum value of 1 and a maximum value of 5 will still display the options in the order 1-2-3-4-5. Instead, the reverse_scoring property only affects the scores recorded in your output data. When enabled, it reverses the scoring logic, meaning that selecting a higher value on the scale results in a lower recorded score, and vice versa.
Possible values: true or false
Default value: false

{
  "type": "rating",
  "min": 3,
  "max": 6,
  "reverse_scoring": true
}

Because min is set to 3, participants will see a rating scale whose lowest value is 3. Because max is set to 6, the scale's highest value will be 6. And because reverse_scoring is set to true, responses in your output data will be coded in reverse order: a response of 3 will be coded as 6, a response of 4 coded as 5, 5 coded as 4, and 6 coded as 3.

row_labels¶

UsageExample code

Definition: Optional row labels, each of which corresponds to a separate rating scale in a rating response. This feature can be useful for asking a group of questions in the format of ratings. The option for revealing the individual rows sequentially or all at once can be further tweaked via the sequential property.
Possible values: A list of text labels, each of which appears to the left of the corresponding rating scale.

Considerations for conditional branching

If you need to conditionally branch your experiment based on participant responses from an individual row within a multi-row rating response, use the "response_name|row_label_text" format as the response name in a trigger.
Default value: No default. A default row label of 1 is always implied, meaning that the rating response will have just one scale, but the default text will be an empty string.
See also:: auto_lock_in, sequential

{
  "type": "rating",
  "instruction": "What do you like about your new computer?",
  "row_labels": [
    "Overall speed",
    "Speakers",
    "Webcam",
    "Weight"
  ]
}

sequential¶

Dependencies

Requires row_labels to be defined with more than one label.

UsageExample code

Definition: If multiple row labels are defined, whether or not to present each individual row of rating scale sequentially. When rows are presented sequentially, participants must answer the current question before the next one is revealed.
Possible values: true or false
Default value: false
See also: auto_lock_in, row_labels

Warning

Reaction times collected for multi-row rating responses are always measured relative to the onset of the first rating scale, even when the rating rows are presented sequentially. To calculate the "net" reaction time for a specific row, you can subtract the reaction time of the previous row from the reaction time of the row of interest.

{
  "type": "rating",
  "instruction": "What do you like about your new computer?",
  "row_labels": [
    "Overall speed",
    "Speakers",
    "Webcam",
    "Weight"
  ],
  "sequential": true
}

step¶

UsageExample code

Definition: The increment between two neighboring values on the rating scale.
Possible values: Any number (decimals not supported)
Default value: 1
See also: max, min

{
  "type": "rating",
  "min": 0,
  "max": 100,
  "step": 10
}

Participants will see an 11-point rating scale with 0 as the min, 100 as the max, and every 10-point increment in between.

target¶

UsageExample code

Definition: A single correct answer, or a list of correct answers, in the form of the underlying numeric values of the ratings (i.e., 1, 2, 3...). All target values must be within the range of the rating scale.
Possible values: A single numeric value like 1, or a list of values like [3,1,2] in the case of a multi-row response.

Note

When target is a list, participants’ choices must exactly match all listed targets to be considered correct.

{
  "type": "rating",
  "max": 5
  "target": 3 // if 3 is somehow defined as the "correct" rating
}

target_sum¶

Dependencies

Requires row_labels to be defined with more than one label.

UsageExample code

Definition: The required sum of all rating scores selected by participants in a multi-row rating responses. If the sum is not met, the response will flash and prompt the participant to revise their ratings.
Possible values: A numeric value that is between the smallest possible sum and the largest possible sum of a multi-row response.

Note

As this feature will prevent participants from going forward in an experiment unless their ratings add up to the target sum, it is highly recommended that clear instructions be provided to participants.

{
  "type": "rating",
  "row_labels": ["Question 1", "Question 2", "Question 3"],
  "max": 5,
  "target": 10 // any combination that adds up to 10 will suffice
}

zero_centered¶

UsageExample code

Definition: If set to true, the rating scale will be centered on 0, ranging from -max to +max.
Possible values: true or false
Default value: false
See also: min, max

{
  "type": "rating",
  "max": 6,
  "zero_centered": true
}

Because zero_centered is set to true, the rating scale will be centered at 0, spanning from -6 to 6 (the value of max).

Recorded Data¶

value: the rating selected by the participant (always numeric)
rt: reaction time, defined as the number of milliseconds between the start of the presentation of the scale and the moment when the participant responds (however, see the note for an exception in sequential).