Randomising the order of your questions

Showing the questions in the same order for every interview can introduce biases in the responses to the questionnaire. Answers are often influenced by the order in which questions are asked, often favouring the first and last items, or questions asked earlier in the interview may influence answers given to later questions. If you have blocks of similar questions, respondents sometimes answer all the blocks in the same way. Randomising the order that the questions appear in an interview helps to reduce any answer biases.

Snap XMP provides a number of ways to randomise the order of your questions:

• You can define a group of consecutive questions that will be randomised. This is called a randomised block of variables.
• You can randomise several blocks of variables and this may contain further combinations of fixed blocks of variables or randomised blocks of variables
• Advanced options let you build complex randomisation with nested blocks.

In Snap XMP, you can also randomize the order of question choices in a closed question or the order of rows in a grid question.

Adding the randomised blocks

The randomised question blocks are created and edited in the Randomisation dialog, available in the Questionnaire Design window.

1. Open the questionnaire in the Questionnaire Design window.
2. Click the Randomise icon on the Snap XMP Desktop toolbar. This opens the Randomisation dialog. In this dialog, you can create blocks of variables that randomise the question order in the questionnaire.

3. Click the Add button to start defining your randomisation blocks. This opens the Block Details dialog.
4. There are 2 block types available:
   • A randomised block of variables: This lets you randomise a block of consecutive questions or variables.
   • Randomise several blocks of variables: This lets you build randomisation blocks containing more than one block of variables.

A randomised block of variables

This option lets you define one block containing consecutive questions or variables that will be shown in a random order.

1. Select the Block Type: A randomised block of variables. This enables the First variable and Last variable drop-down lists.
2. In the First variable drop-down, select the variable that starts the block.
3. In the Last variable drop-down, you will see all the variables that come after the First variable. Select the variable that finishes the block.

4. Click OK. The block appears in the list of Questionnaire blocks.

Randomise several blocks of variables

This option lets you define multiple blocks containing consecutive questions or variables that are shown in a random order.

1. Select the Block Type: Randomise several blocks of variables. This defines the main block that will contain other child or nested blocks.
2. Click OK. The block appears in the list of Questionnaire blocks. Now you can add other blocks of variables.

3. To add a block, select the Randomise several blocks item in the list of Questionnaire blocks then click the Add button. This opens the Block Details dialog.

4. There are 2 block types available. (You may see 4 block types when the advanced option is on.)
   • A fixed block of variables: This lets you define a block of variables that will be shown in the order set in the questionnaire.
   • A randomised block of variables: This lets you randomise a block of consecutive questions or variables.
5. Select the required block Type. The First variable and Last variable drop-down lists become enabled.
6. In the First variable drop-down, select the variable that starts the block.
7. In the Last variable drop-down, you will see all the variables that come after the First variable. Select the variable that finishes the block.
8. Click OK. The block appears in the list of Questionnaire blocks.

6. Repeat this process until you have entered all the required blocks.

Fixed Variables

When creating a group of randomised variables, you may wish to fix the position of some of the variables, such as headings or instructions that need to be shown at the top.

Variables within the randomised block can be set as fixed using the Fixed Variable setting.

1. In the Randomisation dialog, select the Block Type: A randomised block of variables. This enables the First variable and Last variable drop-down lists.
2. In the First variable drop-down, select the variable that starts the block.
3. In the Last variable drop-down, you will see all the variables that come after the First variable. Select the variable that finishes the block. This enables the Select button.

4. Click the Select button. This shows a list of all the variables in the randomised block.

5. Select the variables that you want to stay in the fixed position.
6. Click OK to save the changes.
7. The Details section shows the block information, including the fixed variables.

Removing blocks

1. In the Randomisation dialog, select the block that you wish to remove. If you select a block with child blocks, then this action removes all the child blocks as well as the block selected.
2. Click the Remove button.
3. You are asked to confirm the deletion. Click OK to remove the blocks, otherwise click Cancel.

Editing blocks

1. In the Randomisation dialog, select the block that you wish to edit.
2. Click the Edit button.
3. Edit the Block type, First variable and Last variable.
4. Click OK to save the changes, otherwise click Cancel.

Errors

When there are errors in the questionnaire blocks, you will see a red cross next to the blocks that have an error. If you save the blocks with an error, then the Randomise icon is shown with a red cross .

For example, the randomisation blocks cannot contain overlapping groups of variables. If this happens, a red cross is displayed next to the questions that overlap. When you select the block with an error, a description of the error is shown in the Details section.

Conflicts with routing and text substitution

If your survey uses routing or text substitution, you need to consider the effects of randomising the question order, as routing and text substitution use the answers to previous questions.

For example, if you create a randomisation block containing three questions: Q1, Q2 and Q3 where there is also routing set on the questionnaire that Q2 and Q3 are shown only when the first answer is selected in Q1 (Q1=1).

Here are all the possible combinations:

• Q1, Q2, Q3: randomisation OK (because it is the original sequence)
• Q1, Q3, Q2: randomisation OK
• Q2, Q1, Q3: randomisation cancelled because Q2 needs to be after Q1
• Q2, Q3, Q1: randomisation cancelled because Q2 and Q3 need to be after Q1
• Q3, Q1, Q2: randomisation cancelled because Q3 needs to be after Q1
• Q3, Q2, Q1: randomisation cancelled because Q3 and Q2 need to be after Q1

In this example, you can see that routing is prioritised and the randomisation is cancelled when Q1 is not first in the randomisation sequence. Removing Q1 from the randomisation block, would allow both randomisation of Q2 and Q3, and routing to work.

Examples

In these examples, the question numbers have been included to show the randomisation. Usually, you would not show the question number so the respondent is unaware that the questions are randomised.

Creating a randomised block of variables

This is a group of individual, consecutive questions that are shown in a random order when the respondent is completing the questionnaire.

1. Open the questionnaire in the Questionnaire Design window. This example has 5 questions about an instructor’s performance.

2. Click the Randomise icon on the Snap XMP Desktop toolbar. This opens the Randomisation dialog.
3. Click the Add button to start defining the randomised block of questions. This opens the Block Details dialog.
4. Select the Block Type: A randomised block of variables. This enables the First variable and Last variable drop-down lists.
5. In the First variable drop-down, select the variable that starts the block.
6. In the Last variable drop-down, you will see all the variables that come after the First variable. Select the variable that finishes the block.

7. Click OK. The block appears in the list of Questionnaire blocks.

When the questionnaire is completed by a respondent they will see the questions in a random order. From the question numbers you can see that the questions are in a different order than the questionnaire design.

Randomising several blocks of variables

You can randomise blocks of questions. The two options are

1. Randomising the blocks, but keeping the questions within those blocks in the original order
2. Randomising the blocks, and also randomising the questions within those blocks

Randomising several blocks that contain questions in the original order

1. Open the questionnaire in the Questionnaire Design window.
2. Click the Randomise icon on the Snap XMP Desktop toolbar. This opens the Randomisation dialog.
3. Click the Add button to start defining your randomisation blocks. This opens the Block Details dialog.
4. Select the Block Type: Randomise several blocks of variables. This defines the block that will contain the other blocks.

5. Click OK. The block appears in the list of Questionnaire blocks. Now you can add other blocks of variables.
6. To add a block, select the Randomise several blocks item in the list of Questionnaire blocks then click the Add button. This opens the Block Details dialog.

7. Select the block type: A fixed block of variables. The First variable and Last variable drop-down lists become enabled.
8. In the First variable drop-down, select the variable that starts the block.
9. In the Last variable drop-down, you will see all the variables that come after the First variable. Select the variable that finishes the block.
10. Click OK. The block appears in the list of Questionnaire blocks.

11. Repeat this process until you have entered all the required blocks.

Randomising several blocks that contain questions in a random order

1. Open the questionnaire in the Questionnaire Design window.
2. Click the Randomise icon on the Snap XMP Desktop toolbar. This opens the Randomisation dialog.
3. Click the Add button to start defining your randomisation blocks. This opens the Block Details dialog.
4. Select the Block Type: Randomise several blocks of variables. This defines the block that will contain the other blocks.

5. Click OK. The block appears in the list of Questionnaire blocks. Now you can add other blocks of variables.
6. To add a block, select the Randomise several blocks item in the list of Questionnaire blocks then click the Add button. This opens the Block Details dialog.
7. Select the block type: A randomised block of variables. The First variable and Last variable drop-down lists become enabled.
8. In the First variable drop-down, select the variable that starts the block.
9. In the Last variable drop-down, you will see all the variables that come after the First variable. Select the variable that finishes the block.
10. Click OK. The block appears in the list of Questionnaire blocks.

11. Repeat this process until you have entered all the required blocks.