Randomizing the order of question pages When you present a respondent with a survey, their answers are influenced by the order in which questions are presented. This is known as an order-effect. Because of this, you may wish to randomize the order of the questions, so that this effect is as small as possible. In Snap you can randomize the order of possible responses in a multi-choice question, rows in a grid question, or question pages. This worksheet explains how to randomize the order in which question pages are presented to the respondent using the Randomize (or Randomise) tool. For simplicity, it will be referred to as Randomize throughout this worksheet. Screenshots may show Randomise. Background In a list of options, people are more likely to choose the first option (primacy effect) or the last option (recency effect) rather than the middle options. In a list of questions, people will be more tired when they answer the later questions, and tend to put less effort into answering them. If you have blocks of similar questions, respondents will sometimes just answer all the blocks in the same way that they answered the first block. In a multi-choice question, you can randomize the order in which the possible answers appear (this is not a good idea if you are asking people to select an answer on a scale from Very poor to Very good, but can be helpful if you are asking people to select their favourite item). This is described in the topic Setting code rotation on a question. You can randomize the order of grid questions within Snap. This is described in the topic Question rotation. You cannot randomize the order of codes within a grid question. You can also randomize the order of pages of questions (so that the questions on the pages stay in the same order) by using the Randomize tool. (It is known as Randomise in the UK and Europe.) If your survey uses routing or text substitution, you cannot simply randomize pages, as the questions with routing and text substitution need to know the answers to previous questions. When randomizing pages in these surveys, you must take care to keep the order of these questions, otherwise the routing and text substitution will not work correctly. Summary of steps This worksheet assumes that you are creating a Snap WebHost survey. The steps will be the same if you are conducting an interview using one of the Offline outputs, such as kiosk, tablet, or PDA. If you have published a Web:HTML survey, the steps are different. You can find out more about Web:HTML surveys in the section Applying Randomize to a Web:HTML survey. If you are applying Randomize to a questionnaire with routing or text substitution, you will receive a message. Click [OK] to carry on. Step 1: Creating your survey This worksheet assumes that you have created a survey that asks peoples views’ on supermarkets. There are six supermarkets in the survey, and each has its own identical set of questions on a single page. After the questions on the supermarkets are a few demographic questions (You can get a trivial example of this type of survey if you click and download it to your computer from here). This worksheet describes how to randomly order the supermarket pages, while keeping the introduction page at the beginning and the demographic questions at the end. Create your survey, considering what order to have the questions in. Break it up into pages containing sets of questions that need to be presented in a specific order. Add any randomizing to question codes (using the variable properties Ordering option) or question grids. Publish your survey. Take note of the place where it is stored (the Path) and the Access Name (the name of the published file) as you will need to browse for it later. Step 2: Randomizing your question pages Open the Randomize program. You would normally do this by Selecting [Start] on the Microsoft toolbar, then type Randomize (this will be Randomise if you are in the UK) in the ‘Search programs and files’ box. An empty Randomize window opens. Click the [Browse] button and search for the published survey. If you are using the sample survey provided, you will be given a warning about text substitution. Click [OK] to carry on. The text substitution is on the fixed first page so it will not be affected by randomization of subsequent pages. Click [Add] to define a group of pages to randomise. The Add Group window opens.In this worksheet, the first and last pages of the 8-page survey must stay in place, and the middle six pages will be presented in random order. Set the first page in the group to 2, and the last page to 7. This leaves page 1 and page 8 in their correct places. (This worksheet is using a Snap WebHost example. If you are publishing to HTML, these instructions will not work. In Web:HTML surveys a list of pages in the survey will appear in the Group List pane after the survey has been selected.) Set the rotation format to Random. This means that the six pages are displayed in random order. Forward displays the pages in the order in the questionnaire, but choosing the first page at random, and looping round when you get to the last page until all pages are shown. Reverse displays the pages in the reverse order from that in the questionnaire, starting at a random page. Inverse randomly starts at either the first page going forward or the last page going backward (for Kiosk, PDA and tablet surveys, they will display alternately in forward and reverse order). Click [OK] to save your new group. It will appear in the Randomize pane. Click [Process] to make those pages appear in random order in the survey. If you change the survey within Snap and republish you will need to apply the Randomize settings again. Conclusion This worksheet has shown you how to randomize the order that pages are presented to the viewer for a simple Snap WebHost survey. This information also applies to offline surveys. For information about rotating pages in Web:HTML surveys see the section: Applying Randomise to a Web:HTML survey. The behaviour for some options in the Randomize tool varies according to the output that you are building. For example, in a web:HTML or Snap WebHost survey, respondents may complete surveys at the same time and it is impossible to give alternate respondents the questions in forward and reverse order. However, if respondents are completing surveys on a kiosk or a PDA, you do know the order of the respondents, and you can alternate the order for each respondent. You can find out more about rotating question codes in the topic Setting code rotation on a question. Randomizing the order of grid questions within Snap is described in the topic: Question rotation. If you would like to find out more about the effects of question order in surveys, there is much research available, for example, Bishop and Smith’s 1997 paper: Response order effects in public opinion surveys; The plausibility of rival hypotheses. There is help available for the Randomize tool if you click the [Help] button on the tool windows. If there is a topic you would like a worksheet on, email to snapideas@snapsurveys.com