How to repeatedly assign participants to conditions in Qualtrics

In one of our studies using Qualtrics, we came across the problem of randomly assigning participants to different experimental conditions repeatedly.

The point was that we wanted everybody to experience a manipulation depending on condition, then have everybody fill in the same measures, and then present some more materials again depending on condition.

Randomizing the experimental conditions ONCE is easy, but how to do that repeatedly?

Our quite simple solution was to create embedded data fields that store the assignment to condition, and then branch according to that field.

To do this, in the survey flow you set up in the beginning a randomizer

Click survey flow and add a new element: Randomizer

Under this element, add the conditions you want to randomize by pressing add new element here: Embedded data.

On the embedded data element, type “condition” and the label of the condition (e.g. number) on the value field. Do this for every condition.

Go back to the randomizer and choose to “randomly present 1 of the following elements” and tick “evenly present elements“.

At this point, it should look like this (example with four conditions):

Below, add a new element: Branch

In branch, select “If” Embedded Data, type “condition” and select Is Equal To, and type the name/number of the condition

something like this:

Under the branch add a new element, that will be the block containing the desired stimulus for that condition

Do the same for all the experimental conditions.

it should look like this (again, example with four conditions):

In this example, the Blocks Entitativity and ManCheck are done for all participants in the same manner.

After that, you can then repeat the branching and do more condition-specific manipulations or measures.

If you have the problem that there is only very little change between your conditions, for instance a different label, there is alternative: Store the label itself in an embedded data field and insert it with Piped Text.

(written by Pedro Marques and Thomas)

Advertisements

21 thoughts on “How to repeatedly assign participants to conditions in Qualtrics

  1. thx for this post. I randomized my experimental conditions based on your suggestions. However, Qualtrics does not present the embedded data (=conditions) evenly. My study uses eight different groups (=conditions). So far, for instance, condition 8 was presented five times, but condition 7 only once.

    Why does that happen? How can I solve the issue? (I am sure that I ticked the box “present elements evenly”).

    It is very important that I have about the same number of respondents for each condition.

  2. Hello, I guess I’m just a beginner because you said: “Randomizing the experimental conditions ONCE is easy”. My question is how do I do the easy part? I hope you can help

    1. you put the two conditions in two different blocks, go to the survey flow, create a randomizer, and move the blocks underneath that randomizer. if your manipulation is between, you tell the randomizer to present only one element; if it is within, you present both.

  3. Hi, I also had four conditions for my experiment and I did exactly how it is guided here. However, when I preview my survey, the survey always skips the “randomized” conditions and jumps to the end of the survey – recording the answers and stuff. Any suggestions? Thanks

    1. Thanks for replying so fast, this is great since I am now working for my Master’s thesis.

      Now, back to my conundrum, I tried to tweak a bit the survey flow, it turned out I had to put the randomizer before the embedded data set and also before the branch. Now it works like charm.

      Previously, I put the branch right below the embedded data without putting an additional randomizer. If this is how it is supposed to be, just a feedback, perhaps you should edit your post starting from “Below, add new element:branch..” because I presumed I did not have to put another randomizer.

      Also, for the last block(s) that are not included to one of the conditions – showed to all respondents regardless of the conditions – it should be put on par not with the randomizer. Because, from what I saw on the second screenshot, you put the entitativity and mancheck blocks on the same level with the randomizer, did you not?

      Anyway, thanks for the helpful post and comment. This saved me big time! ๐Ÿ™‚

  4. Hi Thomas,

    I have another question: I randomly assigned participants to a condition (4 options possible). I downloaded the data to SPSS, but where can I find which participant had a specific condition? I hope I did not missed anything in advance.. Regards, Berbel

    1. Hi Berbel, I’m currently facing exactly the same issue. Can you tell me how you solved this? Thanks, Kristina

  5. Thanks for this information.

    I am currently setting up Qualtrics to allocate people that sign up into three groups for a RCT.

    I would like to select “evenly presents elements” as roughly even allocations would be great. As this is no longer “simple randomization”, I assume it is a form of “restricted randomization”

    My question is: is the algorithm consistent with:

    – Permuted-block randomization or blocked randomization;

    – Adaptive biased-coin randomization methods (of which urn randomization is the most widely known type): The probability of being assigned to a group decreases if the group is overrepresented and increases if the group is underrepresented.

    – Or something else?

    I guess I will need to know this for my paper.

    Thanks again for your help
    Alison

      1. I talked to Qualtric about the above. If you use the “evenly presents elements” it is not true randomisation. E.g if you have three groups, the first person is randomly assigned to one group out of three, the second person is randomly assigned to one of the two remaining groups and the third person is put into the remaining group. I don’t know what this is called, but it is not deemed “true” randomization.

  6. Hi there! I checked the box ‘evenly present elements’, but still in one of my condition (out of 4), there’s no more than 17% of the people in it, and in another one there’s over 30%. This bothers my data.. ๐Ÿ˜ฆ How can I recover it? Is it possible to just assign the next coming respondents into the condition (in which I need more people)? Thanks!

  7. it’s likely that people cancel the study more often in one of hte conditions. check your qualtrics for unfinished datasets. I would advice against playing around with the randomization, there is too much risk to confound the conditions with whatever is changing. better try to find out why people are cancelling one condition more often.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s