Here you can learn about the different Requirement options available in a Gorilla Experiment, and about Time Limits you can set on Experiment completion.
Requirements allow you to restrict access to your experiment based on certain characteristics/conditions. There are four categories of restriction: Device Types, Browser Type, Location and Connection Speed. Any participant who doesn't satisfy the requirements you stipulate will be prevented from accessing your experiment. They will be sent to a default page explaining why they are not eligible to take part.
Requirements are set on a per Experiment basis. They can be viewed and changed from an experiments 'Recruitment' page. In your Experiment, select the 'Recruitment' tab.
Image above shows the Experiment Recruitment tab, from which you can change Experiment Requirements.
In the bottom right-hand quarter of the tab you will see the Experiments current Requirements. By default, these are set to 'No Restrictions' in each category. To change the Requirements, select 'Change Requirements'.
To learn more about each Requirements category, select from the options on the left-hand side.
To learn more about participant recruitment and the options available, check out the 'How Do I Recruit Participants?' page.
By default, participants can take part on all web enabled devices. You can optionally limit by device types by selecting the options that you want to accept. You can accept:
If participants try to access your experiment from a device type that is not allowed, they will reach a special page that tells them that their current device is not allowed and that they can try again later from a different device.
We use an external library to detect whether a participant is using a phone. This library has a list of phones which we then exclude. It's not - unfortunately - an exhaustive list so some phones do still get through.
Note 1: Modern Apple devices are able to 'spoof' the browser, with iPhone's and iPad's identifying themselves as Computers. We strongly recommend combining these device requirements with directly asking participants what device type they are using and branching within the experiment tree as appropriate. This will give you the best chance of correctly filtering users based on device type.
Note 2: If the participant clicks on a link from within a mobile app, the experiment may open within the app, rather than in a full browser such as Safari or Chrome. If this occurs, participants may not be able to complete tasks. We strongly recommend you ask participants to copy and paste the link for use on mobiles, rather than clicking it.
Image above shows the Requirements menu that appears when you click 'Change Requirements'. To access the Device Types settings, tick the box next to 'Device Types', then click on the options you want to allow.
The image above shows the Requirements section of the Recruitment tab once Device Requirements have been set. Here we can see that participants using Computers will be allowed to complete the experiment (the icons are green) but participants on Mobile Phones or Tablets will be denied access (icons are red).
Previewing what the experiment looks like on different devices is very useful in deciding what type of devices you want to allow for your experiment and what changes you should make to account for different devices your participants may use.
See our Device Preview Tutorial on how to use Chrome's browser developer tools to view what your experiment looks like on mobiles/tablets. You may also want to consider using the Layout Customisation settings in the Task Builder to optimise the layout of your task for different device types.
By default, all browser types are allowed.
There can be small differences in how different browsers display certain things. For example, how a dropdown menu looks or operates when clicked on. Or else how special characters are displayed. There can also be differences in the accuracy (number of significant figures) a reaction time can be recorded in from different browsers.
For some research, where these small differences have little or no significant bearing on the hypothesis under investigation, which brower type a participant uses will not be important. In these cases you will not wish to restrict participants based upon browser type.
However, for other research, where the element in question is fundamental to your hypothesis and you have detected a noticible difference between browsers, you’ll want to ensure that all participants use the same browser or selection of browsers when they undertake your experiment. A typical example of this case is, if you are using a code task which uses cutting-edge features that are only available in a specific browser, and therefore won't work in other browsers, you will want to limit your participants to using only this browser.
Note: If the participant clicks on a link from within a mobile app, the experiment may open within the app, rather than in a full browser such as Safari or Chrome. If this occurs, participants may not be able to complete tasks. We strongly recommend you ask participants to copy and paste the link for use on mobiles, rather than clicking it.
Image above shows the Requirements menu that appears when you click 'Change Requirements'. To access the Browser Type settings, tick the box next to 'Browser Types', then click on the options you want to allow.
The image above shows the Requirements section of the Recruitment tab once Browser Requirements have been set. Here, we can see under the Allowed heading that all browsers have been allowed, including unrecognised browsers, which has the same effect as setting no requirements.
By default, anyone in any location can take part in your experiment.
You may want to limit respondents by country. You can do this by providing a list of countries from which you will allow participants.
Some people obfuscate their IP Address, so that the country cannot be detected, you can decide whether to allow these participants or not.
If participants try to access your experiment from a location that is not allowed, they will reach a special page that tells them that their current location is not allowed and that they can try again later from a different location.
Image above shows the Requirements menu that appears when you click 'Change Requirements'. To access the Location settings, tick the box next to 'Limit by Locations'. Then, using the country code list as a reference, enter the two letter country codes of locations you'd like to allow participants to complete the experiment from, separated by commas.
The image above shows the Requirements section of the Recruitment tab once Location Requirements have been set. Here, we can see that only participants within Great Britain will be allowed to take part.
Gorilla uses a lookahead system to ensure that everything a trial needs to display properly (such as videos and images) are downloaded before the trial starts. This way, a participant's connection speed does not affect the accuracy of their reaction times.
The only scenario in which a participant may experience pauses between trials while the loading catches up is if it takes longer to load a trial than it does to perform one. This means that either their connection speed is too slow, or that the task is trying to show lots of large images or videos very quickly. You can solve this by either using smaller images or lower quality video, or alternatively by setting the required connection speed to be higher.
For the majority of cases, most modern connections will suffice, and so by default there is no restriction. However, if your experiment has some high-bandwidth elements, you should consider setting a minimum connection speed.
Image above shows the Requirements menu that appears when you click 'Change Requirements'. To access the Connection speed settings, tick the box next to 'Limit by connection speed'. Then enter the minimum connection speed participants need to have to enter your experiment.
Note that the connection speed detected by Gorilla will be different to the connection speed that a participant may detect for themselves using a service like Speedtest. Services like Speedtest determine the maximum theoretical speed of an internet connection in ideal conditions. This is useful for establishing whether your internet service provider is indeed providing you with the right connection speed, but this is always going to be greater than the connection speed experienced during normal use.
The test carried out using the Gorilla Connection Speed Test will generally be a more accurate representation of the participant’s connection speed under current conditions.
If participants try to access your experiment using an internet connection that is too slow, they will reach a special page explaining this and that they can try again later when their connection speed is better.
The image above shows the Requirements section of the Recruitment tab once Connection Speed Requirements have been set. Here, we can see that only participants with a minimum speed of 5Mbps will be able to participate in the experiment.
Time Limit, found on your experiment recruitment page, allows you to automatically reject participants who do not complete your experiment, or who take longer to complete it than is considered reasonable.
Once a Time Limit is set, in hours and minutes, participants who reach the Time Limit will be automatically rejected, but will be allowed to finish their current task, before being redirected to the Finish Node.
You may wish to set a Time Limit because ‘Live’ participants reserve tokens, contributing towards your recruitment target. This means that participants who drop out without finishing your experiment can prevent more participants from entering your experiment until they are rejected.
Whilst you can reject participants manually, this requires monitoring your recruitment progress closely. Instead, you may choose to set a Time Limit to automate this process.
We suggest setting a Time Limit that is far longer than it could reasonably take to complete your experiment. For example, If your experiment should take 15 minutes to complete, you might set your time limit at 2 hours.
For this reason, we do not recommend using Time Limits for longitudinal studies. In a longitudinal study, the reasons for taking a long time to complete a study are much more numerous, which makes the padding you'd want to give the time limit excessively large and hard to estimate. When you can see your attrition and rejection numbers, you may wish to revise your Time Limit, and would then have to manually include the participants you’d automatically rejected.
Additionally, depending on ethics and your recruitment service, you will likely still have to pay participants who only complete the first half of your study for completing the first section, so you may wish to make use of their data.
Note: When using the Time Limit with recruitment services that offer a similar Time Limit, make sure that your Gorilla experiment Time Limit matches the time limit set in the recruitment service.
