Examples using fully online / browser-based platforms
Example: Psychophysical task (UC Irvine)
Software and Platform
Google Firebase was used to host the experiment. Its free offerings are generous. We were able to collect data for more than 100 participants over the course of two weeks without purchasing higher storage or traffic limits. To give a brief account of the steps required to host an experiment on Firebase:
- A Firebase project is created using the Firebase web console
- The Firebase CLI (command line interpreter) is installed on the computer where the code is being edited.
- The CLI is used to login to the account associated with the Firebase project. The user may then specify the code files to be deployed to the web. A URL will be automatically generated on first deployment.
In addition to hosting, Firebase provides methods for storing data (e.g., spreadsheets of trial parameters and participant responses). These methods are branded as Cloud Storage. Some work was required to set up the experiment to call the Firebase storage functions correctly and save participants’ data in the desired format. The tutorials available on the Firebase website were helpful, but it took time to get everything running properly. Once everything was working, updating code was easy and performance was reliable. Both the Firebase web console and CLI can be used to update code, monitor website performance, and download saved data.
Participants were recruited from an institutional subject pool. A direct link to the experiment was posted on the subject pool website. Undergraduate students were offered course extra credit as compensation. A team of research assistants manually checked participants’ recorded data and dispensed credit.
From the front page of the experiment, a participant was able to download the Study Information Sheet (IRB materials). The front page also included rules for receiving extra credit as well as procedural warnings (e.g., “Do not refresh the page.”). The participant used checkboxes to confirm that
- they were using headphones or earbuds,
- they were seated in a quiet place with no distractions, and
- they agreed to take part in the study.
The experiment would not progress until all boxes had been checked.
The participant received instructions from a video with voice-over. The video described the nature of the stimuli (using a piano roll as reference) and the task required of the participant. This was followed by a practice block of the task.
Level calibration was not attempted (in giving consent, participants indicated that they were using headphones adjusted to a “comfortable” level and were in a quiet environment; this was sufficient for our purposes). The experiment asked participants to listen to pure-tone sequences and indicate whether each sequence contained frequencies from a musically major or musically minor set (single interval, 2AFC). Feedback was given after every trial. Participants could take breaks at their leisure, but they were expected to complete the experiment within one hour. The experiment itself did not enforce this time limit.
An easy semitone discrimination task was included among the conditions of the experiment. When supervision (e.g., via video call) is not possible, we recommend including catch conditions like this. Every listener with normal pitch processing was expected to perform this task easily; however, almost half of participants performed no better than chance in this condition. We suspect that this may be due reduced compliance outside of the lab setting as errors with the stimulus playback were not reported. These participants were excluded from the analysis. The distribution of the response variable (d’ sensitivity) was simulated for a guessing participant; the inclusion criterion was to perform above the 99th percentile of this distribution.
Several examples of code are available via GitHub (one may search on GitHub for “jspsych” or “hearing experiments”). The source code for the experiment described here has not yet been made publicly available, but the code for a simplified version is available via the link at the top. To learn more about the flow of the experiment (e.g., consent solicitations, demographic surveys, instruction set, etc.), the full experiment is playable here
Contact for more information: Sebastian Waz.