SchedulingAndPayments
1. Scheduling and Payments
| Platform name: | Does this platform support visit scheduling/logging? | Does this platform support participant recruitment? | Does this platform support participant payments? | Does this platform support online consenting? | Can the platform be linked to services that handle scheduling, recruitment, and payment (e.g., university subject pools, MTurk, etc.)? |
| Amazon MTurk | No | Yes | Yes | Yes | No |
|---|---|---|---|---|---|
| Cognition.Run | No | No | No | No | Yes |
| Django | No | No | No | Yes | Yes |
| Gorilla | No | No | No | Yes | Yes |
| hearX | No | No | |||
| ispring | Yes | No | No | No | No |
| Jacoti | No | No | No | No | No |
| jspsych | No | No | No | Yes | Yes |
| MATLAB | No | No | No | No | Yes |
| PART/BGC Science | No | No | No | No | Yes |
| Prolific | Yes | Yes | Yes | No | N/A |
| Psychstudio | N/A | N/A | N/A | Yes | Yes |
| PsyToolkit | No | No | No | Yes | Yes |
| Qualtrics | Yes | Yes | No | Yes | Yes |
| SHOEBOX | No | No | No | No | No |
| SpeakPipe | No | No | No | No | No |
| TabSINT | No | No | No | Yes | No |
| TeamHearing | Yes | No | No | No | No |
2. If scheduling or recruitment are natively supported, how are visit logs and payment records stored/obtained?
| Platform name: | Integrated server/downloadable | Integrated server/non-downloadable | Written to another online/cloud server (list compatible options in comments) | Written to a lab/institution server | Written to testing device |
| Amazon MTurk | REQUIRED | — | — | — | — |
| Cognition.Run | — | — | — | — | — |
| Django | — | — | — | — | — |
| Gorilla | — | — | — | — | — |
| hearX | — | — | — | — | — |
| ispring | — | OPTIONAL | OPTIONAL | OPTIONAL | — |
| Jacoti | — | — | — | — | — |
| jspsych | — | — | — | — | — |
| MATLAB | OPTIONAL | OPTIONAL | OPTIONAL | OPTIONAL | OPTIONAL |
| PART/BGC Science | — | — | — | — | — |
| Prolific | REQUIRED | — | — | — | — |
| Psychstudio | REQUIRED | — | — | — | — |
| PsyToolkit | REQUIRED | — | — | — | — |
| Qualtrics | OPTIONAL | — | — | — | OPTIONAL |
| SHOEBOX | — | — | — | — | — |
| SpeakPipe | OPTIONAL | OPTIONAL | OPTIONAL | — | — |
| TabSINT | — | — | — | — | — |
| TeamHearing | OPTIONAL | OPTIONAL | OPTIONAL | OPTIONAL | OPTIONAL |
3. Miscellaneous comments on recruitment and scheduling that would help clarify or expand on responses:
| Platform name: | |
| Amazon MTurk | There is a LARGE subject pool on Amazon Turk of all sorts of types. Depending on the task, and the difficulty, I often get a full experiment completed with dozens/hundreds of subjects in an hour or two. |
| Cognition.Run | The website supports linking to/from Prolific, as described in this video: https://youtu.be/O628dNA7WCc |
| Django | The platform can be used to design web servers that implement whatever logic/flow we want. It can thus log visits to the server by participants and place scheduling constraints on participation. However, recruitment and payment are wholly separate problems and falls to the user. |
| Gorilla | There is a variety of mechanisms of recruitment allowed on Gorilla, including through third-party platform to unique or general URL to online experiment. There is no scheduling functionality that I am aware of. Participants can access the URL link anytime. The platform can also run multi-session longitudinal/training studies, which may allow functionality such as minimum/maximum time between sessions (which I have not confirmed). |
| hearX | Not clear if the platform can be adapted or modified to allow for subject payment and consent. Clinical test device. |
| ispring | The platform is primarily for online e-learning. It has a companion service ispringlearn.com (with a separate subscription) which allows for created tests to be posted there and then participants and their data could be tracked and they can be enrolled with their own individual accounts. |
| Jacoti | This platform is meant to be a direct to consumer product. However, hearing professionals and employers can invite patients/employees to use the "JacotiHearingCenter" and "JacotiListenApp" to track their hearing and fit their hearing profile in the app. No mention of consent and no formal scheduling interface. |
| jspsych | This is a purely front-end library (i.e., control how things appear in a browser). Thus, a consent page can be displayed. They have rudimentary support for integrating with MTurk, but not with other platforms. |
| MATLAB | There is no pre-configured support in MATLAB for scheduling/participant management, but it is possible for those systems to be developed by individual labs. It should be possible to link a MATLAB web app as an external task in MTurk. |
| PART/BGC Science | PART does not have built-in support for scheduling, but there is a Brain Game Center web-portal linked to the client-side version of PART (called BGC Science) that supports recruitment and online consenting, as well as sends emails containing session progress/completion data. In addition, session scheduling can be handled through BGC Science, which can set times at which various test batteries are available to the user. |
| Prolific | Participants recruited using Prolific are typically redirected to an external website where the actual consenting and other task completion will happen. Upon completion, a confirmation code can be provided by the external website that participants can submit to Prolific to get paid. The researcher using Prolific can approve or reject participant submissions and pay bonuses if they choose to. |
| Psychstudio | Automatic referral from and redirection to Positly, Prolific and Sona-systems |
| PsyToolkit | This is really intended to be the software in which the experiment is coded. You can embed experiments in a survey that is sent to a participant to complete online and which can be linked to in other scheduling programs. |
| Qualtrics | Using Qualtrics to recruit will cost extra. |
| SHOEBOX | I don’t think it is capable of adding on these applications, but they have an active support team. Perhaps these could be added to future platforms. |
| SpeakPipe | |
| TabSINT | |
| TeamHearing |