Frequently asked questions:

What is "remote testing?" How does it differ from conventional (in-lab) testing?

In the most general sense, remote testing is any data collection that take place outside of a laboratory setting. This could include field research or questionnaires filled out in the community setting, but our focus here is on experiments that involve presentation and response to auditory stimuli. This is not to be confused with research on parapsychology, which is sometimes described as remote perception testing. There are many advantages to remote testing, notably access to subjects who cannot come to the lab. Testing subjects remotely can save time that would otherwise be spent travelling to the lab and extend the population available for study. Remote testing also typically involves some loss of control — stimulus control, control over the test environment, reduced information above individual subjects, and reduced opportunities to monitor or reinstruct subjects.

In-lab testing administers response collection ("tasks") in controlled environments that minimize distractions, provide the necessary resources (hardware/software), and support the storage and analysis of response data. Remote testing, in contrast, has less control over the task environment, may utilize hardware/software resources that vary significantly across participants, and depend on new procedures for storing and transmitting response data from the participant to the experimenter.

What is it good for?

There are many advantages to remote testing, notably access to subjects who cannot come to the lab. Testing subjects remotely can save time that would otherwise be spent travelling to the lab and extend the population available for study.

What are some platforms for remote testing?

The Wiki section on Platforms includes extensive information about different types of available platforms, along with specific capabilities of individually identified platforms.

What approach should I use for remote testing?

There are really three big questions you need to answer when deciding how to set up an auditory experiment for remote data collection: what hardware will I use, what software will I use, and who are the subjects I want to test. In all three cases, the alternatives range from convenient and less controlled to more time consuming and well specified.

hardware: calibration & interfaces

  • loose control of auditory stimuli with respect to calibration & frequency response
  • flexible graphic and temporal specs
  • no “special” data collected (e.g., ambient noise levels)

Any user hardware

e.g., PC & headphones

or more controlled solutions (below)

  • stimulus level & frequency response defined within ~5 dB
  • controlled graphic and temporal specs
  • some specialized data collection capabilities (e.g., touch screen input)

specified hardware

e.g., iPad

or more controlled solution (below)

  • strict control of level and freq. response
  • other specialized measurement or interface required (e.g., calibrated ambient noise recordings, software permitting)

lab hardware

e.g., deliver tablet & sound level monitor

software: data handling and experimental control

  • standard procedures w/out modifications required

Preconfigured hearing research packages

e.g., PART

or more controlled solutions (below)

  • custom stimuli
  • standard interface and response sufficient

build-your-own systems

e.g., Gorilla

or more controlled solution (below)

  • custom procedures or real-time processing
  • non-standard interface or data collection (e.g., voice recordings, hardware permitting)

fully custom scripts

e.g., MATLAB or Python

subjects: demographics & instruction

  • anyone can participate
  • no one-on-one instruction required

anonymous & unsupervised

e.g., Mturk

or more controlled solutions (below)

  • targeted population
  • specialized instruction (e.g,. via zoom)

“by invitation” access

most hardware/software will work

  • populations requiring real-time supervision (e.g., children)
  • protocols requiring rigorous training

supervision by proxy (e.g., parent)

experimenter-administered protocol hybrid model (e.g., training in person, data collection at home)

most hardware/software will work