To get valid external measures of ScriptingRT’s precision, we built an device that can detect onset of light on a screen, and fire a solenoid to press a button.
This device can basically simulate a human participant in a simple reaction time task: The device waits for the onset of a stimulus (a white screen), and presses a button with a solenoid when the stimulus is detected.
We call this the Electro-Mechanical Turk (EMT)
The same device can also be used to only measure presentation times on a screen (i.e., without any button presses).
Here is a clip showing how it works. The screen in the background turns from black to white. The diode is attached to it. When the screen turns white, the solenoid fires, and the trial ends, with the screen going back to black. (In later tests, we fixed the solenoid vertically to a tripod, which is more stable.)
Below, we publish the schematic and the code of the EMT. We license this with a Creative Commons Attribution Share-Alike license, which allows for both personal and commercial derivative works, as long as they credit us and release their products under the same license. The development of the EMT was supported by FCT (project PTDC/PSI-PSO/101366/2008 “FlashDMDX”).
The EMT is based on an Arduino Uno. The photodiode is connected to it in combination with a voltage comparator, in order to allow for adjustment of the brightness levels. The solenoid we use needs 12 V – 1 A; because this is too much for the Arduino, we use an external power source and a transistor to switch it on and off. This circuit also has an LED added to show when the solenoid is firing, and a switch to turn the solenoid power on/off. In addition, we built in one LED and one switch that are directly connected to the Arduino and can be programmed from there (we use them to start and indicated measurement).
The circuit was implemented on an Arduino prototype shield. We used a Osram photodiode BPW 34 and an Intertec ITS-lz-2560 d-12vdc solenoid.
The circuit was developed and built by Gonçalo Paiva, Pedro Antunes, and João Fonseca in Lisboa. Everything was packed in a handy box with two buttons, two LEDs, one switch for the solenoid, and the potentiometer to adjust the brightness level. The diode is attached to a cord and can be placed on the screen. The solenoid we fixed to a tripod, in order to place it above a keyboard.
The Arduino sketch was programmed by Thomas Schubert. It’s straightforward: It waits for the button connected to the Arduino to be pressed and then starts measuring (signaled by the LED). The measurement signals a change in brightness when two consecutive readings differ from the previously detected state, and then fires the solenoid (or turns it off again). The Arduino does not provide any serial output.
Alternatively, there is an Arduino sketch that does not engage the solenoid. Instead, it uses only the photodiode to measure for how long the screen turned white (or, more precisely, how long it takes to switch from white to black, and back to white again). It collects the timings for 250 changes with microsecond accuracy, and then outputs them to the serial port. With this sketch, you can measure precision of presentation times in your software.
Note that the EMT is a cheap DIY setup; total hardware costs are around 150 €. It’s precision is necessarily limited, and we use it mainly to compare one device to another, and one software to another, rather than making claims about the absolute timing precision. Nevertheless, our results show that the EMT measures on average with millisecond precision.
We tested the EMT both measuring presentation times and reaction times, using various softwares and response boxes or keyboards. Our measures show that the minimal reaction time of the EMT itself is somewhere around 46 ms – this is the complete circle from detecting light, executing code in the Arduino, firing the solenoid, and then pressing the button. This may seem large, but a large part of it is simply due to the solenoid circuit and the solenoid itself firing. What is more important is the variability. The best response device that we have used so far, a serial response box from PST connected to E-Prime 2, showed that the total standard deviation of response latencies is about 1 ms – here including both the EMT and the response box and its measurement. The variability of the EMT itself is thus less than 1 ms.
When measuring presentation times only, we also get a satisfying precision. For instance, when we use DMDX to present 250 changes from white to black at the maximum rate of a monitor flatscreen monitor at 60 Hz (connected via HMDI), the theoretical length of each stimulus should be 1000 ms / 60 = 16.66 ms. The timing tool of DMDX, TimeDX, confirmed that each tick was 16.664 ms long. The average stimulus length measured with EMT was 16.629, with a SD of 1.109 ms.
We will add more results collected with the EMT soon.
If you are interested in building your own EMT and need more information, let us know. If you want us to build one for you, please also let us know.
If you want to buy something similar in a more professional version, check out the Black Box Toolkit.