Dev8D developers conference

Have been at London's Dev8D since Wednesday learning a great deal about what can be done with software and particularly the interest from the side of the computer scientist into artistic applications of their work.

The whole event has been accommodating allowing both myself and fellow VRU technologists the opportunity to host a couple of arts based talks as well as involvement in workshops. One of which incorporated physical computing with programming. This has been invaluable because it has firmed by knowledge of microcontrollers and how to make use of sensors to control other devices.

I've picked up valuable knowledge about the amplification and signal conditioning aspects of my biofeedback project. I know know that i seriously need to take thermal noise into account when choosing resistive components as well as carefully positioning the precision analog Op amps. In terms of BVP (blood volume pulse) information I think i will only need a x10 gain to move a 10mV-200mV signal to something a little more noticeable to the Arduino analog inputs. These have a 5/1023 (v) resolution. Also from what i have noticed the Nexus-4 does not amplify analog sensors (except for unity gain reinforcement) after the sensor stage. This is interesting.

For now, the next step for me is to power the sensors (if needed, some are passive) and measure what signal i get from them, and work out the most effective amplification stage (as little and noise free as possible). I'm hoping for a respiratory and BVP measurement that this gain will not need to be an awful lot.

Asides from the electronics side I've done a lot into the visualisation side of things. This is effectively the visual feedback element and involves manipulating playback speeds and colour saturation of a heartbeat video. I will upload a quick example at some point in the next few days.

Tiring week but fully enriching.

Arduino take 1

Took delivery of an Arduino Duemilanove a few days ago now. Since then i've been trying to break the back of a few things i had to resolve to ensure everything communicates correctly.

Firsty -

Once amplified, the signals sampled by the Nexus sensors need to be recognised in Max.

Secondly -

Messages from Max need to be communicated out the serial port to the Arduino's digital pins. Once done external electronics can be controlled through Max and the microcontroller.

Thirdly -

There may be times when simple on and off messages will not provide enough control to the actuator electronic devices (hardware/electronics that are controlled by the Arduino and Max - as an end game). There needs to be some kind of Digital to Analogue conversion either through max or after the software stage.

Input/Output communications

From research in Tom Igoe's 'Making things talk' and looking around arduino.cc (playground) i came across two solutions. The first was the SimpleMessagingSystem (SYS) provided by max/MSP examples where the Max constantly samples input pin voltages from the Arduino and converts the resultant ASCII messages into something usable in Max. In this case the Arduino and software are locked in a symbiotic relationship where messages are constantly sent and received at a set baud/sample rate. I've come across this system before and i found it to be lag significantly when the Arduino was being driven. In addition the SYS is more of an voltage sampler then something that was designed for sending data to the Arduino and because of this the output stages are not that well designed.

Igoe, T (2007) Making Things Talk Cambridge: O'Reilly ISBN-13 9780596510510

Simple Messaging System @ arduino.cc

The second solution was offered form a generic communications protocol called 'Firmata'. Unlike SYS it uses the Arduino as a slave device, committing all communication actions to MAX (or PD/Supercollider depending on the application). I found this to be a better piece of firmware mainly because it allows simple representation of the Arduino's analog input pins and should (with some manipulation of the MAX patch) allow simple Digital communication with the Arduino's output pins.

Some additional information at Firmata.org

After some routing i managed to use the Firmata protocol to effective set up triggers in Max so that i could set the Arduino's digital pins high or low (logic 1 or 0, 5v or 0v) on command. This would suit such applications as the activation of vibration devices or essentially anything that requires a static DC signal.

Where things get more difficult is when we consider a device such as a heating resistor. A device such as this could be used, for example, to heat a chest strap attached to one person that mimics temperature changes or stress level changes in another person. As the source information (skin temperature/GSR) is constantly changing, the voltage driving the heating resistor would also have to change respectively.

Potentially i could use some kind of Pulse Width Modulation set up within Max to send intermittent and rapidly changing logic levels to emulate voltages between 0 and 5v. This works well with lighting where a pulse with half its cycle at logic 1 (5v) and half its cycle at logic 0 (0v) will appear as if the LED is lit at 2.5v. Light however has the ability to change its state near instantaneously, heat does not. Will this work? I do not know.

The other option of course is to use a regulate the voltage outside of Max using smoothing capacitors similar to AC-DC conversion. This however is more complicated and requires more electric components. More components in a live installation means more chance of something going wrong.

PWM

Example of PWM voltage averaging - Micromouseinfo.com

BioSensors - The input stage

I've been working recently with a biofeedback device called the Nexus-4. It is a physiological monitoring unit that measures the following:

Blood Volume Pulse sensor

(in order to achieve Heart rate meaurments)

GSR

(Galvanic Skin response or Skin conductivity - used to measure states of emotional arousal in lie detectors)

Skin temperature Sensor

(to a high resolution - again indicating stress levels)

Respiration Sensor

(relative expansion of the abdomen or thorax during inhalation and exhalation

The data is captured by the unit and displayed in various visual forms in a piece of software called Bio-trace.

The data could of course be mapped and used as triggers in some form or another however capturing raw data is tricky especially when considering medical data. What is being measured by the Nexus-4 is precise and subtle and if required in real-time requires some careful consideration.

The following flowchart helps explain the decisions required at this stage.

(follow this link to a larger version)

Biofeedback.....To begin...

I've been interested in the capture of Physiological data since I did my dissertation a couple of years ago. The work was on Music Therapy and how sound can be used to manipulate our autonomous functions such as heart rate, and breathing rate. Many physicians claimed to be able to regulate and in turn entrain these unconscious processes using music in order to relax, or stimulate the listener, depending on their end goal. I also spent some time looking at how neurological functions could be affected using similar methods of entrainment. This method used difference signals to produce slow frequency 'beating' effects that were meant to stimulate respective levels of alpha, beta, theta, and delta information within the brain.

Although the brainwave entrainment stuff is of course hazy and inconclusive, the physiological work, in terms of heart rate, and GSR (galvanic skin response) etc was quite effective. These are the areas I wanted to explore further in an artistic context.

How can I use physiological data and the ideas of Bio-Feedback within my practice? First of all I think i want to remove the unconscious aspects of our bodily rhythms to expose how precise and sensitive those processes are. I feel its important to appreciate such relationships and then potentially highlight how we are abusing them by intervening in our natural chemistry by introducing stimulants and depressants like nicotine and alcohol.

Finally, how would it feel if you were subjected to another persons physiological rhythm? If all external stimuli (sound, vision, touch) was working against your personal responses how would your body react? Would it feel awkward, or would it attempt to reach a resolution?