Assessment day

This posts serves to document what i feel went well and what could be improved in regard to the work presented on the 29th April.

Set-up

Successful -

1. All networking worked well, allowing efficient presentation of the media
2. HRMI device and all Arduino hardware worked straight away
3. HRMI heartbeat capture was strong

To be improved -

1. Blackout curtaining did not do what i wanted it too. I did not have enough material to make a closed 'cell-like' arrangement. I probably should have used the blackout space even though i found it overbearing because the media screens were very close to the participant. This may however have added to the immersive, intrusive, and intimate nature of the piece. The material did however hide the tables well so the space was darker.
2. The breathing sensor was difficult to fit and the way it attaches to the performer needs reassessing
3. All USB wires etc were a little too short in practice

Performance

Successful -

1. Showing the real time physiological changes in a performer was a smart choice in terms of presenting an initial impact and establishing a connection
2. Sound and visuals were successful and would work even better with minimal tweaks
3. The grandfather clock ticks worked well to give the heartbeat visual added value. If the vibrofeedback device worked it would have felt a lot more 'locked-on'
4. The 'shock' worked well as an overall feature in two specific ways. Firstly the 'button' worked well and delivered a shock to the leg of the performer. The shock provided a change in heart rate and clearly stimulated the performer visually and in most cases audibly! Secondly it played upon anticipation. This was achieved through the charging sound a few seconds before the shock itself, the performer squirmed as they knew it was approaching. The countdown also provided a longer set anticipation and in turn a heart rate change. The screen itself was presented so the performer could just about see it. This meant he knew roughly when he was about to be shocked again

To be improved -

1. Intimacy was not as effective with 5 people in the room. In practice a 1-1 experience would have triggered more anxiety and uncertainty about what was happening. The participant would have asked themselves more questions about the stimuli around them
2. The 'Acknowledgement of Death' certificate may have carried more weight in a 1-1 setting
3. The Heartbeat visual needs to be calibrated taking into account the elevated resting heart rate the performer experiences in the situation they are in. This requires a different scaling of the colours presented
4. The vibro-feedback system was omitted because it was not working correctly. This was a power issue and is now resolved. The omission however lost a big part of the relationship that the participant builds with the performer. This must be fully functional
5. I forgot to explain the functionality of the button presented to the participant
6. The Countdown screen although successful was situated slightly too high, therefore it was initially missed by the participant
7. There needs to be a clear ending to the piece without diluting the message. Originally i chose to 'kill' the performer once again, blaming this on one too many an electric shock. As the bringing to life (which was probably not too clear either) of the performer was meant to symbolise the reconnection of the relationship between you and your simplest internal physiology, would killing the performer send the message that over stimulation of this relationship will eventually destroy that relationship again? I'm not sure this is the message i want to convey

The piece will be refined and re-presented in June

Lee

C-P-Useless

Probably bought this on myself through a desire to saturate the viewer with video and audio! BUT if i run 3 max patches, 2 HD videos, 1 SD video, 5 samples and Module 8 together for some reason my MacBook Pro cannot handle the strain on its CPU......odd that :D Oh yer, i missed out constant serial communication....

OK, so the main issue is the HD heartbeat video. Standard playback is fine but I've found that continually changing the playback rate causes saturation of my laptops memory. This causes the video to jitter.

Luckily the problem has actually prompted me to think about how i am going to present the 3 different videos in my piece. The heartbeat video, the countdown video, and the live webcam feed. Ideally i need to present each video on a different screen and run in fullscreen (utilising presentation mode through the p-list editor on Max/MSP). I can only really do this by using either 2 external screens on one laptop (where i only have 1 video out so would require a splitter) or use 3 separate machines.

The latter is the solution i have gone for taking account of the performance issues. To solve, i found a way to send Max/MSP messages over a wireless LAN to control each videos playback from a master computer. Once i set up a new network using my Macbook as a host it worked perfectly within minimal lag-time.

HRMI interfacing

Have been using the Polar HRMI device for a while now.

Advantages

Accurate beat for beat sampling of your pulse rate

The ability to be accurately sampled by the Arduino

Disadvantages

Difficult to interface into Max/MSP through conventional methods

Requires close proximity and parallel alignment to the polar strap transmitter to work reliably

The HRMI interfaces with the Arduino micro controller through I2C protocols. Theses protocols rely on code within the Arduino environment to interpret the values (including averaging and smoothing) from the HRMI module. In this context i want to use raw heart rate data and manipulate it within Max/MSP therefore i have no need to utilise the averaging sections of the HRMI board. I found it more accessible to simply tap the output from the receiver part of the module giving a raw information from the transmitter in the form of small voltage pulses whenever a heart beat is received.

Averaging stages within Max

Once the values were sampled through the Arduino's digital inputs I producing a patch that created a running average of the last 4 values. I did this because although the heart speeds up and slows down quite erratically on a consecutive basis, it is difficult to perceive this unless you have a while to become accustom to a particular range. Using an averaging method, the outcome exhibits a typical heart rate across, say 5 seconds, where the value meanders around a central point. When the heart is stimulated the effects are more obvious as it rises quite quickly and falls a little slower then it normally would. In the context of my heart visual this allows the viewer to see that a change has occurred, giving them time to engage with what is happening.

Decode Digital Sensations Exhibition, Victoria and Albert Museum

Last week i went down the the V&A museum London to have a look at Decode, The digital design sensations exhibition, to try and get a bit of inspiration for future work, final projects etc. Heres a few of the pieces i thought were interesting.

Dandelion, by YOKE. This one is part of the interactive section. It uses IR detection and positioning to blow the seeds off a dandelion and almost like using a hairdryer, blow them around a virtual garden. It is extremely realistic and accurate.

Opto-Isolator, by Golan Levin. A mechanical eye that mirrors the viewers gaze. I like this because of its responsive nature. It opens up an intimacy unlike that of the other pieces.

Dune, by Daan Roosegaarde. This piece responded to you through a field of rods lighting up and insect like clicks as you passed beside it. It did not seem to respond as well as it probably should have though. No idea why.

Flight Patterns, by Aaron Koblin. I liked this because of it was one of the better visualisation pieces. It uses an averaged daily flight plan data to draw what is essentially the outline of the US. It also displays the time and number of flights in the air showing the very calculated nature of air transit and also commenting on mans intervention with the sky.

The visit definitely got me thinking about what different approaches i could take in my final project. A lot of the work was heavily coded and used bespoke programs (usually C++ based). I'm clearly going to have to think about spending some time in processing or Python....ouchh...

Food for thought.

Serial communication follow up

I have pretty much figured out now that the combination of Arduino analog sampling, some scripting within the microcontroller, serial communication, and max interfacing yields significant lag. Even with changing baud rates there is no getting away from the rounding errors which occur from the two stage sampling process (one at the Arduino board, and the other from max). This essentially means that the setup will only deliver averaged heart beat values where the data errors increase the faster the heart is beating.

Variables:

1. Method of communication is Serial. Does the information captured from the BVP device need to go to MAX straight away?

// Yes, i cannot do the signal conditioning out of the MAX environment. It would also be incredibly difficult to achieve using script within the Arduino programming environment.

2. Change the microcontroller?

// Possibly, although a change like that would basically be a back to basics scratch meaning i would have to interface a different microcontroller with MAX/MSP and figure out its programming language. After all this it may not be any faster as baud rates are set within the serial protocol.

3. Change the method of microcontroller capture?

// This refers to digital sampling rather then analogue. This is much more reliable and faster yielding greater accuracy. I tested this out by combining a switch with the 3.3V output from the Arduino. I noticed that no matter how rapidly i pressed the switch i got accurate unrounded heartbeat values exceeding 300 bpm. This however was a direct relationship between the Arduinos power supply and its digital inputs with no electronics slowing things down.

//This does mean however that if the HRMI (heart rate monitor interface) device delivers a 3v pulse then i should be able to achieve accurate readings. The issue here is whether the microcontroller will 'see' the 1ms pulses as they are extremely short. Another output on the HRMI gives a 6ms pulse. This may be more useable. Awaiting delivery.

4. Use the HRMI to produce a graphical display outside the MAX environment and use the result to affect MAX.

// For example. Have the HRMI produce a flashing light accurate to the pulse rate and sample that using a LDR. This of course would have to use the analogue inputs of the Arduino so may not be any better. The other option is to try and build a digital switch of some kind. The issue with all these ideas are that i cannot intergrate the Maxuino (digital and analogue pin to Max interface) with any scripting i would need to control the HRMI through the microcontroller.

I have however seen application in processing that use an Arduino and the HRMI to produce an accurate bmp reading over serial communication. Perhaps using that application i can use OSC (open sound control) to relay the readings to Max. For this however i think i would need a little help.

Time is pressing.

Testing time!

Its been a bit of a scary week in all honest in regard to this project. Plenty of realisations, compromise but in hand with this came a lot of learning.

I noticed my pulse sensor was working a bit oddly when max reported values of only 50, 54, 60, 66, 75, and 85 (some more onwards) seemingly the gaps were becoming further apart. I tried power supplys etc, trying to change almost every component, the arduino itself got replaced etc etc. No luck.

After a LOT of research and a lot of thinking i starting looking at other options including the Polar Heart Rate Monitor Interface (HRMI) board by Dan Julio. I got in contact with him and we talked about how the HRMI transmits data through ASCII code to other programming environments. We talked about other solutions to getting a stable heart rate into an Arduino and he also mentioned an inductance device that also picks up the the transmissions for a Polar strap sensor and outputs as a clean 3V signal.

He went on to mention how serial connections between the Arduino and environments (such as PD and MAX) experience lag and bad averaging. This seems to be what is happening here. Because of lag times the board cannot physically sample many more then around 10 samples per second. This makes sense as at 60bpm i was experiencing 10 different values but at 85bpm only around 8. Unfortunately the higher the heart rate, the higher the error which of course makes things difficult.

SO...

I am going to get a HRMI and try and get it working serially at first then attempt to interface it with Max/MSP via either logic level serial or I2C protocol. If Max will not accept such messages then i will have to make do with some averaging functions and use the work i already have. If this is the case i don't see a huge problem as the heart beat will still meander around 60-75 bpm until electricution events where it will jump up very quickly. The issue may be that the reported values at the higher heart rates will jump quite erratically between 90-120bpm. If this is the case i may change the way i report the visual, particually in regard to colour, where it works on a three tear scale, blue for resting (50-65bpm), yellow for active (70-85bpm), and red for stimulated (90-120bpm)

So im going to have a bit of a think i reckon before spending £50 on the HRMI and £15 on the polar strap. Up side is the devices will be much more stable in terms of reporting the heart rate (no need to signal condition etc like i'm having to do at the moment), and it is better attached to the performer. The current finger attached device has a tendency to react badly to lighting level changes.

Modul8 and Midi control

This week i've been experimenting with Modul8 and how triggering video transformation effects can add value to the visuals tied into the sensors.

The effects within the program are fairly easy to use and would work well alongside the respiratory sensor for example, which could pull apart the video feed (grid and distort in Modul8) in respect to the expansion of the chest through breathing.

I have found the best way to do this is to use Midi to control the relevant parameters in Modul8. This is easy when using a midi keyboard as the program maps physical midi signals automatically. It finds it difficult however to recognise virtual midi ports such as those set up by Max.

Scouring forums and the like it seems it is either possible through Pure data using Midipipes but i don't really want to start using another programming interface as this adds more potential problems. There are hints that Abelton live may help by sending midi through API. I can then use Max for live to open my patches and hopefully establish midi control that way.

If not i may have to use keyboard triggers instead, but although these can be automated in max, they cannot be assigned to faders and sliders in Modul8, only momentary switches

Sensor calibration

The weakest link of my system so far is the translation of data from the BVP sensor. I have found that there is only a small margin of error to capture a true pulse signal above thermal noise levels and nominal blood flow within the finger.

Left uncalibrated Max will either miss 'pulse' events or perceive the occasional burst of noise as a trigger. As the visual works on an average bpm (using a tap tempo), changing its colour in respect to a rising or falling average. If max recognises the data as two very close pulse events then this will be interpreted as a rapid rise in bpm and therefore colour.

I can see two ways of solving this. Firstly is to use the max window to observe directly what the data is doing. Within this data it is much easier to see what values to set thresholds at to provide the most accurate translation. This worked well and meant that there were only minimal instances where beats were missed (there tended not to be any unwanted events in close succession).

I now need to make some kind of averaging stage where if a beat is missed it will not affect the bpm. At the moment the only way i can think of doing this would be to compare the last bpm result with the present one and only accepting changes which are within a particular threshold. It is unlikely your heart will slow by half within a single heartbeat so this should work. To accommodate for higher pulse rates its important that the threshold is a proportion of the previous bpm value and not a constant number.

Functions could be similar to below where x is the previous bpm and y is the current value:

Accept y if - y >= (x - 1/3x) so at 60bpm y would be accepted at 40bpm or above

Reject y if - y < (x - 1/3x) so at 60bpm x would remain at 60bpm if y is at 39bpm or under

This allows higher heart-rates to accept more sudden drops, which is likely after sudden startling etc.

i.e. 120bpm can safely drop to 80bpm without being rejected. Just have to figure out how this would work within Max.

Max interface

Kind of works!

Had to do a lot of scaling within Max to get this to work with any degree of stability. Now with a little calibration and what a 'settling in period' (where the sensor sits on the finger for a minute to stabilise and detect) it works quite well.

The video above displays a very simple visual that represents the wearers heart-rate. The sensor seems to provide minimal lag.

To further improve its performance in terms of delivery of data i tried adding gain to the signal through an op amp but i experienced strange results. It may be that i need to use better quality components. However, as mentioned in previous posts i am keep to keep signal processing stages to a minimum as i am not using medical grade equipment and there is a good chance i will introduce even more noise to the system. For now it is stable enough to work with until another solution arises.

Mock up testing of Sensors

Several things bugged me.

1. Do the sensors work if powered without the base device

2. Will my extension connectors work i.e. soldered correctly, no significant leaks

3. What signals will return from the +sig and -sig outputs

4. What scale are the return voltages

I used a battery voltage supply of 12V as it was what i had lying around and guessed a swing of 6V with respect to ground wasn't too much more then 5V.... I then powered the sensors and just used a multimeter to read what returned from sensors. The respiratory sensor uses stretch sensors to change a resistance, therefore it works with relative voltage changes when the band is expanded. As it requires no calibration it works pretty well.

The BVP device looks as if it needs a little more calibration and care with what supply voltages it is given. The sensor returned voltages that did move between 120mV and 160mV in a pattern that did look as if it followed a typical heart rate. This needs clarification by using faster signal sampling. This would be provided by the Arduino. It was encouraging however to see that if you removed the sensor off the finger it stopped returning voltages out its signal channels.

Scale

Is a relative change of 20-40mV from the BVP enough to work with bearing in mind the Arduino's input pin resolution. The maths (5/1023) suggests a 4mV change can be detected however when you begin working within Max small changes like this are unstable and scaling them doesn't tend to yield great results.

It would be favourable to remove the need for any gain stages after signals are returned from the sensor however this may be inevitable. This would open up a whole new chapter of signal conditioning and require low noise op amps and a cleaner power supply. We will see!

Sensor Follow up

Extending the sensor connectors

Finally solved using some pretty tight soldering, insulation tape, jumper wires and several clamps.

Powering the Nexus Sensors

My last post now makes me laugh quite a bit with some of the solutions i was offering! After a little more research into potential difference (PD) i tried a few things.

Firstly i tried an op amp to try and invert 0v to 5v to give a -5v to 0 output. This doesn't work of course because you have to power the op amp with a PD enough thats high enough to swing between -5v and 5 volts which is of course 10V. I tried to do this using a few batteries. I then found out that i am not using a common earth by using two different voltage supplies. So i needed to use the Arduino to power everything. Luckily i later found out that the microcontroller has the facility to use an external power source. Using this knowledge i managed to provide a 0V, 5V, and a 14V supply from the device. Using a potential divider and i can pull the 14V level to 10V which give me a 10V PD. I then will use 5V as a virtual earth, 0V then translates to -5V, and 10V to 5V. This gives me the power and reference levels i need to power the Nexus sensors whilst providing a common earth.

However, when i get return signals from the BVP device (in particular because it requires calibration) it will be in respect to 5V as a virtual ground. As the Arduino only samples analog inputs from 0 to 5V at a resolution of 1023, i will have to again have to employ another signal conditioning stage to work within these levels. Of course with small voltage work it is important to try to avoid as many potential noise producing phases as possible. Hopefully though, if i am only using potential dividers the only real noise problems (providing good component placement and decent interconnect) will be thermal noise from resistors.

Next hurdle - The sensor

I'm happy with the visualisation i have now (will upload an example soon) and now its time to fully tackle the sensor connectivity problem.

The first stage, which i've already overcome, regarded the way i can use the sensor and manipulate it without destroying a £300 piece of hardware. After a little digging around i found the type of connector the nexus used for its auxiliary input, a LEMO 0B 5-pin. I found the tech sheets on it and after A LOT of looking around found a company which makes a female equivalent.

Next few problems -

Firstly the female connector needs extending so that i can work with the signals and power it correctly. At the moment the adaptor is extremely narrow. This maybe solvable using some jumper wires and some very delicate soldering.

Secondly the sensor needs powering. For the nexus tech sheets it shows that the auxiliary input to the amplification pack (power etc) provides +5, -5v, and 0v to the sensor. I am presuming here that the sensors require powering and that they are not passive. The BVP sensor (heart-rate monitor) works with Infrared so i guess they need powering. The issue is of course how much current to provide it with. From the tech sheets this suggests 10mA - clearly i don't want to damage the components. I need then to be able to provide a stable power supply. The Arduino only provides 5v and 0v so this means i may have to use an inverting unity gain amplifier to change the polarity of the signal. After this i should be able to pull the current levels down fairly easily if too high.

The third issue of course concerns the way we are able to detect if the sensor is actually doing anything. I have a digital multimeter but this will not be precise enough to figure out exactly what is going on. The final two pins of the LEMO connector provide a positive and negative signal return. Do i need to use both these? Does the base unit augment the signal using a difference amplifier using -sig and +sig as inputs? this would make sense however its a presumption.

Essentially i need an oscilloscope, a bit of solder, a couple of simple op amps for the power supply, and some patience.

EEG, Biodata performance examples

I thought it best to list a few examples of similar work to help describe the direction my current project is going in.

INsideOut - Claudia Robles - Germany 2009

The performer, who is surrounded by sound and images, interacts with them using an EEG (electroencephalogram) interface, which measures the performer’s brain activity. Those sounds and images -already stored in the computer- are modified consequently by the brain data via MAX/MSP-Jitter. Hence, the performer determines how those combinations will be revealed to the audience. Images are projected to a screen and also onto the performer, while sounds are projected in surround.

Empathizer - RobotLab - 2000

Linking human brain and robot

The installation 'empathizer' gave visitors the opportunity to experience an extreme interconnection between man and machine. The experimental environment consists of two industrial robots and a brain interface by which the visitors can connect with the machines.

Brain voltages are measured and analyzed. Then the data is forwarded to the robot control units. The image of the brain signals is projected on the robots. They give a representation of the data through their movements and always perform new and individual configurations - interaction without physical action.

Terrain01 - Ulrike Gabriel - 1993

The interactive AL-installation consists of cybernetic vehicles. the movements of the active 'life' depends entirely on the intensity of the light being projected onto the colony of robots. A brainwave sensor, placed on the head of the interactant, measures his or her brain activity, which is then sent to the system and controls, in turn, governs the intensity of the projected light. The more intense or erratic the viewer's brain activity, the less light strikes the robots and the more apathetic the behavior of the colony; or the weaker the brain impulses (the more relaxed the viewer), the more chaotic the movements of the robot colony become ...

Thought Conductor - Bruce Gilchrist et al. 1997-2001 various versions

Thought Conductor was inspired by the notion: what could have occupied the mind of the musician David Tudor when he performed John Cage's silent composition 4'33"? In reality he would have been preoccupied with keeping track of time, but this led to the idea of a musical performance where musicians are responding not to an arranged score, but instead to a direct manifestation of a conductor's thought processes on stage. For the Thought Conductorperformances, passages of musical notation remixed from the contents of a database were called up in the real time of the performance. "...score and sound aquire an immediacy which is characteristic of neuroscientific imaging in general, but which in this specific context lends new meaning to the notion of a 'live' performance. It is tempting therefore to suggest that Thought Conductor is a techno-scientific portrait that captures the inner kinetic melody of the individual who sits at its centre..."

Mariam Fraser, excerpt from abstract for Making Music Matter.

For the OX1 Festival the database archive made from the Oslo based composers was supplemented with The EEG files and notations from six Oxford based composers. The same procedures were followed. For the performance at the Hollywell Music Rooms several members of the festival audience participated. Again they were invited on stage, connected to the biomonitor and asked to engage in creative thought: an origami enthusiast recollected making a paper lotus flower; an accountant thought about a financial projection for a board of directors; a journalist recalled her last piece of arts criticism; an academic contemplated his tie...

Brainwaves and Plants - Miya Masaoka - 2002

This one is particularly odd

Presented as part of Lincoln Center Out of Doors, Homemade Instrument Day in New York, Pieces for Plants is an interactive sound installation for laptop, synthesizer, and the American semi-tropical climbing Philodendron. Versions of the piece have also been presented in a musical setting in which the plant participates as a member and soloist within an instrumental ensemble.

In the piece, a plant’s real-time responses to its physical environment are translated to sound. Highly sensitive electrodes are attached to the leaves of the plant. Scored movements by a human “plant player” stimulate physiological responses in the plant that are monitored via the electrodes and biofeedback wave analysis. The “plant player’s ” proximity, touch and interactions with the plant are then expressed in sound via midi and synthesizer. During the piece, the plant is brought to a range of physical/psychological states, from calm to agitation.

Dev8D developers conference

Have been at London's Dev8D since Wednesday learning a great deal about what can be done with software and particularly the interest from the side of the computer scientist into artistic applications of their work.

The whole event has been accommodating allowing both myself and fellow VRU technologists the opportunity to host a couple of arts based talks as well as involvement in workshops. One of which incorporated physical computing with programming. This has been invaluable because it has firmed by knowledge of microcontrollers and how to make use of sensors to control other devices.

I've picked up valuable knowledge about the amplification and signal conditioning aspects of my biofeedback project. I know know that i seriously need to take thermal noise into account when choosing resistive components as well as carefully positioning the precision analog Op amps. In terms of BVP (blood volume pulse) information I think i will only need a x10 gain to move a 10mV-200mV signal to something a little more noticeable to the Arduino analog inputs. These have a 5/1023 (v) resolution. Also from what i have noticed the Nexus-4 does not amplify analog sensors (except for unity gain reinforcement) after the sensor stage. This is interesting.

For now, the next step for me is to power the sensors (if needed, some are passive) and measure what signal i get from them, and work out the most effective amplification stage (as little and noise free as possible). I'm hoping for a respiratory and BVP measurement that this gain will not need to be an awful lot.

Asides from the electronics side I've done a lot into the visualisation side of things. This is effectively the visual feedback element and involves manipulating playback speeds and colour saturation of a heartbeat video. I will upload a quick example at some point in the next few days.

Tiring week but fully enriching.

Arduino take 1

Took delivery of an Arduino Duemilanove a few days ago now. Since then i've been trying to break the back of a few things i had to resolve to ensure everything communicates correctly.

Firsty -

Once amplified, the signals sampled by the Nexus sensors need to be recognised in Max.

Secondly -

Messages from Max need to be communicated out the serial port to the Arduino's digital pins. Once done external electronics can be controlled through Max and the microcontroller.

Thirdly -

There may be times when simple on and off messages will not provide enough control to the actuator electronic devices (hardware/electronics that are controlled by the Arduino and Max - as an end game). There needs to be some kind of Digital to Analogue conversion either through max or after the software stage.

Input/Output communications

From research in Tom Igoe's 'Making things talk' and looking around arduino.cc (playground) i came across two solutions. The first was the SimpleMessagingSystem (SYS) provided by max/MSP examples where the Max constantly samples input pin voltages from the Arduino and converts the resultant ASCII messages into something usable in Max. In this case the Arduino and software are locked in a symbiotic relationship where messages are constantly sent and received at a set baud/sample rate. I've come across this system before and i found it to be lag significantly when the Arduino was being driven. In addition the SYS is more of an voltage sampler then something that was designed for sending data to the Arduino and because of this the output stages are not that well designed.

Igoe, T (2007) Making Things Talk Cambridge: O'Reilly ISBN-13 9780596510510

Simple Messaging System @ arduino.cc

The second solution was offered form a generic communications protocol called 'Firmata'. Unlike SYS it uses the Arduino as a slave device, committing all communication actions to MAX (or PD/Supercollider depending on the application). I found this to be a better piece of firmware mainly because it allows simple representation of the Arduino's analog input pins and should (with some manipulation of the MAX patch) allow simple Digital communication with the Arduino's output pins.

Some additional information at Firmata.org

After some routing i managed to use the Firmata protocol to effective set up triggers in Max so that i could set the Arduino's digital pins high or low (logic 1 or 0, 5v or 0v) on command. This would suit such applications as the activation of vibration devices or essentially anything that requires a static DC signal.

Where things get more difficult is when we consider a device such as a heating resistor. A device such as this could be used, for example, to heat a chest strap attached to one person that mimics temperature changes or stress level changes in another person. As the source information (skin temperature/GSR) is constantly changing, the voltage driving the heating resistor would also have to change respectively.

Potentially i could use some kind of Pulse Width Modulation set up within Max to send intermittent and rapidly changing logic levels to emulate voltages between 0 and 5v. This works well with lighting where a pulse with half its cycle at logic 1 (5v) and half its cycle at logic 0 (0v) will appear as if the LED is lit at 2.5v. Light however has the ability to change its state near instantaneously, heat does not. Will this work? I do not know.

The other option of course is to use a regulate the voltage outside of Max using smoothing capacitors similar to AC-DC conversion. This however is more complicated and requires more electric components. More components in a live installation means more chance of something going wrong.

PWM

Example of PWM voltage averaging - Micromouseinfo.com

BioSensors - The input stage

I've been working recently with a biofeedback device called the Nexus-4. It is a physiological monitoring unit that measures the following:

Blood Volume Pulse sensor

(in order to achieve Heart rate meaurments)

GSR

(Galvanic Skin response or Skin conductivity - used to measure states of emotional arousal in lie detectors)

Skin temperature Sensor

(to a high resolution - again indicating stress levels)

Respiration Sensor

(relative expansion of the abdomen or thorax during inhalation and exhalation

The data is captured by the unit and displayed in various visual forms in a piece of software called Bio-trace.

The data could of course be mapped and used as triggers in some form or another however capturing raw data is tricky especially when considering medical data. What is being measured by the Nexus-4 is precise and subtle and if required in real-time requires some careful consideration.

The following flowchart helps explain the decisions required at this stage.

(follow this link to a larger version)

Biofeedback.....To begin...

I've been interested in the capture of Physiological data since I did my dissertation a couple of years ago. The work was on Music Therapy and how sound can be used to manipulate our autonomous functions such as heart rate, and breathing rate. Many physicians claimed to be able to regulate and in turn entrain these unconscious processes using music in order to relax, or stimulate the listener, depending on their end goal. I also spent some time looking at how neurological functions could be affected using similar methods of entrainment. This method used difference signals to produce slow frequency 'beating' effects that were meant to stimulate respective levels of alpha, beta, theta, and delta information within the brain.

Although the brainwave entrainment stuff is of course hazy and inconclusive, the physiological work, in terms of heart rate, and GSR (galvanic skin response) etc was quite effective. These are the areas I wanted to explore further in an artistic context.

How can I use physiological data and the ideas of Bio-Feedback within my practice? First of all I think i want to remove the unconscious aspects of our bodily rhythms to expose how precise and sensitive those processes are. I feel its important to appreciate such relationships and then potentially highlight how we are abusing them by intervening in our natural chemistry by introducing stimulants and depressants like nicotine and alcohol.

Finally, how would it feel if you were subjected to another persons physiological rhythm? If all external stimuli (sound, vision, touch) was working against your personal responses how would your body react? Would it feel awkward, or would it attempt to reach a resolution?