Assessment day

This posts serves to document what i feel went well and what could be improved in regard to the work presented on the 29th April.

Set-up

Successful -

1. All networking worked well, allowing efficient presentation of the media
2. HRMI device and all Arduino hardware worked straight away
3. HRMI heartbeat capture was strong

To be improved -

1. Blackout curtaining did not do what i wanted it too. I did not have enough material to make a closed 'cell-like' arrangement. I probably should have used the blackout space even though i found it overbearing because the media screens were very close to the participant. This may however have added to the immersive, intrusive, and intimate nature of the piece. The material did however hide the tables well so the space was darker.
2. The breathing sensor was difficult to fit and the way it attaches to the performer needs reassessing
3. All USB wires etc were a little too short in practice

Performance

Successful -

1. Showing the real time physiological changes in a performer was a smart choice in terms of presenting an initial impact and establishing a connection
2. Sound and visuals were successful and would work even better with minimal tweaks
3. The grandfather clock ticks worked well to give the heartbeat visual added value. If the vibrofeedback device worked it would have felt a lot more 'locked-on'
4. The 'shock' worked well as an overall feature in two specific ways. Firstly the 'button' worked well and delivered a shock to the leg of the performer. The shock provided a change in heart rate and clearly stimulated the performer visually and in most cases audibly! Secondly it played upon anticipation. This was achieved through the charging sound a few seconds before the shock itself, the performer squirmed as they knew it was approaching. The countdown also provided a longer set anticipation and in turn a heart rate change. The screen itself was presented so the performer could just about see it. This meant he knew roughly when he was about to be shocked again

To be improved -

1. Intimacy was not as effective with 5 people in the room. In practice a 1-1 experience would have triggered more anxiety and uncertainty about what was happening. The participant would have asked themselves more questions about the stimuli around them
2. The 'Acknowledgement of Death' certificate may have carried more weight in a 1-1 setting
3. The Heartbeat visual needs to be calibrated taking into account the elevated resting heart rate the performer experiences in the situation they are in. This requires a different scaling of the colours presented
4. The vibro-feedback system was omitted because it was not working correctly. This was a power issue and is now resolved. The omission however lost a big part of the relationship that the participant builds with the performer. This must be fully functional
5. I forgot to explain the functionality of the button presented to the participant
6. The Countdown screen although successful was situated slightly too high, therefore it was initially missed by the participant
7. There needs to be a clear ending to the piece without diluting the message. Originally i chose to 'kill' the performer once again, blaming this on one too many an electric shock. As the bringing to life (which was probably not too clear either) of the performer was meant to symbolise the reconnection of the relationship between you and your simplest internal physiology, would killing the performer send the message that over stimulation of this relationship will eventually destroy that relationship again? I'm not sure this is the message i want to convey

The piece will be refined and re-presented in June

Lee

C-P-Useless

Probably bought this on myself through a desire to saturate the viewer with video and audio! BUT if i run 3 max patches, 2 HD videos, 1 SD video, 5 samples and Module 8 together for some reason my MacBook Pro cannot handle the strain on its CPU......odd that :D Oh yer, i missed out constant serial communication....

OK, so the main issue is the HD heartbeat video. Standard playback is fine but I've found that continually changing the playback rate causes saturation of my laptops memory. This causes the video to jitter.

Luckily the problem has actually prompted me to think about how i am going to present the 3 different videos in my piece. The heartbeat video, the countdown video, and the live webcam feed. Ideally i need to present each video on a different screen and run in fullscreen (utilising presentation mode through the p-list editor on Max/MSP). I can only really do this by using either 2 external screens on one laptop (where i only have 1 video out so would require a splitter) or use 3 separate machines.

The latter is the solution i have gone for taking account of the performance issues. To solve, i found a way to send Max/MSP messages over a wireless LAN to control each videos playback from a master computer. Once i set up a new network using my Macbook as a host it worked perfectly within minimal lag-time.

HRMI interfacing

Have been using the Polar HRMI device for a while now.

Advantages

Accurate beat for beat sampling of your pulse rate

The ability to be accurately sampled by the Arduino

Disadvantages

Difficult to interface into Max/MSP through conventional methods

Requires close proximity and parallel alignment to the polar strap transmitter to work reliably

The HRMI interfaces with the Arduino micro controller through I2C protocols. Theses protocols rely on code within the Arduino environment to interpret the values (including averaging and smoothing) from the HRMI module. In this context i want to use raw heart rate data and manipulate it within Max/MSP therefore i have no need to utilise the averaging sections of the HRMI board. I found it more accessible to simply tap the output from the receiver part of the module giving a raw information from the transmitter in the form of small voltage pulses whenever a heart beat is received.

Averaging stages within Max

Once the values were sampled through the Arduino's digital inputs I producing a patch that created a running average of the last 4 values. I did this because although the heart speeds up and slows down quite erratically on a consecutive basis, it is difficult to perceive this unless you have a while to become accustom to a particular range. Using an averaging method, the outcome exhibits a typical heart rate across, say 5 seconds, where the value meanders around a central point. When the heart is stimulated the effects are more obvious as it rises quite quickly and falls a little slower then it normally would. In the context of my heart visual this allows the viewer to see that a change has occurred, giving them time to engage with what is happening.

Decode Digital Sensations Exhibition, Victoria and Albert Museum

Last week i went down the the V&A museum London to have a look at Decode, The digital design sensations exhibition, to try and get a bit of inspiration for future work, final projects etc. Heres a few of the pieces i thought were interesting.

Dandelion, by YOKE. This one is part of the interactive section. It uses IR detection and positioning to blow the seeds off a dandelion and almost like using a hairdryer, blow them around a virtual garden. It is extremely realistic and accurate.

Opto-Isolator, by Golan Levin. A mechanical eye that mirrors the viewers gaze. I like this because of its responsive nature. It opens up an intimacy unlike that of the other pieces.

Dune, by Daan Roosegaarde. This piece responded to you through a field of rods lighting up and insect like clicks as you passed beside it. It did not seem to respond as well as it probably should have though. No idea why.

Flight Patterns, by Aaron Koblin. I liked this because of it was one of the better visualisation pieces. It uses an averaged daily flight plan data to draw what is essentially the outline of the US. It also displays the time and number of flights in the air showing the very calculated nature of air transit and also commenting on mans intervention with the sky.

The visit definitely got me thinking about what different approaches i could take in my final project. A lot of the work was heavily coded and used bespoke programs (usually C++ based). I'm clearly going to have to think about spending some time in processing or Python....ouchh...

Food for thought.

Serial communication follow up

I have pretty much figured out now that the combination of Arduino analog sampling, some scripting within the microcontroller, serial communication, and max interfacing yields significant lag. Even with changing baud rates there is no getting away from the rounding errors which occur from the two stage sampling process (one at the Arduino board, and the other from max). This essentially means that the setup will only deliver averaged heart beat values where the data errors increase the faster the heart is beating.

Variables:

1. Method of communication is Serial. Does the information captured from the BVP device need to go to MAX straight away?

// Yes, i cannot do the signal conditioning out of the MAX environment. It would also be incredibly difficult to achieve using script within the Arduino programming environment.

2. Change the microcontroller?

// Possibly, although a change like that would basically be a back to basics scratch meaning i would have to interface a different microcontroller with MAX/MSP and figure out its programming language. After all this it may not be any faster as baud rates are set within the serial protocol.

3. Change the method of microcontroller capture?

// This refers to digital sampling rather then analogue. This is much more reliable and faster yielding greater accuracy. I tested this out by combining a switch with the 3.3V output from the Arduino. I noticed that no matter how rapidly i pressed the switch i got accurate unrounded heartbeat values exceeding 300 bpm. This however was a direct relationship between the Arduinos power supply and its digital inputs with no electronics slowing things down.

//This does mean however that if the HRMI (heart rate monitor interface) device delivers a 3v pulse then i should be able to achieve accurate readings. The issue here is whether the microcontroller will 'see' the 1ms pulses as they are extremely short. Another output on the HRMI gives a 6ms pulse. This may be more useable. Awaiting delivery.

4. Use the HRMI to produce a graphical display outside the MAX environment and use the result to affect MAX.

// For example. Have the HRMI produce a flashing light accurate to the pulse rate and sample that using a LDR. This of course would have to use the analogue inputs of the Arduino so may not be any better. The other option is to try and build a digital switch of some kind. The issue with all these ideas are that i cannot intergrate the Maxuino (digital and analogue pin to Max interface) with any scripting i would need to control the HRMI through the microcontroller.

I have however seen application in processing that use an Arduino and the HRMI to produce an accurate bmp reading over serial communication. Perhaps using that application i can use OSC (open sound control) to relay the readings to Max. For this however i think i would need a little help.

Time is pressing.

Testing time!

Its been a bit of a scary week in all honest in regard to this project. Plenty of realisations, compromise but in hand with this came a lot of learning.

I noticed my pulse sensor was working a bit oddly when max reported values of only 50, 54, 60, 66, 75, and 85 (some more onwards) seemingly the gaps were becoming further apart. I tried power supplys etc, trying to change almost every component, the arduino itself got replaced etc etc. No luck.

After a LOT of research and a lot of thinking i starting looking at other options including the Polar Heart Rate Monitor Interface (HRMI) board by Dan Julio. I got in contact with him and we talked about how the HRMI transmits data through ASCII code to other programming environments. We talked about other solutions to getting a stable heart rate into an Arduino and he also mentioned an inductance device that also picks up the the transmissions for a Polar strap sensor and outputs as a clean 3V signal.

He went on to mention how serial connections between the Arduino and environments (such as PD and MAX) experience lag and bad averaging. This seems to be what is happening here. Because of lag times the board cannot physically sample many more then around 10 samples per second. This makes sense as at 60bpm i was experiencing 10 different values but at 85bpm only around 8. Unfortunately the higher the heart rate, the higher the error which of course makes things difficult.

SO...

I am going to get a HRMI and try and get it working serially at first then attempt to interface it with Max/MSP via either logic level serial or I2C protocol. If Max will not accept such messages then i will have to make do with some averaging functions and use the work i already have. If this is the case i don't see a huge problem as the heart beat will still meander around 60-75 bpm until electricution events where it will jump up very quickly. The issue may be that the reported values at the higher heart rates will jump quite erratically between 90-120bpm. If this is the case i may change the way i report the visual, particually in regard to colour, where it works on a three tear scale, blue for resting (50-65bpm), yellow for active (70-85bpm), and red for stimulated (90-120bpm)

So im going to have a bit of a think i reckon before spending £50 on the HRMI and £15 on the polar strap. Up side is the devices will be much more stable in terms of reporting the heart rate (no need to signal condition etc like i'm having to do at the moment), and it is better attached to the performer. The current finger attached device has a tendency to react badly to lighting level changes.

Modul8 and Midi control

This week i've been experimenting with Modul8 and how triggering video transformation effects can add value to the visuals tied into the sensors.

The effects within the program are fairly easy to use and would work well alongside the respiratory sensor for example, which could pull apart the video feed (grid and distort in Modul8) in respect to the expansion of the chest through breathing.

I have found the best way to do this is to use Midi to control the relevant parameters in Modul8. This is easy when using a midi keyboard as the program maps physical midi signals automatically. It finds it difficult however to recognise virtual midi ports such as those set up by Max.

Scouring forums and the like it seems it is either possible through Pure data using Midipipes but i don't really want to start using another programming interface as this adds more potential problems. There are hints that Abelton live may help by sending midi through API. I can then use Max for live to open my patches and hopefully establish midi control that way.

If not i may have to use keyboard triggers instead, but although these can be automated in max, they cannot be assigned to faders and sliders in Modul8, only momentary switches