After presenting the hacked optical mouse to a few people at HONF during the cellsbutton#04, Yogyakarta International Media Art Festival, I have been working with Budi aka vj manticore, from DeadMedia.fm on improving the pd patch to use GEM for the visualization. Muuuuuch better now, than my earlier canvas spaghetti code, thanks a lot Budi!
With the new implementation we can now use the whole spectrum of live image processing and analysis, like openCV for motion tracking, or additional sound synthesis. hope to jam the worms soon…
Niiiiice, I am back to cellsbutton#04, organized by HONF… the best festival ever!
The first few days were already busy, lots of interesting workshops, parties and discussions. also i met a lot of people, both new but also good old friends. I already got some first soldering action, and tried ways to make a diy version of the diy makeaway bitbadge. sadly i got the wrong site of pcb and cant use the shim for screen printing the solder paste. but with some patience we managed to get a few going…
also had some talks and workshops about new hackteria tools, such as the hacked optical mouse and the hacked PS3 eye mounted on an old microscope stage. i worked with akbar, a young microbiologist from UGM, on it and tested the DIY3 eye microscope for use with a haemocytometer to count yeast cells, which we then presented at the workshop at UGM later.
I was also happy to finally meet Georg Tremel again, from biopresence. He got some great recent work on DIY plant tissue culture and uses it with a blue flower, which is genetically modified for aesthetic reasons. He also did a workshop at UGM and hopefully the “Moondust” will grow there soon… and we got to have him collaborating for hackteria soon!
For more information about this project check the recent posts on this blog.
I just updated the pd-patch and the serial communication with the arduino. it now allows the frame-grabbing of the full 18×18 pixels from the mouse sensor, adns2610, with a reasonable framerate of 8fps. see download link below.
in addition i used the hacked optical mouse to detect the motion of nematodes, anguila aceti, and got some really clear results, both in the quality of the image and a substantial change in the SQL value. due to some grabbing issues when recording the screen the values are a bit delayed in the image above.
Just done further experiments with the hacked optical mouse sensor, in this case a Avago ADNS2610, from a cheap wired logitec optical mouse. So i managed to connect the optical sensor to the arduino and send all the data, including the image from the 18×18pixel sensor, through a serial connection to puredata (pd). The sensor is a real high-end device, it calculates the optical flow, thus gets out motion and the speed of the mouse, it detects focus by a SQL-value, which is proportional to the number of features the sensor detects on a surface. and other values such as maximum and average pixel and shutter speed can be read from the registers.
The frame rate of the transmitted image is sadly relatively low, cos the ADNS chip is not meant to present this info at higher speed. still its reasonable, when reading all 18×18 pixel a framerate of roughly 5 fps is feasible, if you only dump the first couple of lines you can increase it to higher framerates.
After building a patch to visualize all these data from pd, i started experimenting with placing microorganisms on top of the sensor, keeping the optics intact, but replacing the LED with an RBG, so i can control illumination.
I got some really amazing results, it seems i can detect single cell microorganisms using just the mouse sensor and an arduino. see the movie below. while the sensor only detects motion if the whole image is shifted, it seems that when a microorganism (Blepharisma Japonicum) passes the field of view, there is a substantial change in the SQL-value.
further i will try to do some simple image processing on the arduino directly, such as simple background substraction.
Download Code
The code for the arduino and puredata can be found here (work in progress):
* The arduino code, uses two libraries: pitches.h and SimpleMessageSystem.h. the first is just used to play a nice little welcome melody, the second is only needed to communicate back from pd to change the dumpWidth of the frame. copy them to your libraries folder.
* The pd patch uses two externals, “convert” to communicate with SimpleMessageSystem” (again only from pd to arduino not vice-versa) and a second one, “pd_fire_grad” to apply a look-up-table (LUT) to the pixels for better visualization.
look for older projects here…
Mechatronic Art
Hackteria
Open Source Biological Art
funded by the Bundesamt für Kultur
Migros Kuturprozent
