Here is an introductory survey for me to get a background on the use of digital media by animal behavior researchers. If you study animal beavior, please fill it out!
https://docs.google.com/forms/d/1_hGb-XKGDJraD0mIr49n6tlneIbtJjzdpXYi9tekF5c/viewform
Thanks!
After the Gamboa Talk, I shadowed Peter’s work in the Field. He is about to launch into his official experimentation, and is spotting a couple more trees to use in his tests. While the jungle is full of cecropia trees, only a certain small demographic can fit into the constraints of Peter’s experiment. Some factors are due to the theories being tested by the experiment itself (the tree needs to have a healthy, mature colony living inside it), and also due to physical limitations (if the tree is too tall, he can’t reach the leaves to perform the experiments).
The night dropped quickly out in the jungle, and we, still full of energy from the weekend, decided to just keep driving as the pipeline road grew narrower and narrower.
Peter stopped the car at one point when he saw a large flashing bug. We hopped out of the car and I grabbed it. It gave me a startle as it started snapping hard into my hand (with about the strength of a human flicking). Soon the lightning beetles were flying in from all over the depths of the woods, drifting silently out of the darkness.
I started filling my hands with the snapping fireflies as they descended upon our truck. They were as long as a stick of gum, and had not only the brightest luminescent abdomens I had ever seen, but also additional different colored “running lights” on their shoulders.
We took the vial back to the ridge, and showed off our catch. Then we setup a camera and recorded some long exposures of them crawling over us as we let them go.
This experience inspired another performance which we will perform (and describe) later.
The Digital Naturalism Logo is a stylized version of one of ‘Etienne Jules Marey’s diagrams of the wing movement behavior of a bee. Marey developed novel techniques for simultaneously extracting and sharing information from the animal world, and helped give birth to the new medium of film. His tools (sensors, triggers, and temporal photography) promoted strange new ways of encountering the world. Digital Naturalism strives to promote this style of research in the era of new behavioral tools to view ethology as a new medium of expression.
Gamboa is a strange town at the end of a highway across Panama full of scientists, townies, resort goers, and canal workers. It sits in the middle of the country directly at the intersection of the Chagres River and the Panama Canal. It was originally an American township built for canal zone workers, but as the US relinquesed control it became more Panamanian.
Now it is often heavily used by the Smithsonian Tropical Research Institute as both a launching off point for travel to Barro Colorado Island, as well as the site of many scientific research labs. Scientific residences are often in modified dock worker houses which have been reappropriated as luxury condos for rent from the Gamboa Rainforest Resort. A primary field site for many individuals is along Pipeline Road. This trail along an old oil pipeline transects some of the best parts of the jungle, and serves as a tether from which to sink into its depths.
Erin Welsh, a STRI scientist posing in front of typical Gamboa Architecture (also a sloth).
Here’s a website / blog / resource summarizing daily life along with tips and tricks for figuring out his strange place http://gamboalife.com/
Glass frog that hopped onto May Dixon while she processed an interesting bat
Low morning with lots of computer stuff. Nice and rainy out but still couldn’t sleep. Things brightened up with brief chat with Kitty and then initial interviews and shadowing of Toni.
Today is Toni’s full induction to my research. We go over the papers and what we plan to do. Then I shadow her daily work. She walks around houses in the neighborhood where they have weight monitoring stations set up coupled with RFID readers for bats coming and going. She climbs up ladders under these stilted houses to change the batteries and collect their data.
Next, we visit her flight cages next to the woods. She shows off the full area where she will test bat decision making under sleep deprivation conditions. The bats will move through simple maze in the first cage, through a connecting hallway, and then through a complex maze in order to get to the fake robotic tungara frog signaling food. The tricky problem we are thinking of now, is how to keep the bats awake in the least stressful manner.
This is part of a much larger research project studying the effect of sleep across many different animals. The awesome Barrett Klein, whom I met the previous summer and discussed paint pens with, has already done such experiments with honeybees and wasps. For these he used his insect inseminator, which consisted of magnets glued to backs which could be wirelessly agitated by large magnets outside the nest. I found out about this when I sent him my recipe for magnetic ants last fall.
First talk is scheduled for the day. A 15 minute quick “teaser” introduction to “Digital Naturalism” scheduled for after Kathy’s talk about Arbuscular Mycorrhizal fungi.
Peter meets girl named Steffie from Konstanz who is leaving today but is very interested in my work. She is part of an Ant Course that is ending today, so I did a quick impromptu lunch presentation with them also. Feeling pretty good about how quickly I am able to tap into lots of different people. I guess I was more prepared than I thought.
Yesterday, Peter and I spent the day on computers getting a backlog of work taken care of. I also busted out some ATTinies to give Peter a first glimpse of how they work, and also to reassure myself that they do still work. With any of these, even banal seeming, peices of technology I have to remain suspicious of their functionality until I can actually make them perform the plan I originally envisioned (even if I had done it 100 times before). For a quick demo, I went from a Blink Example (But with a vibration motor), to a slightly more sophisticated light controlled vibration motor. Peter then sparked on the idea, from this light sensitive device, of simulating a cockroach’s light avoiding behavior by having a bristlebot that would freak out when exposed to light, and calm down when hiding in the shadows. He basically thought up a 1-Dimensional Braitenberg vehicle. Took a brief computer reprieve in the small forest within Gamboa. Found snotty goop on tree. Single Leafcutter ant showed me how much my pinkie finger was like a leaf. To be fair i was testing how quick their jaws could close using my finger as a trigger.
Spent today trying to see if I could pack all the parts necessary for such a vehicle (Perf-board, light sensor, resistor, ATTiny85, LED, and battery) into the shell of a cicada that we found. You get some great effects with the LED in there, but the coin-cell batteries are still much to large to fit. It’s a good reminder of how small insects are (even the large ones). The whole device does still work with most components mounted on it, and the battery on a tether. Perf boards are trickier to use than I had thought. Maybe I should look up proper technique.
All of my core participants are given their own official journals in which to keep their thoughts. I offered several different sizes, shapes, and abilities of journals so they they could choose on that best suited them. Both Toni and Peter opted for the smaller sized journals with Peter choosing the waterproof one (that you can’t use pens on), and Toni opting for the top-bound mini-drawing pad.
They are tasked with writing an entry everyday, and tackling these four guideline question/tasks (which are glued inside the journal).
—————————–
Questions for Research Journal
“What is Digital Naturalism” PDF “Inspirations” PDF Design Considerations for Ethology PDF
I compiled several different projects at the intersections of performance art, literature, design, physical computing, and cybiotic animal-machines. The goal was to have a rich reference guide for scientists to browse, investigate, and potentially adapt to their own approaches for engaging with or disseminating their research. I also made another guide, “What is Digital Naturalism” to quickly explain my research, when encountering interested parties in non-digital places (like the jungle).
For visual and textual knowledge sharing, this project faced interesting design challenges. I wanted to make booklets (by hand, since it was the cheapest) that the biologists could bring with them into the field and reflect on, in situ, during down time. My thought was that this could be additionally inspirational, but it also posed extra challenges. The booklets would need to be tough to survive in a field biologist’s backpack, and also not be too heavy.
This lead to my design of 1/4 page sized booklets, printed off our common GT printer (double sided), which I then sewed together for durability (and because the staplers were too short and the staples fell out too easily).
I instructed peter to take two techno-artifacts from our messy biocrafting station which we were setting up (it has a sign now!). We would use these to enact some sort of performance prototyping a behavioral tool. As with any performance, I find myself always internally cynical. I default to being my own devil’s advocate despite the purported performative basis of my research. Before hand, i always have a nagging feeling of, “this is stupid, we are going to dance around and pretend, and nothing will be accomplished because we know what’s going to happen.”
But all this serves to continually prove my point where, by forcing myself or others to physically realize a theoretical model of action, the doing immediatly 1) reveals the mental crap clogging up the original idea, 2) inspires and manifests unthought of arrangements, and 3) polishes the revealed good parts that were supporting the original idea.
At the end I had him bow and it seems like it would be a good idea to come up with a set of these performance self signals (ahhh there is a word for these in ethology that i cannot remember right now) where a creature performs certain actions that put it into and pull it out of a certain behavioral state. The self-signals can help put the performance into a focused activity for better reflection.
Showed the film, The White Diamond, by Werner Herzog this evening between jobs, and everyone loved it!
Am now about to [truck pulls up] help Santiago and some of the bat crew test their high-speed camera they are borrowing from Wcislo. Through pure perserverence in fighting unmatching equipment (they only had a nikon adapter, but canon lenses for the high speed camera), I made the strange discovery that canon Eos-M lenses can mount directly to the nikon. We haven’t been able to check and see if the lens distance isn’t too screwed up or anything, but it certainly fits just fine right in there.
Hannah Perner-Wilson combines conductive materials and craft techniques, developing new styles of building electronics that emphasize materiality and process. She received a B.Sc. in Industrial Design from the University for Art and Industrial Design Linz and an M.Sc. in Media Arts and Sciences from the MIT Media Lab, where she was a student in the High-Low Tech research group. Since 2006 Hannah has collaborated with Mika Satomi, forming the collective KOBAKANT. In 2009, as research fellows at the Distance Lab in Scotland, KOBAKANT published the website titled How To Get What You Want, where they share their textile sensor designs and DIY approach to E-Textiles.
May is going away quickly. Woke another morning with too much to do and too many opporunities to potentially lose to sleep in. Got lots of official things setup this morning: Gamboa Talk (mini) on monday, Gamboa Workshop next monday, Lunch with Bill Wcislo on tuesday, meeting with batlab on friday.
From the little bit that I got to talk with her on thursday, and from the things that others have told me about her (like her bat-themed play in bulgaria), she seems like a great fit for the Guinea Pig Slot #2. They are going on a trip climbing up a volcano though, so in an effort to waste less time, I gave her a rapid induction. Sort of like a battlefield promotion. I’ll have to follow up with her when she gets back next week [Mosquitoes killing me know as I write this waiting outside the batlab at night…need to move] She ‘s so enthusiastic I can’t wait to see what comes out!
This morning I tagged along with Chris and Victoria on a bullet ant nest hunt which proved to be apocryphal. Instead we worked on ways to test aggression in leaf cutter ants. Having trouble at first, because the ants weren’t biting through the flat foil, but then we though to fold the foil in a bit of a fan shape, and they were able to grab a purchase on it then with their mandibles. This made neat aesthetically pleasing holes that were revealed when held to the sun.
Lunch with Peter. Canned food and chips in the park – no spoons. Then we prepped for some free-form jungle exploring, nominally helping kathy look for monkey comb seeds.
Before I even got here, the scientists in this strange vibrant community were already coming up with great ideas for experiential performances involving the animals.
Morpho-Smack
Walking through the jungle, all naturalists are randomly struck by the impromtu passing of a brilliant blue morpho fluttering by. Now they will be physically struck also! An adaptation of the traditional “punch – buggy” or “slug-bug” games, this is a new tradition that Ummat started doing which serves a fun recognition ritual for an event that is commonplace but unpredictable.
Drunk Natural History
Based on the “Drunk History” concept by Derek Waters at FunnyorDie http://www.funnyordie.com/drunkhistory , several of the people in gamboa have apparently been discussing recording descriptions of animal behavior by inebriated scientists, and then going out in the field and recreating these performances. FunnyorDie’s popular original concept could provide an excellent framework for reflexive performances since a) it has a readily understandable script for action and b) can serve as a productive use of leisure time.
This could be quite like Rossellini’s Green Porno series but perhaps sillier than strange.
Experimental Solidarity
Toni, one of the Bat-girls working under Rachel Page, was talking about a 24 – hour observation experiment she was going to have to do with her bats. Since this was also going to be such an endurance feat on the human’s part she also wanted a way to document and experiment upon herself. She does not have all the details decided yet, but really wants to submit herself to the same sort of tests that the bats are undergoing during this test. For instance while the bat was being kept awake and subjected to mazes at regular intervals, she was going to time herself playing a “memory” game with cards.
Rock in the Jungle
Peter Marting’s band, Ptarmigan is actually slated to have a performance here on june 15 (which unfortunately I will be gone for). This is due to a lucky coincidence of his fellow band-mates coming down to visit in combination with an open-mike night.
In keeping with last year’s traditions, I show up at La Tienda to buy my first set of food and supplies during that no-man’s zone between 1-2 pm. Gives a good opportunity to write down my first field entry.
Got an hour of sleep after unpacking and am now scouting locations for setting up the Digital Biocrafting station. I was happy to see that my tiny version of the old biocrafting station was still intact!
The school house will probably be ideal if they let me have a desk. All of this would be much easier if I could bank on actually being officially funded or not, and could then proceed on a particular strategy for acquiring a tiny bit of room to set up for the community. But, even as May draws to a close, I still have to wait to see what will become of me this summer. Either way, glad I was able to organize the trip on what I already had.
Had a relaxing yet also invigorating morning straight into the jungle. Walking from the airport to the car at 3am Ummat poses me a question that brings a large smile to me: “So, are you excited to go back into the jungle? More excited than tired? Let’s go to the observation tower!” Am already feeling the time crunch though in only my first few hours here.
Stopped by Rachel Page’s office and chatted with her and May Dixon, so fun to see them again. With a new bat-girl, Toni, I dropped off some of my first sets of “Digital Naturalism” propaganda. It seemed to do its job of initially providing an attractive bundle that invites further contemplation. Excited to see how this pans out. One of the excellent parts of having ready-made material when entering a zone of activity intensity is that it can provide a brief mental reprieve to oneself Where I am now constantly thinking and list-making about all the things I needed to do before I came down, and all the multiple facets I must now plan, it is nice to be able to just pull out something in a situation, and have that responsibility of thorough explanation offloaded to an object.
Surprised at how much I perfectly remember about Gamboa and specific pathways in the jungle. For such a relatively short stay that I had last time, I have so many vivid memories seared into my brain.
Performances discussed today:
– Toni, Rachel, and May indpendently brought up doing a 24 hour observation where they would also include fun tasks for the
Last year before my trip I had little knowledge about what awaited me. Back on that trip the mission was to collect as much high quality footage of social insects as possible. So I packed three DSLRs, two gopros, tons of SD cards, hard-drives and backup hard-drives, and lighting equipment. It was easy to pack for the mission because the mission was straightforward, but I had no idea what the living environment would be like at all. Thus, that year I went overboard on survival supplies due to my cluelessness about the everyday life. I packed tents and sleeping bags and a full bee suit. I had mosquito nets and tarps and space blankets. I didn’t realize then that we would be staying in re-appropriated tropical resort villas next to the jungle instead of in the jungle.
I now face the reverse problem. I have a very good concept of how the area functions and what supplies I truly need for general life, but for carrying out my research I had to stay as open-ended as possible. I ended up with a kitchen-sink style, basically bringing as much of my digital toys and physical computing / biocrafting equipment as possible.
Determining the way to optimize the packing to avoid 200 dollar overweight fees on the airline, as well as balance out pesky TSA rules about carrying potentially hazardous material like LIPO batteries or CO2 canisters was tricky and took me 3 days of thinking to sort. I tried to be overly cautious since most of the equipment is out of my own pocket, and I included notices for the TSA officers and a full inventory printout that I placed in each pelican case. This social hacking attempted to over-weigh the suspiciousness of my boxes full of wires and strange devices with the pseudo-authenticity of an “official” research scientist going into the jungle with lots of “official” forms.
Earlier, I posted a video of leafcutter ants claiming that it contained a secret code. Well it’s true! Here’s how to crack the code, and how I encoded my messages in the first place.
The astute observer may take note that the ants carrying leaves only travel in one direction (towards the nest). In fact, this is the entire underpinning to the code. When I presented the puzzle to my lab, the response I got that was closest to correct was from Prof. Tucker Balch who stated that the first thing he would do is “chart the number of leaf carriers visible in each frame over time and look for patterns in that time series.” Good thinking Tucker!
The first step is to create a signal out of the leaf-carrying ants. To do this, one can simply take the green channel of the image and adjust a threshold until just the leaves are selected. To get more exacting data you could try to apply additional filters like blurs, dilations, etc on top of this thresholding. You can even use professional video compositing software like After Effects and “key-out” the green color. These additional improvements are not really necessary however, you can stay pretty crude.
Next, because there might be some extra foliage around the edges, you will want to crop to a region of interested just around the ants.
Example image targeting just the green leaves from the video.
The video should now be entirely white (255) in areas where the leaves are present, and entirely black everywhere else. I then made a simple script that tallies up all the white pixels (detections) present in every frame, and it saves all this data as a CSV. When I pop open this CSV file in an open-source equivalent to Microsoft Excel, and chart the results, I get something that looks like this:
Ahh, that looks like it might contain some sort of signal. Now’s the time for the cryptographic skills. Your first intuition should be that Andy isn’t that big on cryptography, and will probably just use the first temporal coding sequence that comes to his mind, Morse Code. If one takes the slightly wider pulses to be dashes and the slightly narrower ones to be dots, you can pull out this pattern:
Or translated from Morse->English: GT CPL
The Georgia Tech Computational Perception Laboratory (where I work).
Yay! I also have some additional videos where the ants say a couple other messages like “Digital Media” and, of course, “Hello World.” I even made a special message to the class of my cool Biology teacher sister. I will post them here when they are ready.
Both videos from this puzzle say exactly the same message, it is just that one, the first video, was recorded further down the stream which gave lossier data, so more human, visual intuition was required. The reason this data was lossier will be explained below. Additional props go to DM student Rebecca Rolfe who uncovered the unintentional Rebus of the video, “Soon there will be no leaves left” (Get it? Get it? Ants are carrying all the leaves to the right….)
How did the ants know how to communicate this message? Well they probably didn’t.
Earlier in the summer, I wanted to test Leafcutter responses to temporary barriers. It turns out that if the barrier is only there for a short amount of time (<1 minute), the ants will just sort of pool-up behind it instead of walking around (note, this is not true of other ants, like Army Ants).
To get more precise results, I built a simple servo-device controlled by an arduino which was attached to a fluon coated plate. While I was cutting-off, and re-enabling the flow of ants, I realized I could also program this device to send ant-based messages in this fashion. Thus after lots of experimentation, and a long hot day sitting in the jungle with my ant tollbooth, I found a workable formula for sending dashes and dots, and made the servo go up-and-down correspondingly to whichever message I wanted to send.
Of course, I wasn’t 100% certain that it worked until I got back home and analyzed it myself!
The video below contains a secret message:
I’ll reveal the secret and how to figure it out on Tuesday, so you’ll have all of Labor day weekend to ponder.
First one to email me with the correct answer gets a prize (don’t email the whole list- serve and ruin the fun!).
First one to email me with how to arrive at the correct answer gets mild applause 🙂
——————————-
Hint 1: To work with the video, it may be easier to download it all at once. Just pop the link into http://www.savevid.com/
Hint 2: (don’t use the hint unless you have to!): This video is slightly easier to decipher:
Not sure if people have been doing this for dozens of years in the insect world already (maybe this is common practice?), but I just ran some successful tests on an idea I had that I think could lead to some fun experiments.
I was reading the fantastic book, The Ants, and they were discussing how one of the interesting characteristics of working with superorganisms is that you can non-destructively turn them into mutants. Regular organisms tend to permanently change when you drastically alter them. You cannot do a quick test with an ape to see how it behaves differently if you remove its liver, or chop off its arm, and then return it to normal the next day. Regular organisms are quite fragile in this way.
As Holldobler and Wilson point out, though, Superorganisms are quite flexible. Interesting experiments can be done where various parts of the being are removed, like temporarily taking out all the ants in the soldier caste, and the hive can be returned to its original state at the end of the day.
I was trying to think of ways in which you could speed up this process and have a constantly, rapidly mutating colony. How might one be able to quickly remove a select target group of individuals from a nest (without sitting there hunting with tweezers).
Then I thought of magnets!
Magnetic paint already exists, but apparently only in aerosol or latex paint, not good insect friendly paint like enamel paint. So i tried out making my own. First I had some success with mixing iron filings with enamel paint.
My first test subject, a palmetto bug, was sucessfully recaptured after her daring escape using her newfound magnetic properties.
But these filings were so large that they would rip through the paint and snap out over time. Then I found some (secretly) magnetic powder (that also has the benefit of being flourescent!), and I mixed it with the enamel. This gave me a nice smooth, very attractive paint which worked on large or very small surfaces!
Magnetic Powder Paint subject with many test-swatches.
The key ingredients, magnetic (fluorescent) fingerprinting powder, and enamel paint.
I was thinking you could make a certain target group magnetic, and use a switchable, on-off magnet to collect them and put them back in the nest. But then I was thinking of other fun things you could do with them!
For instance, if all your temnos were painted magnetic, you could use a tiny magnetic rod to more gingerly pick a single one out of a nest instead of worrying about hurting it with tweezers. Also we were thinking it might make them more capacitive, and their movements could be picked up by stuff like ipad multi-touch surfaces (not tested yet). Field scientists that need to recollect the same, marked bugs, can also use this to wave powerful magnets around in the bushes for recapture. The coolest idea, though is that you can somewhat manipulate their behaviors at will!
Determining if wet paint has become magnetic (pretty good, needs a bit more powder
Hand-painting small spot on live ant
Here’s a video I made doing a very informal test about how ants respond to a fellow who seems completely normal most of the time, but who sometimes starts flipping-out and sliding around crazy a la “The Exorcist.” The ants treat the magnetic ant completely normally until she freaks out, then they pounce and start biting her head. After the freak-out, they keep biting for a little bit, and then let go, and everything’s back to normal. On top of visuals I provide a andy’s-been-programming-too-long crazy narration of the events 🙂
I am the lead researcher on several scientific biotracking projects at Georgia Tech
Last serious trip on Pipeline. What am I filming? The road. Camera pointed down the road. Regular and HDR video.
Could be used for detecting Blue Morphos, but also sitting with and observing this slice through the forrest.
First thought is just boring. Dead forrest with an occasional jungle truck or Blue Morpho. But there are some decent soft metrics one could probably pull from this. This constant, jungle monitoring camera. Daily weather patterns, wind, leaves falling, sounds, tree movement, general movement along transect, traffic patterns.
Chased butterflies. Figured out how to make Hamatam army-ants retreat.
It was a different day knowing that these probes and pokes would be my last.
Thinking about how tracking shots on a dolly look so beautiful because they are a rigid, grammatical way of representing 3D information. Each tracking shot is the temporal equivalent of a gorgeous data visualization.
Thought up a possible full title for my thesis-
Digital Naturalism: Cybiotic Media and Digital Biocraft for Exploration and Dissemination
6/21/2012
Let’s hurry up and get some facts down. Quick for memory.
Early to bed. Up at 7:30 but not going till 9. K_____
Just went to cecropia lot for last bits of footage, sugar water testing and termite rebuildings.
Turns out they can rebuild one of these tunnels within an hour.
Long-ish lunch discussing experimental ideas for Peter with Stephen. Back to parking lot. Tested whether ants prefered the taste of human blood over sugar water. They really seemed to like the blood.
Met with Yann returning from San Blas.
———–
Wanted to go to Bambi talk. It’s impressive how organized biologist are in this STRI commune. (http://www.stri.si.edu/english/about_stri/seminars/index.php)
The talk was all booked however. Peter said his roomate wasn’t going, so I subbed for him. Turns out that guy wasn’t actually registered, but when I showed up to the guard guy, my real actual name was on the list. (Of course I didn’t notice that until after I told him I was Willie, got awkward when I told him my real name was andy)
Got a ride in the boat.
Saw Ummat’s talk. One of the best I’ve been to out here!
Came back to record Kenro’s interview, and attend Victoria’s going-away party.