I’ve thought for a long time about how a Wii Nunchuk would be perfect for controlling a puppet. It has an accelerometer, a joystick and two buttons.

Accelerometers are sensors that measure proper acceleration – the orientation of an object relative to the Earth’s gravity. You can use an accelerometer to determine the direction that an object is facing. Probably the least expensive accelerometer is the Wii Nunchuk – off brand versions can be bought for $10 at big box stores. Additionally, they can be bought as chipsets that plug into arduinos – chips are usually start around $12 in physical stores and cheaper online, but you miss out on the benefits of the extra buttons/ joystick. Also, you can buy a Wii Nunchuk at a bigbox pharmacy at midnight on a whim.

Cut the end off the nunchuk and plug it into your arduino. Plug your arduino into a computer.
I’m using the wiichuck adapter from Todbot

It spits out serial data to whatever software you want to use. Sadly, you do need some scripting/ programming knowledge here. Fortunately, you can use all kinds of things, including node.

Here’s what the data looks like:
{“accx”:180, “accy”:125, “zbut”:0, “cbut”:0, “joyx”:122, “joyy”:130}

There was some interest in the assets and how they work. Here are the cutup pieces that I used:

Arduino Sketch

For an upcoming project with CTT, we decided to make a giant etch-a-sketch out of a quadcopter, a UV laser, a giant glow in the dark wall and two turntables. As part of a bonus project I made a real etch-a-sketch that is controlled by 2 turntables. The turntables replace the knobs by controlling two gearmotors that spin forward or reverse in very tiny increments. In addition, this approach allows for computer controlled illustrations and perhaps doing things like uploading an image and having it draw on screen – I’m almost done this. Automated Etch-A-Sketches have been done before, but I’m using my own approach.

This works by using Time Coded Vinyl. TCV is a vinyl record album that has a timecode pressed into it instead of a “traditional” audio track. The TCV is interpreted by a hardware controller and sent to a computer via USB connection.

The Traktor 2 Scratch Pro software interprets the position of the playhead using the data received from the hardware controller. There is a 1 second track that loops based on the record input for each of the two turntables and it’s the playhead position on this track that is used for the midi out. This is then sent from Traktor 2 to a virtual midi device (LoopBe1) which is software that’s designed to behave as though it were a piece of midi hardware. This is specified in Traktor 2 Scratch Pro under Preferences > Controller Manager.

^{NO YOU CHECK YOUR OUTPUT ROUTING, BITCH! btw, this is a s’up tag, so, you know, s’up?}

I’m using Node.js to receive the data and send to the arduino. The Node.js application uses Justin Latimer’s node-midi to receive the midi in data and Voodoo Tiki God’s node-Serialport to send serial messages to the arduino.

The arduino sketch receives the messages via serial and uses a Pololu MC33926 to send messages to the two gearbox motors.

I’m not going to include my whole node.js application because I’m a bit crap at node and embarassed, but here’s the whole server.js sketch. It should be all you need after you install the dependencies mentioned above:

The motors either advance or reverse in 10ms increments, hammering the motor shield with constant, tiny instructions. Because of the short increments, it appears to be continuous and fairly responsive to the record movements.

Dragon Pumpkin Spring from Christopher Lewis on Vimeo.

The Microsoft Kinect has a lot of powerful motion technology, but it also has an under appreciated microphone array which is great at localising audio, reducing background noise and text recognition.

On June 8, 2013, I gave a presentation at FITC’s Spotlight: Hardware. It was a great event that covered many topics related to hardware and physical computing. As silly as it is, I presented what I think is one of the best ways to get into making physical interactive installations is by doing “mouse hack”.

A mouse is the quickest and easiest way to include digital interaction in real objects with the added bonus that mice are fairly inexpensive and people often have a few kicking around. The goal is to add a couple of wires that will be another way to perform a click. Because a mouse click is an “OS” level interaction, no or few programming skills required. Also, using a wireless mouse means you can create a wireless digitally enabled interactive object. If you do need to buy a mouse, $5.99 is the most inexpensive price I could find in the real world – I’m sure you can find cheaper online. The important thing to remember is that mice are inexpensive, so you shouldn’t be afraid of opening them and maybe screwing up.

Tools:
Mouse
Wire – anything will do, but hookup wire is the best
Wire Strippers – an exacto works – “automatic” wire strippers are the best
Screw Driver
Pliers (optional)
Solder (optional)
Solder Gun (optional, but highliy recommended
Electrical tape – so helpful

Soldering scares a lot of people, but it’s a great technique and will make the whole thing easier. I think this is the best video I’ve seen about how to solder:

Insert the battery into the mouse to make sure it works. If the mouse has one of those annoying on/off switches, turn the mouse on and then remove battery. These switches pop out and get lost, so you want to make sure that the mouse is on in case you can’t get the switch back in position.

Often the screws are hidden under LRF. (little rubber feet) Pull off the LRFs to reveal the screws, and set them aside. Unscrew the case and open up the mouse.

Take a photo to remember where things go. Sometimes it’s good to tape parts that look loose – the battery housing and mouse wheel, in my case. Look for the thing that might be a button – it’s marked SW1 on this particular mouse – presumably for “Switch 1″.

Put your hand over the mouse to make sure it’s the correct button – nothing bad will happen if you press the button.

Pull the circuit out. Things might pop out. Oh well.

There are two little nubs directly under the button. Touching these nubs together is the same as clicking the mouse button – nice!

Strip the wires. I like to tie a knot in them so they stay close together and are easy to work with.

Put the wire through the stripper and align it to the correct guage. If you don’t know, then eyeball it.

Use some pliers to wrap the stripped wire around the nub. Solder it into position. If you’re afraid of soldering, you can use electrical tape, but it won’t be as reliable.

By this point the on/off switch has probably fallen out. Worry about it later or throw it away. Good thing you made sure the mouse was turned on earlier.

Try your best to put some tape between the two wires – if they touch by themselves it will count as a click, so you want to avoid that. You can use hot glue if you have trouble – it’s not the best approach, but it’s the easiest.

Put your finger over the mouse to make sure you got the correct button – I’ve screwed this up a few times.

Shove everything back together

Cut a notch in the front so the wires have a place to come out of the housing.

Close the mouse, turn it on, plug it in and touch the wires together – that’s it.

You could embed these mice in:
Footplates, switches, fridges, cookie jars, medicine cabinets, pill bottles, exercise equipment, etc.

Counters

You can now use the mouse to count things. This can be done easily by creating a simple html page with a button. Positioning the mouse over the button and touching the two wires together counts as a click. This requires very little HTML:

Click Count Demo

Tying the button click to something like a google analytics event would hypothetically allow you to permanently store your results online.

Why is this approach better than a physical count clicker? Storing data digitally, not requiring mechanical activation and being able to add this to almost any object are definite benefits.

Trigger

Audio or video can be easily triggered by a mouse click if the media player is in full screen mode. Quicktime, Youtube, Flash and VLC can all be setup to do this.

Still images can be triggered using Keynote/ Powerpoint or a picture viewing app. like irFanView

You can even make an html based presentation by using Javascript to hide the mouse and advancing a background image. In The Evolution of Type Demo with graphic by Mikaela Couch, the background advances with each mouse click. If the mouse wires were embedded in a physical object, such as a printing press, you could make pulling a lever the trigger action to advance the timeline display.

Come meet me and members of Creative Technologists of Toronto when I present at Spotlight: Hardware on June 8. Use the discount code ‘cttoronto’ for 10% off.

Dragon Pumpkin – Halloween 2012 from Christopher Lewis on Vimeo.

I decided to add a bit of “Trick” back into Halloween. The Dragon Pumpkin is a voice activated fire breathing pumpkin; saying the words “Trick or Treat” triggers a fireball. What’s more awesome than fireballs?

The Windows Kinect has a 4 microphone array and amazing speech recognition capabilities. Using the Speech Basics example that is packaged with the Kinect SDK, a C++ application recognizes the phrase “Trick or Treat” and sends a serial message to the Arduino. A simple sketch on the Arduino receives the serial message and turns on pin 13 – fairly standard – which fires a solenoid circuit triggering the fireball.

This whole project is extremely dangerous, so I officially discourage anyone from attempting to make a fireball. I’m skipping that step of my build intentionally. However, I would recommend getting in touch with your nearest Hacker Space if you’re interested in learning how to make a “poofer”. I wouldn’t advocate anyone else playing with fire, but what I learned about controlling an arduino via Kinect voice commands is quite useful.

The most difficult part of the whole project was getting Visual Studio installed and running.

For those interested in getting a Kinect for Windows talking to arduino, here’s a light overview of what I did:

Kinect Voice Recognition

I’m not sure if I’m allowed to redistribute the Microsoft Kinect examples, so instead of supplying a file, here are the steps I took:

1. Download and install Visual Studio 2012
2. Download and install Microsoft Kinect SDK, Toolkit and Microsoft Speech Platform
3. Open the Microsoft Kinect Toolkit Browser
4. Go to the Samples: C++ tab
5. Scroll down to the Speech Basics D2D and Install
6. Open the Speech Basics D2D in Microsoft Visual Studio 2012
7. Edit the SpeechBasics-D2D.grxml file. This is where you add the words. My file looks like this:
   <grammar version="1.0" xml:lang="en-US" root="rootRule" tag-format="semantics/1.0-literals" xmlns="http://www.w3.org/2001/06/grammar">
     <rule id="rootRule">
       <one-of>
         <item>
           <tag>TRICK OR TREAT</tag>
           <one-of>
             <item> trick or treat </item>
           </one-of>
         </item>
       </one-of>
     </rule>
   </grammar>
8. In SpeechBasics.cpp add your Trick or Treat action. For example:
   const SpeechTagToAction Map[] =
       {
           {L"TRICK OR TREAT", TurtleActionTrickOrTreat}
       };
9. In TurtleController.cpp I added the following at the top of the file:
   #include<stdio.h>
   #include<stdlib.h>
10. Here’s where the Kinect messages the arduino via serial. In TurtleController.cpp add a case for your Trick or Treat action to the switch statement in the “void TurtleController::DoAction(TurtleAction action)” code block. I got rid of the cases for all the other directions:
    case TurtleActionTrickOrTreat:
            m_CurrentDirection = NextDirection(m_CurrentDirection, false);

            // We take a left turn to mean a counter-clockwise right angle rotation for the displayed turtle.
            m_CurrentRotationAngle -= 47;

                    FILE* port;
                    system( "MODE COM4: BAUD=9600 PARITY=n DATA=8 STOP=1" ) ;
                    port = fopen( "COM4:", "wb" ) ;
                    printf("1");
                    fprintf( port, "1" ) ;
                    fclose( port ) ;
            break;
        }

11. Edit SpeechBasics.cpp and replace the SpeechTagToAction method with the following
    const SpeechTagToAction Map[] =
        {
            {L"TRICK OR TREAT", TurtleActionTrickOrTreat}
        }
12. In TurtleController.h replace the “enum TurtleAction” block with the following:
    enum TurtleAction
    {
        TurtleActionTrickOrTreat,
        TurtleActionNone
    };
13. That should be it – I think… Compile and you should have something that recognizes your Kinect and sends a serial message to your arduino. If the modifications work, the Turtle should turn 45 degrees everytime you say “Trick or Treat”.
    

The Arduino Sketch

1. I’m using a pretty simple sketch, modified from the blink sketch and Arduino Serial Communication:
   int firePin = 13;
   int state=0;
   void setup() {
       pinMode(firePin, OUTPUT); // pin will be used to for output
       Serial.begin(9600); // same as in your c++ script
   }

   void loop() {
     if (Serial.available() > 0)
     {
       state = Serial.read(); // used to read incoming data

       switch(state)// see what was sent to the board
       {
         case ’1′: // if the the one was sent
           flameOn();
           delay(750);
           flameOff();
         break;
         case ’0′: // if 0 was sent
           flameOff();
         break;
         default:
         break;
       }
     }
   }
   void flameOn(){
     digitalWrite(firePin,HIGH);
   }
   void flameOff(){
     digitalWrite(firePin,LOW);
   }

2. Run the sketch. When the Serial message “1″ is sent to the arduino, the on-board led or whatever is on Pin 13 will light up.

serialFlameControl.ino

The solenoid circuit is attached the Pin 13 on the arduino. It allows for a Transistor to activate a 12v solenoid without frying the arduino board. The circuit was taken from Controlling Solenoids with Arduino.

More photos

Special Thanks to Demi at SplitElement Inc. for the use of the Kinect for Windows and to LB for the voice talent.

In October 2011, I was Art Director for performances of Cavalleria Rusticana and Pagliacci, directed by Jessie Fraser and performed by Opera Belcanto. The two operas are often performed together as a double bill since they’re both fairly short and are thematic and stylistically similar.

I created a digital backdrop in the form of a fullscreen Flash application that could be controlled by the theatre staff. Hitting different shortcut keys would initiate different sequences of the backdrop which were timed to match the performance.

Both operas featured a timelapse video of the sky, captured from my balconey, to evoke the passage of time.

During the introduction of Cavalleria Rusticana, elements of the small Sicilian town, Vizzini, fade in to establish the location. The town was created by Photoshopping elements from a variety of sources. Some of the elements were taken from rights-cleared photos of the real town Vizzini. Additional details were taken from my own photography, such as a stone fence photographed at Riverdale Farm; it’s a Toronto/ Vizzini mashup.

At one point during the Cavalleria Rusticana, there is an Easter procession. To fill out the number of performers, I created a sequence featuring silhouettes of procession participants, triggered by shortcut key.

To match the content of the opera, a wine shop and church were included as prominent elements in the backdrop. Additionally, a British style vardo (gypsy wagon) is parked behind the wine shop; I added the vardo to connect the two operas. It’s implied that the performers were performing in Vizzini before appearing in Pagliacci. At the risk of spoilers, both operas feature murder following affairs; I was amused with the idea that murderous rage was like a disease that the Pagliacci characters brought with them from Vizzini. I included the original poster for Pagliacci on the wine shop wall in my depiction of Vizzini and again in Pagliacci.

To transition between the two operas, I gradually faded out elements of Vizzini and played the sky timelapse quickly in reverse to return to day.

Pagliacci was set in the small town of Taormina where travelling performers arrive to entertain the town. I decided that the performers would set camp and perform in a secluded or less travelled area on the outskirts of town, similar to the locations that circuses always seem to be located. The covered area on the left and brick wall are both from the pig pen at Riverdale Farm.

A shortcut key was added to darken the backdrop to create a more intimate scene.

http://www.flickr.com/photos/blacksanta_69/6270895286/in/set-72157627830034153/

During the curtain call, the backdrop fades out to bring the audience back to reality. I’m proudly receiving my adulation on the very far left.

Earlier this year, Twitter introduced Web Intents as an easy way to Tweet, Reply, Retweet, Favorite, and Follow. Web Intents are meant to be used with popups, which means I can easily adapt from my previous Tweet Button for Flash technique: add a javascript popup to the page, set some variables and call the popup from Flash.

Twitter introduced a Follow Button that behaves similarly to the Tweet Button, but their claims that it, “uses the same implementation model”, is the tiniest bit bullshit. The Follow Button relies on Javascript in ways that we can’t easily use with Flash.

Our Web Intent Follow Button opens a popup window with a Mini-profile page, as pictured above. The Web Intent Follow Button uses either one of the following two values:

• screen_name is the name of the twitter user you’d like to follow. e.g. @blacksanta_69
• user_id is the id of the twitter user you’d like to follow. I had a hard time finding my twitter id so I’m using screen_name instead.

Like before, you drop the ‘www’ when using Web Intents.

1. The Popup Javascript function needs to be added to your html page in the <head> part of the page. The width and height are hardcoded in the javascript to reflect the Twitter recommended dimensions of a Tweet window. However, this can easily be passed as a parameter from Flash if you want.

2. Create a button in flash with the instance name btn_follow and add the following code. It’s worth noting that this works everywhere except within the Flash IDE, so test it on the html page

Gives you:

Source FLA Here

In HTML

Gives you:
Follow

Because there are so many easy to use features with Web Intents, the logical next step would be to build AS3 classes.

I have created a Camera-B-On TV-B-Gone. This fairly simple mod allows me to use my TV-B-Gone as a camera remote for my Nikon D90. In fact, this will work as a shutter remote for a lot of Nikon cameras.

If you have a USBTinyISP you can easily make a Camera-B-On by upgrading your TV-B-Gone.

Download Source CameraBOn.zip

Why make a Camera-B-On out of a TV-B-Gone?

I’ve made camera remotes before using an arduino board and an IR LED, but there are issues with getting a good range and the form factor is a little bulky.

The Tv-B-Gone is compact, effective and comes built with the ability to have 2 states making it an attractive option. The Tv-B-Gone is about $10 cheaper than a brand name wireless release with the added bonus that you can fuck up TVs. Plus, the Tv-B-Gone was built to be a universal remote and it would be fairly easy to extend it to trigger many cameras. Right now my model only triggers Nikon cameras but it would be really easy to add others.

Unfortunately, most cameras need to be set to “remote” mode for this to work, so it’s not like you’re going to be triggering other people’s cameras – at least not to my knowledge. A guerrilla camera tool would be fantastic, but I don’t think it’s feasible.

1. Adding the Nikon code

The first step was determining the IR Code that would trigger my camera. When I built the arduino version of the shutter release, I used Lucky Larry’s arduino intervalometer code, which includes the timing I needed.

Armed with the timing information, I downloaded the source code and read Lady Ada’s notes on the TV-B-Gone.

The program on the TV-B-Gone microcontroller cycles through about 115 known TV “power” codes. Each signal code is a series of very fast flashes of infrared light generated by an IR LED on a remote. These patterns of on and off times are chosen to try to avoid triggering the wrong devices. A camera’s wireless shutter release remote operates the same way.

For the Nikon wireless shutter the times look like this, in microseconds:
2000 on, 27850 off,
390 on, 1580 off,
410 on, 3580 off,
400 on, 63200 off

This signal is sent twice every time the shutter is meant to be “released”.

With the TV-B-Gone, the TV codes have been compressed to fit as many as possible on the ATTINY85V chip. Fortunately, Lady Ada describes how this happens well enough for me to do the same conversion by hand:

1. I converted Lucky Larry’s times from microseconds to the appropriate unit by dividing by ten – I don’t think there is a name for that unit, but it’s the value used in the source code.
   const uint16_t code_nikonD90Times[] PROGMEM = {
   200, 2785,
   39, 158,
   41, 358,
   40, 6320,
   };
2. I described the new value as an index in an array of on/off pairs. e.g. [0, 1, 2, 3]
3. and converted that index value to binary: [00, 01, 10, 11]
4. I converted those binary values to bytes and then to hex and doubled them to send the signal twice: 0x1b
5. const uint16_t code_nikonD90Times[] PROGMEM = {
   200, 2785,
   39, 158,
   41, 358,
   40, 6320,
   };
   const struct IrCode code_nikonD90Code PROGMEM = {
   freq_to_timerval(38400),
   8,             // # of pairs
   2,              // # of bits per index
   code_nikonD90Times,
   {
   0x1B,
   0x1B,
   }
   };

I then created a .hex file with the new code added.

2. Programming the TV-B-Gone with a USBTinyISP

The next step was to program the chip with the new .hex file. This process should involve plugging the TV-B-Gone into my laptop using a USB Tiny ISP programmer and software like AVRDUDE. (Tutorial here) Unfortunately, AVRDUDE wouldn’t recognise my TV-B-Gone.

A quick search (here) seemed to suggest that the LEDs draw too much power and might interfere with the chip being recognised. The only suggestion was to disable the resistors or LEDs to program the board. I decided to continue testing and in the process accidentally blew out all four LEDs by using the USBTinyISP power and battery power at the same time. Luckily, this allowed the board to be recognised!

I made a quick trip to Creatron and picked up new IR LEDs and started again. I got the board programmed and the TV-B-Gone worked for both my camera and TV! Hooray!

3. Camera or TV/Camera mode

I decided to make one more mod to the TV-B-Gone.

The board allows you to solder a jumper to control whether it uses North America or European codes. I ditched the European codes and opted to add a camera only mode by soldering a tiny “ON” switch (or is it “NO”?) in place of the jumper. The switch sandwiched nicely between the existing switch and a capacitor. I figured it would be easy to remember “On” is for the camera mode since it’s called a Camera-B-On.

Unfortunately, I had reinstalled the working LEDs. This meant that I wouldn’t be able to program the TV-B-Gone anymore. To deal with this, but not wanting to solder another switch to the board, I created a “detachable joint” jumper>resistor connection as pictured below to purposefully break the board when needed and allow me to reprogram:

Here’s a self portrait from the above video; I guess autofocus doesn’t work:

Future Revisions

The obvious next steps for the Camera-B-On is to add intervalometer functionality and support more camera brands. Intervalometers allow photographers to do time lapse. The fact that cameras don’t come with Intervalometer functionality built in baffles me – it would be so easy to add.

Here’s the arduino version I built a while ago:

In the Summer of 2011, The Grumpy Owl, a fantastic writer and patron of the bespoke, commissioned Breeyn and I to create paper, colour change boutonnières. The boutonnières were an extension of the successful paper dolls project we created for F.A.T. 2011. The Grumpy Owl beat me to the punch (by, like, 2 years) and blogged it previously.

Paper Dolls Boutonnière from Christopher Lewis on Vimeo.

The boutonnières consist of a simple electronic circuit: watch batteries power an RGB LED and each colour is individually controlled using 3 pot switches, allowing for an immeasurable number of variations in colour. Each switch basically increases resistance to each colour, RGB. Hot glue dimples (or nipples, as I prefer), were glued to the outside of the RGB LED allowing for the colour to spread in more directions.

Paper Dolls Boutonnière Electronics from Christopher Lewis on Vimeo.

The electronics sat inside absolutely beautiful paper roses and carnations that Breeyn crafted. The paper flowers are detailed with stems constructed of silver wire.

Perhaps my favourite thing about this project, beyond the beauty of the end product, is the potential to change the significance of the boutonnière for different occasions. For example, Blue or purple carnations are considered a funerary flower.