Dragon Pumpkin

Dragon Pumpkin – Halloween 2012 from Christopher Lewis on Vimeo.

I decided to add a bit of “Trick” back into Halloween. The Dragon Pumpkin is a voice activated fire breathing pumpkin; saying the words “Trick or Treat” triggers a fireball. What’s more awesome than fireballs?

DSC_3961

The Windows Kinect has a 4 microphone array and amazing speech recognition capabilities. Using the Speech Basics example that is packaged with the Kinect SDK, a C++ application recognizes the phrase “Trick or Treat” and sends a serial message to the Arduino. A simple sketch on the Arduino receives the serial message and turns on pin 13 – fairly standard – which fires a solenoid circuit triggering the fireball.

DSC_3951

DSC_3949

This whole project is extremely dangerous, so I officially discourage anyone from attempting to make a fireball. I’m skipping that step of my build intentionally. However, I would recommend getting in touch with your nearest Hacker Space if you’re interested in learning how to make a “poofer”. I wouldn’t advocate anyone else playing with fire, but what I learned about controlling an arduino via Kinect voice commands is quite useful.

DSC_3944

DSC_3935

The most difficult part of the whole project was getting Visual Studio installed and running.

DSC_3933

For those interested in getting a Kinect for Windows talking to arduino, here’s a light overview of what I did:

Kinect Voice Recognition

Kinect for Windows

I’m not sure if I’m allowed to redistribute the Microsoft Kinect examples, so instead of supplying a file, here are the steps I took:

  1. Download and install Visual Studio 2012
  2. Download and install Microsoft Kinect SDK, Toolkit and Microsoft Speech Platform
  3. Open the Microsoft Kinect Toolkit Browser
  4. Go to the Samples: C++ tab
  5. Scroll down to the Speech Basics D2D and Install
  6. Open the Speech Basics D2D in Microsoft Visual Studio 2012
  7. Edit the SpeechBasics-D2D.grxml file. This is where you add the words. My file looks like this:
    <grammar version="1.0" xml:lang="en-US" root="rootRule" tag-format="semantics/1.0-literals" xmlns="http://www.w3.org/2001/06/grammar">
      <rule id="rootRule">
        <one-of>
          <item>
            <tag>TRICK OR TREAT</tag>
            <one-of>
              <item> trick or treat </item>
            </one-of>
          </item>
        </one-of>
      </rule>
    </grammar>
  8. In SpeechBasics.cpp add your Trick or Treat action. For example:
    const SpeechTagToAction Map[] =
        {
            {L"TRICK OR TREAT", TurtleActionTrickOrTreat}
        };
  9. In TurtleController.cpp I added the following at the top of the file:
    #include<stdio.h>
    #include<stdlib.h>
  10. Here’s where the Kinect messages the arduino via serial. In TurtleController.cpp add a case for your Trick or Treat action to the switch statement in the “void TurtleController::DoAction(TurtleAction action)” code block. I got rid of the cases for all the other directions:
    case TurtleActionTrickOrTreat:
            m_CurrentDirection = NextDirection(m_CurrentDirection, false);

            // We take a left turn to mean a counter-clockwise right angle rotation for the displayed turtle.
            m_CurrentRotationAngle -= 47;

                    FILE* port;
                    system( "MODE COM4: BAUD=9600 PARITY=n DATA=8 STOP=1" ) ;
                    port = fopen( "COM4:", "wb" ) ;
                    printf("1");
                    fprintf( port, "1" ) ;
                    fclose( port ) ;
            break;
        }

  11. Edit SpeechBasics.cpp and replace the SpeechTagToAction method with the following
    const SpeechTagToAction Map[] =
        {
            {L"TRICK OR TREAT", TurtleActionTrickOrTreat}
        }
  12. In TurtleController.h replace the “enum TurtleAction” block with the following:
    enum TurtleAction
    {
        TurtleActionTrickOrTreat,
        TurtleActionNone
    };
  13. That should be it – I think… Compile and you should have something that recognizes your Kinect and sends a serial message to your arduino. If the modifications work, the Turtle should turn 45 degrees everytime you say “Trick or Treat”.
    Kinect Application

The Arduino Sketch

  1. I’m using a pretty simple sketch, modified from the blink sketch and Arduino Serial Communication:
    int firePin = 13;
    int state=0;
    void setup() {
        pinMode(firePin, OUTPUT); // pin will be used to for output
        Serial.begin(9600); // same as in your c++ script
    }

    void loop() {
      if (Serial.available() > 0)
      {
        state = Serial.read(); // used to read incoming data

        switch(state)// see what was sent to the board
        {
          case ’1′: // if the the one was sent
            flameOn();
            delay(750);
            flameOff();
          break;
          case ’0′: // if 0 was sent
            flameOff();
          break;
          default:
          break;
        }
      }
    }
    void flameOn(){
      digitalWrite(firePin,HIGH);
    }
    void flameOff(){
      digitalWrite(firePin,LOW);
    }

  2. Run the sketch. When the Serial message “1″ is sent to the arduino, the on-board led or whatever is on Pin 13 will light up.

serialFlameControl.ino

The solenoid circuit is attached the Pin 13 on the arduino. It allows for a Transistor to activate a 12v solenoid without frying the arduino board. The circuit was taken from Controlling Solenoids with Arduino.

Arduino and custom Solenoid Circuit

DSC_3913

More photos

Special Thanks to Demi at SplitElement Inc. for the use of the Kinect for Windows and to LB for the voice talent.

This entry was posted in Projects. Bookmark the permalink.

4 Responses to Dragon Pumpkin

  1. Dan Zen says:

    COOL! Amazing idea ;-) What do you think causes the slight lag?

    • Thanks! I haven’t seen any examples of speech recognition that don’t have a bit of lag, so I think it’s just a matter of processing speed. I did note that it’s faster when the application is looking for one command instead of one of many potential commands; I reduced the commands from ‘Back’, ‘Forward’, ‘Turn Left’ and ‘Turn Right’ to just ‘Trick or Treat’, and that did seem to speed things up. The system the app. was running on was blisteringly fast, but it’s a fairly beefy thing to recognize the audio as a command and then interpret it. I’m only working at 9600 baud, so the serial connection could easily be made faster, but I don’t think it would have a perceptible impact.

  2. Pingback: Before you throw out that pumpkin, make a dragon-o-lantern - Hack a Day

  3. Ann Mossop says:

    …I’m not a Bumble Bee …I’m a Halloween! I see that Halloween is still one of your favorite days! That’s my boy! AM

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>