10 of the most innovative Kinect hacks

Now that the dust has settled over the Kinect bounty wars that had the open source community in a ‘Kinect-driver’ development frenzy and had Microsoft tap-dance around the whole debacle, we can safely return to the comfortable position of reviewing some of the most innovative features to have spawned from last year’s hack-the-Kinect phenomenon.

The release of an open-source Kinect driver for Windows has seen a huge spike in the number of “hacks”, which range from simple 3D user interfaces to robotics, physical science applications and much more. Here’s our extended list of some of the better Kinect Hacks:

1. Autonomous Robotic Control
   The Roomba, iRobot’s popular autonomous robotic vacuum cleaner, manages to navigate a living space and avoid obstacles while vacuuming your floor. Now, should you ever happen to lose your Roomba remote controller, don’t despair: Akihiro Nakamura has ingeniously invented a Roomba controller application using the Kinect which enables you to control the Roomba using simple arm gestures and a quick and easy calibration process. The app uses OpenNI library , ARToolkit and the FRISK Roomba module. So now you no longer have to worry about losing your favorite robot cleaner’s remote control. Of course, the same can’t be said about losing the actual Roomba.
   
2. Play Angry Birds with Kinect and KinEmote
   Angry Birds, Rovio’s birds-vs-pigs puzzle game has taken the mobile community by storm with over 12 million copies of the highly addictive game being purchased from Apple’s App Store since it’s launch in December 2009. Now, using KinEmote – a Kinect software package available for Windows that allows users to control Windows games and apps with an XBox Kinect camera – you can play Angry Birds using your Xbox and Kinect camera! The app, designed by KinEmote Co-founder Joel Griffin Dodd, uses KinEmote’s virtual mouse function to track a wireless controller such as a wireless mouse with the Kinect camera for movement. Check out the awesome video below:
   
3. Under the HUD: Kinect’s motion capture animated series
   What sets the Kinect aside from your everyday gaming, is it’s ability to interface with a whole range of motion control actions to produce some really cool applications. Of course, when it comes to motion capture for animation, the Kinect doesn’t quite seem like a viable, legitimate solution. But the developers of a new animated web series, titled Under the HUD, say otherwise, developing the comedic series using motion capture with the Kinect, OpenNI, Brekel, MotionBuilder and Maya. It’s a really smart way to endorse motion capture software at a fraction of the cost. Check out the “making of” video below:
   
4. Quadrotor Autonomous Flight and Obstacle Avoidance with Kinect Sensor
   Okay, so the title may seem like a mouthful but what it essentially is, is a Pelican quadrocopter (a sort of helicopter but with four rotors), with a Kinect camera attached which zooms around  avoiding objects on a predetermined track. Developed by Berkeley’s Hybrid Systems Lab researchers Patrick Bouffard & Professor Claire Tomlinat, as part of the STARMAC Project, the application uses the OpenKinect driver to gain access to the hardware. The software part’s a little more complicated, incorporating a consensus algorithm to map points on a floor and feeding this planar model into the controller, managed by Ubuntu OS running on Intel Atom  1.6 GHz computer:
   
5. Kinect And Half Life 2 Engine From Yelp
   Another motion capture application, this time to track skeletal movements within 3d space and mapping onto Valve Software’s interactive virtual world Half Life 2 source engine has been developed by Yelp engineer John Boiles. The code is open source and all the technical details are available from his blog. Check out the demo below:
   
6. Multi-touch Adobe Air device
   Being able to use the Kinect as a multi-touch input device for Adobe Air apps such as Google Maps definitely beats having to point, click and zoom with your mouse. The AS3 / OSCeleton library combination parses skeletal joint information into Adobe Air and, using simple hand genstures for zoom and rotate, navigate your way through the application’s interface.
   >center>>/center>

   Using Kinect as a multitouch device in Adobe AIR (Google Maps) from Happy Banana on Vimeo.

7. Kinect for World of WarCraft
   The ever popular MMORPG has also received the Kinect-hack treatment, incorporating more streamlined motion such as leaning forward to walk and using your right and left arms to initiate attacks and perform other feats. This hack comes from the University of Southern California’s Institute for Creative Technologies, whose Flexible Action and Articulated Skeleton Toolkit (FAAST)  lets them quickly harness Kinect’s image processing and motion-sensing powers to basically tweak gaming motion to their desire. The team have already developed several gaming hacks to the Kinect, including Streetfighter and the recently released DeadSpace 2.
   
8. Mouse and key control for Mac OS X with Kinect Gesture
   Going one bold step forward and taking the online Kinect-hacker community from game-centric, hobby-based hacks to more serious noteworthy applications has been the goal of developer T.Nishibayashi who has developed an app which allows him to control his Mac OS X via the Kinect. The interface uses the OpenFrameworks and ofxKinect libraries to render a seamless gesture based control system to the operating system which allows you to drag and drop windows and even utilize the Mac’s multi-window swipe display with both your hands. It seems we’re one step closer to those awesome interfaces Tom Cruise wowed us with in 2002’s sci-fi blockbuster Minority Report.
   

   Kinect Jesture for Mac from T.Nishibayashi (hagino3000) on Vimeo.

9. Point recognition for grocery store items
   Another very ‘real-world’ application, this Innovative Retail Labs system recognizes the exact item you’re pointing to, bringing up a visual display with more engaging detail. While I don’t expect retail outlets to start porting Kinect sensors onto the shelves any time soon, the advantages of being able to map out products while shopping via gesture recognition are many: Apart from the informative data, users could also see a count on stock items, produce and shipping dates and possibly be able to pre-order, sync information to a mobile data device and even share across various social networks – adding value to retail marketing and overall consumer satisfaction.
   
10. Virtual reality with 3D TV
    While the Kinect may have been developed as an augmented reality device, software creator Robert Kooima uses a 3D TV along with OpenNI and the Electro VR application to create the illusion of a virtual reality world. The system tracks the user’s presence and the VR application renders two separate images on the TV, so that the user perceives the images as an extension of reality or a ‘virtual reality’ scenario. We’re still a long way from full on virtual reality environments due to hardware constraints (a virtual reality environment would need to rendered in full 360 degree views) but this software is a small step the right direction.