Kinect Air Cursor: Let your hand be the mouse

cover image for Kinect Air Cursor: Let your hand be the mouse

If today’s Google I/O keynote where they parachuted to the conference center from a Zeppelin while streaming the whole experience on a Hangout via Project Glass wasn’t enough future for you, here is another thing.

As part of the SmarcoS project, we’ve been working on making the Kinect work as an input device for Qt applications. Basically you move your hand in the air, and are able to grab and drop things on the screen.

We call this the Air Cursor. Here is a quick video of manipulating a simple HTML5 application with it:

Now, this may not be the way you want to control the computer you’re working with the whole day. Instead, we see this sort of interface as very useful for large displays in meeting rooms and public spaces.

Instead of a touchscreen that easily gets messy and requires people to stand in front of it, with the air cursor you can use a regular TV or projector, and use your hands to manipulate the information on it. The gestures we use are natural enough that everybody we’ve had trying the tool has figured them out in matter of seconds.

Our Qt Air Cursor is free software under the LGPL license, and is built on top of the OpenNI library, with OpenCV used for recognizing the grab gestures.

I believe this is a great start for using natural interaction to control information software or multimedia applications. Simple gestures like grab-and-drop and swipes work, but there is still a lot of UX territory left to explore.

If you have ideas where this sort of new input techniques could be used, feel free to get in touch. Or simply to integrate the Qt Air Cursor library into your applications.

The Qt Air Cursor was demoed for the first time in this year’s Qt Contributor Summit in Berlin. Our simple “Grab to the Future” example game gathered quite a large audience, with the high score ending up at a respectable 18. You know you’re doing something right when the event catering staff also wants to try your input device demo.