[Plugin] : Unistroke Gesture Recognizer

Edit : <font color="red">Update 08 - february - 2012</font>

New build !http://dl.dropbox.com/u/1412774/oneStrokeGestureRecognizerC2Plugin/pode_1StrokeGestureRecognizer.0.3.zip

Demo : http://dl.dropbox.com/u/1412774/oneStrokeGestureRecognizerC2Plugin/index.html

To appologize, there's a new functionnality : remove a specific gesture, by name

Edit : <font color="red">Update 07 - february - 2012</font>

New build ! http://dl.dropbox.com/u/1412774/oneStrokeGestureRecognizerC2Plugin/pode_1StrokeGestureRecognizer0.2.zip

• Centroid, Bounding Box, Angle
• delete all gesture from the lib (even those given at the start)
• add a custom list of points ! (provided as "X1,Y1|X2,Y2|...|XnYn)

The demo is updated, by the way : http://dl.dropbox.com/u/1412774/oneStrokeGestureRecognizerC2Plugin/index.html

Edit2 : I also added a Custom Gesture Creator : http://dl.dropbox.com/u/1412774/CustomGestureCreator/index.html

http://dl.dropbox.com/u/1412774/CustomGestureCreator/GestureCreator.capx. It's easier for the creation of custom gesture's pointlists...

____________________

I remembered using a library to work with wiimote, helping to recognize gestures made with the gamepad.

That lib is there : http://depts.washington.edu/aimgroup/proj/dollar/

So I made a plugin for C2 today, that helps to recognize gestures made by the user, which uses that lib.

You can get that plugin there : http://dl.dropbox.com/u/1412774/oneStrokeGestureRecognizerC2Plugin/pode_1StrokeGestureRecognizer.zip

I also made a quick demo app, mimicking the one from the original link :

<img src="http://i.imgur.com/6fR9V.jpg" border="0">

http://dl.dropbox.com/u/1412774/oneStrokeGestureRecognizerC2Plugin/index.html

And a demo capx : http://dl.dropbox.com/u/1412774/oneStrokeGestureRecognizerC2Plugin/oneStrokeGestureRecognizerC2PluginDemo.capx

One thing to know : in the original algorithm, the recognition works when there's more than 10 points defined for a gesture.

In a future rev., I'm going to add the possibility to give directly to the plugin a list of points (for example, if you already determined a predefined gesture), and the possibility to load that from a file.

When I have more time, I'm going to integrate the N strokes (multistrokes) Javascript lib from the same team...

Edit : it's up to you to set a level (score) under which the gesture isn't recognized, because the lib will try its best to recognize something anyway. I usually go for something between 75 (it let pass a few bad detection) to 90 (pretty strict detection).

Edit2 : In the original paper, to have a good recognition when you add new gestures, you need to add at least 3 variants of that gesture. So if you want to detect a new gesture, to forget to draw it 3 times and add it 3 times ! :)

Edit3 : the action "Clear Point List" empties any user defined gesture, not gesture pre-packaged with the lib !

I am grinning like your avatar right now.

Wow, Cannot think of a place of where i'll use this in my game, But none the less, Great Plugin!! will look nice in my list of 3rd party plugins <img src="smileys/smiley1.gif" border="0" align="middle" />

I just added two things to the original post I'm just remembering now...

Pode, that is a great plugin !

I figured out how to parse a string and add it via events, only thing is, the default gestures cause a false positive sometimes. Is there any way to take the defaults out?

Very cool and powerful!

excellent!!!!!!!!!

newt : I'm not really sure what you mean by "the default gestures cause a false positive sometimes. Is there any way to take the defaults out?"

If, when you draw something that's not looking something you aleardy have in the library of gesture, the lib will try to match it anyway (but it will get a very bad score). So you can filter false positives with a high score. Is it helping you ?

All : I didn't think about that in the beginning, but with that lib, you can do a "swipe gesture" detection on mobile ;).

I also edited the original post to add something : to have a good recognition when you add new gestures, you need to add at least 3 variants of that gesture.

Ahh, ok the variants, and checking the score should help.

That does make me wonder about adding gestures from externals. I guess you would have to make variants for that as well.

newt : I saw on IRC you proposed an idea to compare old/new points, but I'm not sure where you want to add that.

If it's an idea to simplify new gestures before adding them to the library, it's not necessary. You can use a bunch of points (even if some of them are overlapping), the lib is doing itself the cleaning (it's a kind of Procruste analysis to simplify the input gesture to compare it against the reference).

Some screenshots from the paper :

1: you can see there's some variability in the input gesture

<img src="http://dl.dropbox.com/u/1412774/oneStrokeGestureRecognizerC2Plugin/01.png" border="0" />

2: so the gesture is resampled internally to a predefined number of equidistant points

<img src="http://dl.dropbox.com/u/1412774/oneStrokeGestureRecognizerC2Plugin/02.png" border="0" />

3: the gesture is rotated, always in the same direction, to make it

rotation independent

<img src="http://dl.dropbox.com/u/1412774/oneStrokeGestureRecognizerC2Plugin/03.png" border="0" />

4: finally some error rate regarding recognition, depending on the number of availble samples to compare against (you see there that 3 is the minimum efficient number) - Rubine is another algorithm from the field, and DTW is Dynamic Time Warping, more powerful, but way more ressources hungry (too much for a quick Javascript, at least)

<img src="http://dl.dropbox.com/u/1412774/oneStrokeGestureRecognizerC2Plugin/04.png" border="0" />

Well the question is, if you can avoid introducing duplicate points, would it make any difference?

Also, just to clarify, when adding variants, should you clear the existing points, when adding a new variant?

If you can avoid inputting duplicate point, it's going to speed the cleaning of the shape byt the lib (but I don't how much milliseconds are gained, really).

When you add a new variant, you don't need to clear, because clear empty the list from all user input. I need to add that on the original post.

New build ! http://dl.dropbox.com/u/1412774/oneStrokeGestureRecognizerC2Plugin/pode_1StrokeGestureRecognizer0.2.zip

You can now access the Centroid, the Bound Box and the Angle of the last gesture ! (it helps to create little demo like "Crayon Physics Deluxe ;) : http://dl.dropbox.com/u/1412774/oneStrokeGestureRecognizerC2Plugin/c2physicsDeluxe.capx

You can also delete all the gesture given at the start of the library, and something specially for newt : add a custom list of points ! (provided as "X1,Y1|X2,Y2|...|XnYn)

The demo is updated, by the way : http://dl.dropbox.com/u/1412774/oneStrokeGestureRecognizerC2Plugin/index.html

Cool.

Really nice plug here Pode. Thanks.