Live animation of freehand strokes   4 comments

One of the goals of Andiamo is to implement a system where real-time rotosketching and animation can be performed  live using video material as the source. Two technical elements are required in order to accomplish this goal. One, a mathematical representation of free hand strokes that allows to attach the strokes to moving features while preserving their overall shape. Second, an algorithm to extract and track relevant motions in a video feed.

A Bezier representation of free hand strokes seems to be a convenient model, since it allows to easily control the shape of the curves by means of anchor and tangent points, while being computationally simple. The only difficulty consists in finding the right Bezier parameters for a given user-drawn stroke, which can be potentially composed by thousands of individual points. This is however a problem that has been solved in the late 80’s by Philip J. Schneider, who proposed an algorithm to fit a set of 2D points to a multi-segment Bezier curve (see details here and here). This is in fact the algorithm used in the open source vector graphics tool Inkscape for its freehand mode,  and probably in commercial applications such as Adobe Illustrator as well.

The parametrization of Bezier curves with physical-like constraints allows to accurately attach curves to image features such as lines and edges. The Snakes algorithm, which also dates back to the 80’s, represents the first one of the so called active contour models where the minimization of a certain “image energy function” keeps the curves locked to nearby edges.

I’m investigating these techniques in the context of my work with the live performance tool Andiamo. I ported Inkscape’s mulit-segment Bezier fitting algorithm to Processing. As a first step, I used the traer physics library, to consider each control point along the Bezier curves as a particle connected to the rest of the curve’s particles by a set of spring forces. This creates a curve than can be treated as a “solid” object but also has some flexibility. The following applet offers an interactive demo of these experiments:

Multiple-segment Bezier fitting + Physics Demo

If the curve’s points are attached to anchor points by means of additional spring forces, displacements or rotations on the anchor points are transmitted to the whole curve. The idea is to integrate these curve drawing methods, as well as the physics simulations, into a new version of Andiamo. Andiamo already incorporates a robust feature tracking algorithm, KLT, so it will be easy to combine the Bezier curve drawing witht the feature tracking module.

While researching these algorithms I found the work from computer scientist Aseem Agarwala. He investigated automatic rotoscoping techniques by using exactly the same approach I’m currently developing: bezier representation of hand strokes + KLT feature tracking. These two Agarwala’s projects, Keyframe-Based Tracking for Rotoscoping and Animation and SnakeToonz, illustrate the feasibility of these techniques and their potential application in the context of live performance. This video is particularly illustrative.

Advertisements

Posted February 21, 2009 by ac in Programming

Tagged with ,

4 responses to “Live animation of freehand strokes

Subscribe to comments with RSS.

  1. Hello, compliments for this melting of Inkscape with Processing. I really enjoyed it. I started some time ago a somewhat similar applet, that – while is much less intuituve on the drawing point of view – has the advantage of serving processing code directly from screen. Maybe you could also add some similar UGC feature? I bet it would be very entertaining and would open up a lot of possibilities!

    • Hi! Thanks for your compliments and feedback, Giulio. I think it would be easy to add a feature that converts a given freehand drawing into code. The bezier curves that are generated to fit a freehand gesture are stored in an internal array of BezierCurve objects, and each one of them contains the anchor and control points of the curve.

  2. hi CAC. stumbled across your page. Im noting that the term ‘Live Animation’ is finally making it into the public psychee recently. in 2003 i developed a research project at the RCA and developed a side project coining the term ‘ LIVE ANIMATION in ‘2004′. email me back if you want to chat further… exchange ideas.

  3. Thanks for your comments and feedback!

    Yes, seems that people have been doing experiments with live animation/drawing for a while. But computers are turning “live animation” into a more widely known performance practice. For instance, this is the blog of a Buenos Aires artist doing a lot of “live drawing” work:

    http://dibujarinstantaneas.blogspot.com/

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: