Cross-platform video library for Processing

I recently discovered these Java bindings for GStreamer by Wayne Meissner, and started writing an alternative video library for Processing based on them. My idea is to create an alternative for the built-in Quicktime video library, which is difficult to use on Windows because its dependency on WinVDIG, and doesn’t work on Linux at all.

So I came up with this new gsvideo library, in which I will eventually re-implement all the three classes of the built-in video library: Capture, Movie and MovieMaker. The idea is that the new classes, called GSCapture, GSMovie and GSMovieMaker, will have exactly the same API as the original ones.

Right now, there is a working implementation of GSMovie, which implements all the functionality of the original Movie class, with the exception of the frameRate() and speed() functions. These are the available files:

the library package, that you have to place inside the Processing library folder.

the source code of the library (for now, just the GSMovie class)

And then some examples:

video loop: same as the video loop example that comes with Processing

video scratching: video playback determined by the mouse position

multiple videos: same as video loop, but using to video files

The GSMovie class works exactly the same way as the original Movie, i.e.: data is read with the movieEvent() function, the frame is drawn calling image(), etc.

This library needs GStreamer installed on the system. For Windows, you can download binary packages from here. For OSX, I haven’t found any binary distribution yet, but you can try installing it through Darwin Ports, or manually compiling the GStreamer source code. I know that this latter option seems to work fine, but first you need to install a couple of dependencies:

pkg-config-0.22
gettext-0.17
bison-2.3
glib-2.8.6
libxml2

In Linux, GStreamer usually comes installed by default. If not, binary packages are available for almost all distributions. And there is always the option of compiling from source :-)

20 Responses to “Cross-platform video library for Processing”

  1. I absolutly love this idea. This’ll make life so much easier moving forward.

    What are your thoughts on having GS* classes inherit from their counterparts? I know it isn’t so, well, correct…but that would allow one to make the switch so easily…and let it work with existing libraries. Maybe?

    Movie movie = GSMovie(this, “test”);

    Eh? Also, have you started GSCapture yet? Are you open to public contributions?

  2. codeanticode Says:

    The problem with having the GS* classes inherit from the buil-in video classes is that there might be some internal incompatibilities. GStreamer and Quicktime have a different way of updating the frame data, and handling all of this from the same object would make the source code quite cumbersome. Anyways, since the API of the new GS classes would be exaclty the same as the built-in ones, all you have to do to move from one to the other is to replace the import line:
    import processing.video;
    by
    import processing.gsvideo;

    I haven’t started with GSCapture yet, I know that I’ll have to use the GStreamer input plugins for webcams. In Linux there is a gstreamer-v4l plugin (v4l = video for linux) called Windows or OSX.

    And yes, the project is completely open for public contributions, so I would be more than happy to receive comments, bug fixes, contribs to the code, etc

  3. Thanks a lot for your work; I’m heartened to finally see work going into cross platform video in Processing, and yours looks like the most promising effort.

    GSMovie is performing fine on my Ubuntu 7.10 installation with GStreamer 0.10.14; no skipping or errors. I’ll do some more testing with different formats and sizes, and will get back to you if I get any interesting results.

  4. codeanticode Says:

    Many thanks for you comments and encouragement! I think this project is really exciting, and I’ll try to add as much functionality as possible into the gsvideo library (Capture and MoveMaker classes, etc).

    On a related note, Wayne Meissner, the creator of the java bindings for gstreamer, figured out how to compile gstreamer and all the required plugins on OSX:
    http://groups.google.com/group/gstreamer-java/msg/b0326e6f71d52530?
    so now we’ll be able to test gsvideo on Mac as well.

  5. Wow! I’m really looking forward to having a decent way to create and experiment with movies using Processing on Ubuntu.
    Keep up the good work!

  6. Thanks, this is a great step in a nice direction!

    Your code works in Ubuntu (studio), but the video doesn’t loop… am I missing something?

  7. codeanticode Says:

    Thanks for comments!

    The sample program gstreamer5 seems to be looping fine, but I’ll take another look to check if there is anything wrong. What type of file (and how long the video) were you playing when you saw the problem?

  8. codeanticode Says:

    In fact there was a bug in the calculation of the playback times that caused movies to loop before reached the end. Hopefully the new version I just uploaded fixes that problem:

    http://codeanticode.wordpress.com/2008/02/27/new-version-of-gsvideo/

  9. martin aa Says:

    Hi,

    The looping still does not work in Ubuntu Studio.. I’ve tried mov, mp4 and dv.

  10. Did you try the latest version:

    http://users.design.ucla.edu/~acolubri/processing/gsvideo/gsvideo-lib-0.3.11.zip

    ? I’ve been using it on ubuntu 7.10, and looping works fine.

  11. Hi,

    I can’t reach the file, is the link broken? or is just me? Anyway, can you send me the library?

    Thanks for your work!!!

  12. Hi,

    My website is not working well right now, there seems to be some problem with the server. Anyways, you can download the library from sourceforge:

    http://sourceforge.net/project/showfiles.php?group_id=225389

  13. HI ,

    I just installed the whole library for OS X . . but the performance ist actually slower than using java quicktime or opengl . . why ?

    thanks in advanced

    • Hello,

      Some parts of gstreamer and/or gsvideo probably still need some optimization on OSX.

      It would be very useful if you can report some numbers comparing the performance both with the default video library and with gsvideo. What video formats are the ones where you observe the biggest difference? Also, what version of OSX are you using?

      Thanks,
      Andres

  14. Hello,

    I’m under Debian 5.0 Lenny amd64.
    I’ve a sun-java6-jre from a official repository.
    I’ve a libgstreamer0.10.0 version 0.10.19-3 from official repository and i’ve others gstreamers packages too but in a the same version.

    When I attempt to launch a sketch like loop.pde (it is the same for others sketches), Processing return this message :

    Exception in thread “Animation Thread” java.lang.UnsatisfiedLinkError: Unable to load library ‘gstreamer-0.10′: /usr/lib/libgstreamer-0.10.so.0.16.0: wrong ELF class: ELFCLASS64

    Probably your library is compiled in 32 bits mode and cannot load a 64 bits library.

    Is it possible to compil your processing library in 64 bits mode on my computer ? If I do, how I make this ?

    For multiples reasons I can’t reinstall my system in 32 bits.

    Note : I don’t know java but I have programming and compiling notions.

    Thanks in advanced.

    Eddy.

    • Hello,

      First, make sure that you are using the latest release of gsvideo (0.5, download from here).

      As for using gsvideo in 64 bits system, the problem is not with the library, but with Processing, which comes with a version of java for 32 bits. Some people found that installing the libgstreamer0.10-dev package solves the problem (see the comments on this other post).

      Another trick is to replace the binaries of java that are bundled with Processing with a 64 bits version, for example openJDK. This blog entry explains how to do it. It is in Spanish, but the idea is simple: first install the package java-1.6.0-openjdk-devel,
      then delete the java files that come with Processing by default (rm -R /opt/processing-1.0.1/java) and finally link openJDK to the java folder in Processing (ln -s /usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64 /opt/processing-1.0.1/java).

  15. Hello,

    Thank you for yours tricks : it is operationnal (spanish is not a problem for me). And i’m entierely open source and free now, it’s cool ;-)

    Congratulations : you are speed and great.

    Best regards.

    Eddy.

  16. Very cool!

    I’m trying to make work Video Peggy with your new lib:
    http://www.box.net/shared/g4unkuc9op

    I just need to test it with the webcam since I’m still using your station.mov

    Keep up the wood work, this is amazing :)

    • I’m happy to know you find the library useful. Please make sure that you are using the latest release available on sourceforge. GSVideo has improved substantially since the time of this post.

  17. hi! this is cool stuff! i’m sure linux can be the no 1 solution for physical computing when people like you keep on pushing it further!

    thanks and thanks in advance for all things coming..

    :D

Leave a Reply