Bugfixes in GSVideo

A new release of gsvideo (0.5.1) that I uploaded today to sourceforge solves a bug in the recenlty added GSMovieMaker, which prevented to write frames to a movie file while playing another movie. This has been solved by including an updated version of gstreamer-java (the underlying bindings that allow Processing to communicate with GStreamer).

Also, a couple of new features/improvements:

• GSPipeline with new implementation that allows to run DV-capture pipelines (tested on Windows only)
• added parameter to set movie quality in GSMovieMaker (although it works only for the Theora codec).

GSVideo hacking at Art and Code conference

Between March 4th and 8th I was in Pittsburgh, where I met with Ben Fry, Casey Reas, Ira Greenberg and Daniel Shiffman to work in the integration of GStreamer with Processing (among other things), as part of the Oxford Project. Incidentally, the Art and Code conference organized by Golan Levin at CMU was taking place during that time, so it was also a great opportunity to know the creators and developers of amazing tools such as vvvv, openFrameworks, Scratch, Pure Data, Max/MSP, Hackety Hack (by the famous Why The Lucky Stiff) and Ruby-Processing.

In fact, the progress with GSVideo was quite substantial. I was able to re-implement the MovieMaker class using GStreamer’s AppSrc element, which allows to push pixel buffers from the Processing sketch into the video pipeline. I tested this new feature by creating ogg movie files encoded with the Theora codec. GSMovieMaker also lets you choose the H264 and XVID4 codecs, but at this point don’t seem to work very well.

The other significant step forward was on the Mac front. The latest version of macports (1.7.0) probed to be flawless to compile all the gstreamer modules (gstreamer, gst-plugins-base, gst-plugins-good, gst-plugins-base, gst-plugins-bad, gst-plugins-ugly and gst-ffmpeg). One issue however is that the default portfiles include a lot of unneeded dependencies on things like gnome-vfs and gtk+. The solution to this problem is to create a local repository of portfiles with these dependencies removed. You can download the portfiles I used from here. Also, I included in the resulting binaries the osxvideosrc plugin to do camera capture with the GSCapture object.

I still need to do some more work to create a stand-alone pkg installer of gstreamer for Mac OSX, and also to release anew release of GSVideo that bundles the new gstreamer binaries. However, you can try these preliminary versions:

gsvideo-20090311.zip

opt.zip

The first file is just the gsvideo library package, which goes into Processing’s library folder.  The second zip file contains gstreamer binaries, and it should be unzipped into the root folder of the main hard drive.

The included examples require a little hack in order to work properly, which is the following: before running any gsvideo function, add the following line:

GSVideo.forceGlobalGstreamer = true;

For example, you could add this line right after the size() function:

void setup() {

size(640, 480);

GSVideo.forceGlobalGstreamer = true;

mov = new GSMovie(…

This ensures that gsvideo finds the gstreamer binaries in /opt/local. This hack won’t be needed once the new version of gsvideo is “officially” released. Another caveat is that the only resolution supported by GSCapture  is 640×480, and probably won’t work on any webcam but the iSights built into the macbooks. These limitations reflect the preliminary stage of the osxvideosrc plugin.

GStreamer WinBuilds wiki and forum

After some hard work by Andoni Morales, we finally put together a wiki specifically devoted to the gstreamer installers for windows (and in the future, for Mac OSX as well). There is also a help forum and a mailing list.

New GSVideo with better OSX support

With release 0.4.6 of GSVideo (download here) users on Mac and Windows don’t need to do any extra install, since the gstreamer binaries are included inside the library package itself (Mac version uses the gstreamer binaries that are bundled with the current nightly build of songbird 1.1) . Version 0.4.5 on Windows already had this advantage, but it also presented a number of bugs that probably made it somewhat useless (most notably, gsvideo wouldn’t not be work when installed in a path containing spaces), and the included version of gstreamer had some library dependency issues. GSVideo 0.4.6 solved these two problems.

So, this version should greatly simplify the use of gsvideo on both platforms. However, there is one caveat on OSX: it requires to set the DYLD_LIBRARY_PATH environmental variable in order to work properly. Because of this, Processing needs to be started from the terminal using the following series of commands:

> cd /Applications/Processing.app/Contents/MacOS
> export DYLD_LIBRARY_PATH=/Users/username/Documents/Processing/libraries/gsvideo/library/
> ./JavaApplicationStub

This assumes that GSVideo was installed in /Users/username/Documents/Processing/libraries. Clearly, this is not optimal and I’m currently looking for a workaround. I wanted to release the library anyways so people can start testing it on Mac.

Update (thanks to Chandler McWilliams): DYLD_LIBRARY_PATH can also be set permanently by editing the ~/.MacOSX/environment.plist file (more info here). The other big advantage of this approach is that Processing can be run normally from the GUI. The file gstreamerOSX-readme.rtf included with the release in sourceforge contains an example of environment.plist. But please be careful when following this method, since you can break other applications (and even the whole system) when modifying the environment.plist file (specially if typos are introduced in it). Create a backup of your environment.plist file if it already exists before making any changes, and don’t replace its entire contents with the sample I posted on sourceforge example, just add the DYLD_LIBRARY_PATH variable. Hopefully, this hack won’t be needed in the future as we find a better way to install gsvideo on OSX.

The GStreamer adventure part II: video capture on OSX

This is a short post, just to give a “preview” of the work currently undergoing on OSX and GStreamer. A year ago, Ole Andre Ravnas, the author of OABuild, released a proof-of-concept GStreamer plugin to do video capture on OSX. Here is the full discussion that took place on bugzilla at that time, and here the plugin code that Ole Andre released. I discovered this code very recently, and together with Daniel Shiffman, we made a few changes to make it work on the latest macbooks. The code is pretty much the same as Ole Andre’s original, but with hard-coded values that better match the video format and framerate of the iSight cameras. We were able to compile it on both OSX Leopard (10.5) and Tiger (10.4). Extremely inefficient (100% CPU usage) but still a promising starting point. Download from here. Once decompressed, run ./configure from the command line, and then change to the src subdirectory and run make. Of course, this requires to have gstreamer already installed on the system, we are currently using macports to do it. Fink seems to be another viable alternative to get gstreamer installed on OSX, but I haven’t tried it myself.