GSVideo 0.8   18 comments

A new stable release of the GSVideo library for Processing, 0.8, is available for the Linux, Mac OSX and Windows platforms. This version includes important fixes in memory and thread handling as well as new functionality for media playback and video capture. Follow this link to download the library packages, or keep reading for more details.

Memory handling

Thanks to the work of the Data Arts team at Google Creative Labs (Jono Brandel, George Brower, and Aaron Koblin), which used GSVideo to make a touchscreen video gallery based on the Life in a Day movie, I detected some memory leaks that affected the library when recreating moving objects. Specifically, the following code:

  GSMovie movie = new GSMovie(this, "clip1.avi");
  ...
  movie.delete();
  movie = new GSMovie(this, "clip2.avi");

would result in the memory used by the first video not being properly disposed. This leak was not noticeable when using a small number of video objects, but in the case of the LIAD installation, thousands of videos needed to be continuously played back over a period of several days, which made the leak apparent. This issue has been solved in this new stable release.

Below you can see a documentation video of the LIAD touchscreen gallery, shown during the Sundance Film Festival this year as part of the Life in a Day premiere. The installation is now being exhibited at the YouTube Symphony event in Sydney, Australia.

Threading issues

Charl Botha found the solution for random crashes that often occurred when capturing live video using the GSCapture class. His patch is now part of the 0.8 release. The issue was caused when different threads tried to access the same video buffer. Although this seemed to affect only the video capture, I applied the improvement contributed by Charl to all GSVideo classes, not only GSCapture. So if you were experiencing random crashes with the GSMovie, GSPipeline or GSPlayer classes, then this release might solve them as well.

Local library path

The GSVideo package for Mac OSX and Windows comes with the GStreamer libraries bundled with it. They are located in library/gstreamer/macosx or library/gstreamer/win. However, in certain situations it could be convenient to place the GStreamer libraries in a different local folder. For this purpose there is a new static variable that controls the specification of this folder:

  GSVideo.localGStreamerPath = "/Users/gst/libraries/gstreamer/macosx";
  movie = new GSMovie(this, "clip.avi");

This variable also comes handy when using GSVideo from an Eclipse project, to give more flexibility to the placement of the native libraries. Thanks to Charles Bourasseau for his suggestions and code about this issue.

New functionality

  • The GStreamer libraries bundled in the Snow Leopard 64 bits package include a new capture plugin called qtkitvideosrc, based on the QTKit media framework for OSX. At this point, the qtkitvideosrc plugin is accessible only through the GSPipeline object, but it will be automatically used in GSCapture as well as it becomes more mature.
  • The GSPlayer and GSPipeline classes introduce a new RAW mode that allows to grab the video or audio frames from the stream right after decoding and without any additional transformations (color conversion, for example), just as a byte arrays for custom processing within the Processing sketch itself. Please check the Pipelines/Raw and Player/Raw examples for further details.
  • The GSMovie class now has an additional jump() method for moving into a specific frame in the video. It also adds the length() and frame() methods to query the total duration of the media measured in numer of frames, and the current frame, respectively.
  • Several enhancements and simplifications in the API of all the classes, as well as many bugfixes. Please check the online API reference and release notes for more information.
About these ads

Posted March 15, 2011 by ac in Programming

Tagged with , , , ,

18 responses to “GSVideo 0.8

Subscribe to comments with RSS.

  1. Hi, I’m trying GSVideo 0.8 under Ubuntu 10.04 and Processing 0194 and when I run the “GettingStartedCaptureLinux” example it gives me this error:

    GSVideo version: 0.8
    Exception in thread “Animation Thread” java.lang.UnsatisfiedLinkError: /tmp/jna8705905599975051361.tmp: /tmp/jna8705905599975051361.tmp: mappatura del segmento dall’oggetto condiviso non riuscita: Operazione non permessa
    at java.lang.ClassLoader$NativeLibrary.load(Native Method)
    at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1803)
    at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1699)
    at java.lang.Runtime.load0(Runtime.java:770)
    at java.lang.System.load(System.java:1003)
    at com.sun.jna.Native.loadNativeLibraryFromJar(Native.java:761)
    at com.sun.jna.Native.loadNativeLibrary(Native.java:685)
    at com.sun.jna.Native.(Native.java:109)
    at org.gstreamer.lowlevel.GNative.loadNativeLibrary(GNative.java:48)
    at org.gstreamer.lowlevel.GNative.loadLibrary(GNative.java:45)
    at org.gstreamer.lowlevel.GstNative.load(GstNative.java:42)
    at org.gstreamer.lowlevel.GstNative.load(GstNative.java:39)
    at org.gstreamer.Gst.(Gst.java:59)
    at codeanticode.gsvideo.GSVideo.initImpl(GSVideo.java:129)
    at codeanticode.gsvideo.GSVideo.init(GSVideo.java:86)
    at codeanticode.gsvideo.GSCapture.init(GSCapture.java:391)
    at codeanticode.gsvideo.GSCapture.platformInit(GSCapture.java:370)
    at codeanticode.gsvideo.GSCapture.(GSCapture.java:64)
    at GettingStartedCaptureLinux.setup(GettingStartedCaptureLinux.java:57)
    at processing.core.PApplet.handleDraw(Unknown Source)
    at processing.core.PApplet.run(Unknown Source)
    at java.lang.Thread.run(Thread.java:662)

    It occurs even with version 0.7 of the lib.
    However both versions work perfectly under Processing 1.2.1
    Thank you for your great work!

    • Thanks for the info, I’m able to reproduce this issue also on Mac OSX. The solution would probably involve recompiling the GSVideo lib using the latest jar files from Processing 194+. I cannot do it right now, but I added the issue to the sourceforge tracker and will try try to fix as soon as possible,

  2. hi,

    I am trying to install GSVideo 0.8 on Snow Leopard 10.6.7, and Processing 1.2.1, newly installed.

    I have not installed Gstreamer from macports. Since I am confused about the 32-bit vs 64-bit issue I’ve been reading about elsewhere in forum and on this site, I’m trying to both the 32-bit and 64-bit packages. Both don’t work:

    With the 64 bit package I get this error:

    GSVideo VERSION: 20110203
    processing.app.debug.RunnerException: UnsatisfiedLinkError: Unable to load library ‘gstreamer-0.10′: dlopen(…/sketchbook/libraries/GSVideo/library/gstreamer/macosx/libgstreamer-0.10.dylib, 9): no suitable image found. Did find:
    …/sketchbook/libraries/GSVideo/library/gstreamer/macosx/libgstreamer-0.10.dylib: mach-o, but wrong architecture

    With the 32-bit package I get this error:
    GSVideo VERSION: 20110203

    GLib-ERROR **: gmem.c:405: overflow allocating 1635021569*4 bytes
    aborting…

    Any idea what’s going on?

    • Hi, from the error message you posted, it seems that you are using an older test release (20110203). There is a newer stable release (0.8) that I’d recommend you to use instead.

      As for wether installing the 32 or 64 bits package, if you want to run gsvideo from the Processing, you need the 32 bits one. The 64 bits package is intended to use only from Eclipse at this point.

      • hi ac,

        strangely the ‘latest file’ link on SourceForge points to
        GSVideo-20110203-snowleopard64.zip and I must’ve followed it by mistake.

        I’ve tried it with this file GSVideo-0.8-snowleopard32.zip and I have the same error:

        GSVideo version: 0.8

        GLib-ERROR **: gmem.c:405: overflow allocating 1635021569*4 bytes
        aborting…

        —————

        Process: java [90783]
        Path: /usr/bin/java
        Identifier: com.apple.javajdk16.cmd
        Version: 1.0 (1.0)
        Code Type: X86 (Native)
        Parent Process: JavaApplicationStub [90764]

        PlugIn Path: /Users/username/Desktop/sketchbook/libraries/GSVideo/library/gstreamer/macosx/libglib-2.0.dylib
        PlugIn Identifier: libglib-2.0.dylib
        PlugIn Version: 2801.2.0 (compatibility 2801.0.0)

        Date/Time: 2011-04-07 03:23:35.730 +0800
        OS Version: Mac OS X 10.6.7 (10J869)
        Report Version: 6

        Interval Since Last Report: 574162 sec
        Crashes Since Last Report: 11
        Per-App Interval Since Last Report: 120211 sec
        Per-App Crashes Since Last Report: 11
        Anonymous UUID: 87035FE7-06D6-4DD4-8005-6E8BFE3F84BA

        Exception Type: EXC_CRASH (SIGABRT)
        Exception Codes: 0×0000000000000000, 0×0000000000000000
        Crashed Thread: 23 Java: Animation Thread

        Application Specific Information:
        abort() called

      • I never seen this error before. It is actually not gsvideo crashing, but a required library (glib). I also see you are using Mac OS X 10.6.7. I wonder if this is somehow related. I compiled gsvideo and all the required libs on OSX 10.6.6, which is the version I’m still using on my system.

        I will update to 10.6.7 and will let you know if I find anything.

      • I updated to OSX 10.6.7, and I don’t see the error. I tested with Processing 1.2.1. What version are you using?

      • Hi ac, I am using Processing 1.2.1. Have tried with two Macs (10.6.7) now, and the error occurs on both machines. I also tried switching modes between 32-bit & 64-bit kernel but I also get this error.

        However, I’ve installed Gstreamer 0.10.29 using David Liu’s package http://itee.uq.edu.au/~davel/gstreamer/ and now GSVideo works! :)

      • Ok, thanks for the update. The main difference between the gstreamer binaries that I included with gsvideo 0.8 and those from David’s installer, is that the former are built using newer source code.

        Did you try using gsvideo 0.7? Do you get same error?

      • I simply swapped the GSVideo 0.8 folder in /libraries with 0.7, and got this error:

        processing.app.debug.RunnerException: UnsatisfiedLinkError: Unable to load library ‘gstreamer-0.10′: dlopen(/opt/local/lib/libgstreamer-0.10.dylib, 9): no suitable image found. Did find: /opt/local/lib/libgstreamer-0.10.dylib: mach-o, but wrong architecture

        swapped 0.8 back in and no error.

        FYI I spent quite some time trying to get 0.7 to work on a Mac a couple of months but I had all these errors and gave up since my target app is on Linux anyway. So glad it’s working now.

  3. sorry if this is cross-posting, I have another question related to this previous thread in the Processing forum. To recap:

    I tried version 0.8 also with Ubuntu and video appears in a new window outside of the processing canvas. This problem first appeared with GSVideo-20110116

    My current working version is GSVideo-20110111, where video correctly renders using GLTexture. You said this problem may have to do with Playbin2 replacing Playbin in the newer versions — and some conflict with the Fluendo codecs I was testing.

    Since I think my WIP app is also being affected by this memory leak you mention, and solving the problem seems a bit more urgent, first I wonder if it’s possible to select to use Playbin instead of Playbin2, while keeping the new functionalities v0.8 provides?

    • Hi again. First thing: the memory leaks I mentioned earlier should be fixed in 0.8. Can you run this version on Ubuntu?

      Second: it is currently not possible to switch to Playbin. But, I think it makes sense to add an option in GSVideo to select the desired playbin object. I will try to make a test package including this option.

      Also, an important thing to keep in mind when using GSVideo on Linux: changing the volume in the GSMovie or GSPlayer classes can result in crashes. This is due to a bug in glib versions earlier to 2.6. See these posts:
      http://code.google.com/p/gstreamer-java/issues/detail?id=70
      https://bugzilla.gnome.org/show_bug.cgi?id=640872
      So it is highly recommended you update your copy of glib to 2.6 or newer.

      • Hi ac, thanks for the tip about the glib bug; I have not encountered it yet I’ll be sure to check if I’m affected.

        I’ve tested GSVideo 0.8 with Ubuntu and it works fine. I notice 1080p video is above 25fps with KDE loaded! (I tried playback only / not tried capturing but on that note I just tried capture with OSX and it works!)

        Just for a further note:
        I’m not sure if it’s worth to support playbin since it’s deprecated — the problem I had which prompted the question is that playbin doesn’t autoselect gstreamer-fluendo while playbin2 does… but as you have noted in the forum, it could be gstreamer-java’s problem with fluavsink and since I found this thread, it does appear that the issue has to be addressed at gstreamer-java. Thanks for the help!

      • My suspicion is that we need to write a new wrapper object in gstreamer-java to properly handle the fluendo sink elements.

      • hi again,

        I am now grokking the gsvideo & gstreamer-java code to see what can be done to assist

        I noticed that you just added a new class, GSGLVideo and will try that soon.

        But first a quick question: GSGLVideo uses NativeDataAppSink while GSVideo uses RGBDataAppSink … seems that the difference is that the latter utilises an IntBuffer while the latter uses a (vanilla) Buffer. Is there a reason that GSVideo cannot use a NativeDataAppSink instead?

      • Hello, my idea is to integrate GSGLVideo into GSVideo for the next stable release. Now it is there only for testing and debugging.

  4. Hi, I spent a few days to figure out the error: Unable to load library ‘gstreamer-0.10′ . But there is no hope.
    I tried GSVideo 0.8 and 1.0 in both Processing and Eclipse both 32bit and 64bit. I got the same error.
    I tried Gstreamer 0.10.29 using David Liu’s package. It seemd to be out of date, I could not install it.
    Please help if you could.
    Here is my system info: MacOSX 10.7.2, Processing 1.5.1, Eclispe Indigo ReleaseBuild id: 20110615-060

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 60 other followers

%d bloggers like this: