GLGraphics 0.9.4

The version 0.9.4 of the GLGraphics library is available for download. This release is very significant because it incorporates a number of improvements and fixes suggested over the last few months by users of the library (enhanced Cg and GLSL shader support is one of such improvements), and also because this is the last release before the integration with the ongoing work I’m doing in the 3D renderer for Processing Android. For a full list of changes, fixes and issues please take a look at the release notes.

GLSL and Cg shaders

Using code from Victor Martins‘ shaderlib as the starting point, I revamped the API of the  GLSLShader class, and added two entirely new classes: GLCgShader and GLCgFXEffect. These two provide support for Cg shaders and the effect format CgFX (both created by NVidia). CgFX allows to define sophisticated rendering effects by encapsulating several Cg shaders together. Another interesting thing about CgFX is that there is a visual editor for it, the FX Composer. Cg and CgFX require the separate install of the Cg toolkit from NVidia.

The GLSLShader and GLCgShader classes expose a series of methods to access the parameters of the shader, which could be either uniform or attribute variables in the case of GLSL. Take a look at the following code snippets:

GLSL shader example:

void setup() {
  size(800, 600, GLConstants.GLGRAPGHICS);
  ...
  shader = new GLSLShader(this, "vertex.glsl", "fragment.glsl");
  ...
}

void draw() {
  ..
  shader.start();
  shader.setFloatUniform("fog_factor", 0.1);
  shader.setVecUniform("camera_pos", 0, 100, -100);
  shader.setMatUniform("kernel", 0.3, 1.3, 0.0, -2);

  ... Geometry is drawn here ...

  shader.stop();
  ...
}

Cg shader example:

void setup() {
  size(800, 600, GLConstants.GLGRAPGHICS);
  ...
  shader = new GLCgShader(this, "vertex.cg", "fragment.cg");
  ...
}

void draw() {
  ..
  shader.start();

  // We have to set the vertex program in order to modify its parameters:
  shader.setProgram(GLConstants.VERTEX_PROGRAM);
  shader.setVecParameter("dir", 1, 1, -);
  // Passing the modelview-projection matrix from OpenGL to Cg
  shader.setModelviewProjectionMatrix("worldViewProj");

  // And then the fragment program...
  shader.setProgram(GLConstants.FRAGMENT_PROGRAM);
  shader.setFloatParameter("fog_factor", 0.1);
  shader.setVecParameter("camera_pos", 0, 100, -100);

  ... Geometry is drawn here ...

  shader.stop();
  ...
}

Please consult the online API reference for a full specification of the methods you can use in GLSLShader, GLCgShader and GLCgFXShader to maniuplate shader’s parameters.

NVidia hosts a large library of CgFX effects, many of them looking really great. However, the use of CgFX effects from plain OpenGL/Java code seems to be tricky, as it is explained here and experienced myself. Testing of CgFX in GLGraphics has not been extensive, although the GLCgFXShader class should be able to load full blown CgFX files, which could even link to include files. Your success running CgFX effects from Processing using GLGraphics will depend on the platform, graphics hardware and version of the Cg toolkit installed.

Below you can see the output of the GLSL shader examples included in GLGraphics 0.9.4, the first one a toon shader and the second a fish eye lens effect:

Future plans (GLGraphics 1.0 and A3D)

GLGraphics evolved in a rather unplanned fashion. It started as a library only for OpenGL textures and filters, but it now includes a full Processing renderer, an offscreen drawing surface, VBO models, etc., features that many people find useful for their creative projects (take a look at a sample of visuals generated with Processing and GLGraphics: 1, 2, 3, 4, 5, 6, 7). The problem with this sort of “organic” growth is that many times the API is not entirely consistent, and sometimes is definitely clunky. The overall plan has always been to somehow fix all these issues with an ideal 1.0 release. In the meantime, however, I got involved in the development of the 3D renderer for the Android version of Processing (wiki, forum). This actually made the roadmap for GLGraphics much more clear.

The 3D renderer of Processing for Android, called A3D, is a pure OpenGL ES 1.1 renderer that solves many of the issues found in the default OPENGL renderer of Processing: a somewhat messy mixture of software and hardware rendering (in fact, most of the stages of the graphics pipeline in the OPENGL renderer such as transformation and lighting are implemented in software, while using the GPU only to blit triangles as the last step), reliance on direct rendering (i.e.: glBegin/glEnd with lots of glVertex calls in between) which is very inefficient for large geometries and has been made obsolete in the latest releases of OpenGL. GLGraphics managed to solve some of these issues, but still depends on the OPENGL renderer for most of the drawing operations.

The good news is that A3D implements the entire rendering pipeline in the GPU, using vertex arrays for fast vertex pushing. Furthermore, GLGraphics classes such as GLTexture and GLModel have been integrated as part of the android core (PTexture, PShape3D), and fonts were properly implemented for accelerated text rendering with OpenGL. In fact the 3D mode in Android Processing is already usable, and most of the OpenGL examples that come with Processing work reasonably well on Android devices:

Since during the development of A3D I fixed most of the API inconsistencies present in GLGraphics, the plan is to backport A3D into GLGraphics (removing the OpenGL ES specific features) so the latter becomes the 1.0 version. In this process, some of the advanced functionality of GLGraphics, such as shaders and physical simulations on GPU (particle systems for instance) might be split into a separate library. This means that GLGraphics 0.9.4 represents the last release with the current API. Bugfixes could make their way into eventual 0.9.4.x packages, but the next major release of GLGraphics moving towards 1.0 will take the A3D code into the PC/Mac platform.