Shader API in Processing 2: an updated reference!   11 comments

Processing 2.0 was released almost a year ago, and introduced many exciting improvements in several areas. Among those, a new OpenGL renderer with GLSL shader support. Since that time, the shader API – defined not only by the new functions in the Processing language to load and run shaders in a sketch, but also by the set of uniform and attribute variables inside a shader that Processing automatically initializes with the geometry coming from the sketch – has gone some minor changes and tweaks. The reference section in the Processing website, the PShader tutorial, and the built-in examples are all up-to-date, but the last full listing of all the uniform and attribute variables was mentioned in an older post in this blog, and has gone a bit out of date since then. With the recent release of Processing 2.1.2, I could say (with very high certainty) that the shader API in Processing 2 reached a “stable status” and won’t go through any further changes.

A distinction that was introduced early on in the shader architecture of Processing 2 was that of color, light, texture, texlight, point and line shaders. This notion of “shader type” it only matters to Processing, and is not part of the GLSL language. The type can be indicated by #define line, such as

#define PROCESSING_POINT_SHADER

or

#define PROCESSING_TEXTURE_SHADER

and was used by the OpenGL renderer to pick the right shader to use depending on the specific combination of stroke, lights, and textures at each specific point along the rendering of the sketch’s output. However, the renderer became a little “smarter” in regards to this shader selection process, and so these defines are not longer required (although you can keep them around in the shader code).

The important thing to keep in mind when coding a shader to use in a Processing sketch is the rendering state at the moment when the shader will be invoked by Processing. By rendering state, I simply mean if lights and/or textures are being used in the scene. All the uniforms that I will list below can be added to any type of shader, but if a shader is set to render a scene without lights but it declares a lighting uniform, then that uniform will not be initialized.

Any shaders in Processing can declare the following basic uniform and attribute variables:

uniform mat4 modelviewMatrix;
uniform mat4 transformMatrix;
uniform mat3 normalMatrix;

attribute vec4 position;
attribute vec4 color;
attribute vec3 normal;

uniform sampler2D ppixels;

A couple of convenience aliases: modelview can be used instead of modelviewMatrix , transform instead of transformMatrix, and vertex for position. The ppixels sampler2D uniform – not widely documented – gives access to the previously rendered frame and can be useful to implement iterative effects or simulations (like the Conway’s game of life included in the examples).

Shaders that operate on lit geometry can access the following additional uniforms and attributes:

uniform int lightCount;
uniform vec4 lightPosition[8];
uniform vec3 lightNormal[8];
uniform vec3 lightAmbient[8];
uniform vec3 lightDiffuse[8];
uniform vec3 lightSpecular[8];
uniform vec3 lightFalloff[8];
uniform vec2 lightSpot[8];
attribute vec4 ambient;
attribute vec4 specular;
attribute vec4 emissive;
attribute float shininess;

The lighting model in Processing 2 reproduces that of Processing 1, which in turn followed very closely the lighting model of the fixed-function pipeline in OpenGL 1 and 2, which was limited to 8 light sources (that being the reason of the uniforms arrays having 8 elements).

Similarly, Processing will initialize the following uniforms and attributes when rendering textured shapes:

uniform sampler2D texture;
uniform mat4 texMatrix;
uniform vec2 texOffset;
attribute vec2 texCoord;

The texture sampler gives access to wherever image has been set with the image() or the texture() functions. The Processing API doesn’t allow for multi-texturing, however one can add additional texture samplers and manually link them to a PImage object using the PShader.set() method.

Shaders that receive lit and textured geometry can access any of the variables listed so far.

Finally, we have the shaders that are used to render stroke points and lines. These fall in a separate category, because rendering of stroke geometry is essentially different from that of “normal” geometry, due to the fact that points and lines are screen facing and therefore need a different transformation math (explained in more detail in the section 10 of the PShader tutorial). The additional uniforms and attributes for line shaders are:

uniform vec4 viewport;
uniform int perspective;
uniform vec3 scale;
attribute vec4 direction;

while for point shaders:

uniform vec4 viewport;
uniform int perspective;
attribute vec2 offset;

In both cases, viewport contains the coordinates of the current viewport rectangle (which should be 0, 0, width, height), while perspective is used to pass the value of the ENABLE_STROKE_PERSPECTIVE hint to the shader. When this hint is enabled, then the points or lines are perspective corrected, which means that they will look smaller as they more away from the view position.

Finally, as a (somewhat undocumented) feature in the renderer, you can actually enable OpenGL 3 or 4 profiles, as long as the hardware supports it, and load geometry or tessellation shaders into your sketch!

spheretess

Advertisements

Posted May 8, 2014 by ac in Programming, Uncategorized

Tagged with , , , , ,

11 responses to “Shader API in Processing 2: an updated reference!

Subscribe to comments with RSS.

  1. I just noticed there used to be a pmouse uniform. What was the reason for removing it?

  2. Hi, big thanks for your work on this opengl API in processing, i love that :)
    I don’t know if you have seen this topic on processing forum : http://forum.processing.org/two/discussion/7039/samplercube-and-ambient-reflection
    But i’m stuck since a 6 month on the question about the modelview / modelviewInv matrix (last reply), so if you can help me a bit about that it will be very very great !
    Maybe it’s a bug ?

  3. Hi, I’m just beginning to understand the power of OpenGL specially with creating textured geometries. I’m getting into lighting and I got an error message that says my shader needs to be of TEXLIGHT type when I tried using a point light. Are there any materials on this? Is texlight an openGL file like texvert and texfrag? Much appreciate your help!

    • Hi, these posts are a bit outdated in that the shader type (COLOR, TEXTURE, TEXLIGHT, etc) are no longer required. All you need to do is to define the light uniforms using the naming conventions explained here for a shader you want to use to render lit geometry.

  4. Thanks so much for this. I’ve been searching everywhere on how to tile textures to use in shadertoy shaders and finally stumbled here by accident. Great info. I would politely suggest that it would be great if all this info could be updated and included on the processing website reference. What’s there about pshader and glsl is still very limited and not nearly as detailed as what you have supplied.

    • Hi Martin, thank you for your suggestion! I think that compiling all these materials as a second PShader tutorial on the Processing site could be a possibility.

  5. Thanks for all the great info on shaders and shadertoy porting. Although most of the shadertoy shader work fine in processing, a few like for example, https://www.shadertoy.com/view/XtjSDK and https://www.shadertoy.com/view/4ljGW1, work fine in shadertoy but will throw ‘implicit casting of vec3 to vec2’ errors when ported to processing. The shader code is unchanged except for the usual replacing of the uniforms and renaming of ichannel/fragcoord/fragcolor. Any ideas? I also put the question with my code on the forum here : https://forum.processing.org/two/discussion/15293/certain-shadertoy-shaders-throwing-casting-errors-in-processing

    Thanks for your time.

    • Hi, I think the error is due to differences in the GLSL compilers between WebGL and OpenGL, I will write a more detailed answer on the Forum.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: