2D Selection Outline Shader in LibGDX

For our new and awesome action RTS game Asteroid Fight we wanted a nice selection outline for the units. Everything I found on the web so far was intended for use with 3D games where you could use e.g. the wireframe and render the wires with a thicker line and onto that the actual 3D model so you get the impression of an outline.

So here I will present you a simple fragment shader that you can use to add a nice selection outline to your 2D game.

I will assume that you know LibGDX and that you know how to use shaders with this framework. If not I recommend you go to http://www.gamefromscratch.com/post/2014/07/08/LibGDX-Tutorial-Part-12-Using-GLSL-Shaders-and-creating-a-Mesh.aspx first and work through this tutorial.

GLSL fragment shader code:

#ifdef GL_ES
precision mediump float;
precision mediump int;
#endif

uniform sampler2D u_texture;

// The inverse of the viewport dimensions along X and Y
uniform vec2 u_viewportInverse;

// Color of the outline
uniform vec3 u_color;

// Thickness of the outline
uniform float u_offset;

// Step to check for neighbors
uniform float u_step;

varying vec4 v_color;
varying vec2 v_texCoord;

#define ALPHA_VALUE_BORDER 0.5

void main() {
   vec2 T = v_texCoord.xy;

   float alpha = 0.0;
   bool allin = true;
   for( float ix = -u_offset; ix < u_offset; ix += u_step )
   {
      for( float iy = -u_offset; iy < u_offset; iy += u_step )
       {
          float newAlpha = texture2D(u_texture, T + vec2(ix, iy) * u_viewportInverse).a;
          allin = allin && newAlpha > ALPHA_VALUE_BORDER;
          if (newAlpha > ALPHA_VALUE_BORDER && newAlpha >= alpha)
          {
             alpha = newAlpha;
          }
      }
   }
   if (allin)
   {
      alpha = 0.0;
   }

   gl_FragColor = vec4(u_color,alpha);
}

So what I essentially do is I check every neighboring pixel in the range of u_offset and get the maximum alpha value from those pixels, but if every pixel I check has an alpha value above ALPHA_VALUE_BORDER I set the alpha value to zero, so that means we are completely inside our object, but we only want the outline.

My vertex shader looks as following:

uniform mat4 u_projTrans;

attribute vec4 a_position;
attribute vec2 a_texCoord0;
attribute vec4 a_color;

varying vec4 v_color;
varying vec2 v_texCoord;

uniform vec2 u_viewportInverse;

void main() {
    gl_Position = u_projTrans * a_position;
    v_texCoord = a_texCoord0;
    v_color = a_color;
}

It’s more or less the default vertex shader. Nothing special here.

Loading the shader:

public ShaderProgram shaderOutline;

public void loadShader() {
String vertexShader;
String fragmentShader;
vertexShader = Gdx.files.internal("shader/df_vertex.glsl").readString();
fragmentShader = Gdx.files.internal("shader/outline_border_fragment.glsl").readString();
shaderOutline = new ShaderProgram(vertexShader, fragmentShader);
if (!shaderOutline.isCompiled()) throw new GdxRuntimeException("Couldn't compile shader: " + shaderOutline.getLog());
}

As you can see, in my setup I saved the vertex shader as shader/df_vertex.glsl and the fragment shader as shader/outline_border_fragment.glsl .

And this is how you can use the shader:

// ... previous draw calls ...
batch.end();
shaderOutline.begin();
shaderOutline.setUniformf("u_viewportInverse", new Vector2(1f / width, 1f / height));
shaderOutline.setUniformf("u_offset", outlineSize);
shaderOutline.setUniformf("u_step", Math.min(1f, width / 70f));
shaderOutline.setUniformf("u_color", new Vector3(red, green, blue));
shaderOutline.end();
batch.setShader(shaderOutline);
batch.begin();
batch.draw(textureRegion, x, y, width, height, width, height, 1f, 1f, angle);
batch.end();
batch.setShader(null);
batch.begin();
// ... next draw calls ...

I hope I could end your search for the 2D outline rendering. Hit the Flattr button below if I could help you.

Happy coding and have a nice day! 🙂