2D Selection Outline Shader in LibGDX

For our new and awesome action RTS game Asteroid Fight we wanted a nice selection outline for the units. Everything I found on the web so far was intended for use with 3D games where you could use e.g. the wireframe and render the wires with a thicker line and onto that the actual 3D model so you get the impression of an outline.

outline

So here I will present you a simple fragment shader that you can use to add a nice selection outline to your 2D game.

I will assume that you know LibGDX and that you know how to use shaders with this framework. If not I recommend you go to http://www.gamefromscratch.com/post/2014/07/08/LibGDX-Tutorial-Part-12-Using-GLSL-Shaders-and-creating-a-Mesh.aspx first and work through this tutorial.

GLSL fragment shader code:

#ifdef GL_ES
precision mediump float;
precision mediump int;
#endif

uniform sampler2D u_texture;

// The inverse of the viewport dimensions along X and Y
uniform vec2 u_viewportInverse;

// Color of the outline
uniform vec3 u_color;

// Thickness of the outline
uniform float u_offset;

// Step to check for neighbors
uniform float u_step;

varying vec4 v_color;
varying vec2 v_texCoord;

#define ALPHA_VALUE_BORDER 0.5

void main() {
   vec2 T = v_texCoord.xy;

   float alpha = 0.0;
   bool allin = true;
   for( float ix = -u_offset; ix < u_offset; ix += u_step )
   {
      for( float iy = -u_offset; iy < u_offset; iy += u_step )
       {
          float newAlpha = texture2D(u_texture, T + vec2(ix, iy) * u_viewportInverse).a;
          allin = allin && newAlpha > ALPHA_VALUE_BORDER;
          if (newAlpha > ALPHA_VALUE_BORDER && newAlpha >= alpha)
          {
             alpha = newAlpha;
          }
      }
   }
   if (allin)
   {
      alpha = 0.0;
   }

   gl_FragColor = vec4(u_color,alpha);
}

So what I essentially do is I check every neighboring pixel in the range of u_offset and get the maximum alpha value from those pixels, but if every pixel I check has an alpha value above ALPHA_VALUE_BORDER I set the alpha value to zero, so that means we are completely inside our object, but we only want the outline.

My vertex shader looks as following:

uniform mat4 u_projTrans;

attribute vec4 a_position;
attribute vec2 a_texCoord0;
attribute vec4 a_color;

varying vec4 v_color;
varying vec2 v_texCoord;

uniform vec2 u_viewportInverse;

void main() {
    gl_Position = u_projTrans * a_position;
    v_texCoord = a_texCoord0;
    v_color = a_color;
}

It’s more or less the default vertex shader. Nothing special here.

Loading the shader:

public ShaderProgram shaderOutline;
public void loadShader() {
String vertexShader;
String fragmentShader;
vertexShader = Gdx.files.internal("shader/df_vertex.glsl").readString();
fragmentShader = Gdx.files.internal("shader/outline_border_fragment.glsl").readString();
shaderOutline = new ShaderProgram(vertexShader, fragmentShader);
if (!shaderOutline.isCompiled()) throw new GdxRuntimeException("Couldn't compile shader: " + shaderOutline.getLog());
}

As you can see, in my setup I saved the vertex shader as shader/df_vertex.glsl and the fragment shader as shader/outline_border_fragment.glsl .

And this is how you can use the shader:

// ... previous draw calls ...
batch.end();
shaderOutline.begin();
shaderOutline.setUniformf("u_viewportInverse", new Vector2(1f / width, 1f / height));
shaderOutline.setUniformf("u_offset", outlineSize);
shaderOutline.setUniformf("u_step", Math.min(1f, width / 70f));
shaderOutline.setUniformf("u_color", new Vector3(red, green, blue));
shaderOutline.end();
batch.setShader(shaderOutline);
batch.begin();
batch.draw(textureRegion, x, y, width, height, width, height, 1f, 1f, angle);
batch.end();
batch.setShader(null);
batch.begin();
// ... next draw calls ...

I hope I could end your search for the 2D outline rendering. Hit the Flattr button below if I could help you.

Happy coding and have a nice day! 🙂

Flattr this!

Bookmark the permalink.

3 Comments

  1. Hi,

    First-of-all, thanks! I’ve been looking all over for this!

    However, I tried using your code in a mock-project (basically I created a new libgdx project, and copy-pasted your code) just to see how it would look but unfortunately, all I get is a blank screen. Any ideas what I’m doing wrong? My code is as following:

    import com.badlogic.gdx.ApplicationAdapter;
    import com.badlogic.gdx.Gdx;
    import com.badlogic.gdx.graphics.GL20;
    import com.badlogic.gdx.graphics.Texture;
    import com.badlogic.gdx.graphics.g2d.SpriteBatch;
    import com.badlogic.gdx.graphics.g2d.TextureRegion;
    import com.badlogic.gdx.graphics.glutils.ShaderProgram;
    import com.badlogic.gdx.math.Vector2;
    import com.badlogic.gdx.math.Vector3;
    import com.badlogic.gdx.utils.GdxRuntimeException;

    public class SandBox extends ApplicationAdapter {
    SpriteBatch batch;
    TextureRegion img;
    private ShaderProgram shaderOutline;

    float x = 0, y = 0, height = 256, width = 256, angle = 0, outlineSize = 1f;

    @Override
    public void create() {
    batch = new SpriteBatch();
    img = new TextureRegion( new Texture("badlogic.jpg"));
    loadShader();
    }

    public void loadShader() {
    String vertexShader;
    String fragmentShader;
    vertexShader = Gdx.files.internal("shaders/outline.vsh").readString();
    fragmentShader = Gdx.files.internal("shaders/outline.fsh").readString();
    shaderOutline = new ShaderProgram(vertexShader, fragmentShader);
    if (!shaderOutline.isCompiled())
    throw new GdxRuntimeException("Couldn't compile shader: "
    + shaderOutline.getLog());
    }

    @Override
    public void render() {
    Gdx.gl.glClearColor(1, 0, 0, 1);
    Gdx.gl.glClear(GL20.GL_COLOR_BUFFER_BIT);

    shaderOutline.begin();
    shaderOutline.setUniformf("u_viewportInverse", new Vector2(1f / width, 1f / height));
    shaderOutline.setUniformf("u_offset", outlineSize);
    shaderOutline.setUniformf("u_step", Math.min(1f, width / 70f));
    shaderOutline.setUniformf("u_color", new Vector3(0, 0, 1f));
    shaderOutline.end();
    batch.setShader(shaderOutline);
    batch.begin();
    batch.draw(img, x, y, width, height, width, height, 1f, 1f, angle);
    batch.end();
    batch.setShader(null);
    }
    }

  2. Thank you very much, works wonderfully!
    How would I go about editing the shader so the outline can be drawn outside the bounds of the image?

Leave a Reply