Here we basically want to see how a cartoon shader can be implemented. It works similar to the one that can be found in the Plugin Store.
As you know TheoTown is a pixel graphics game and we don't want nor can modify graphics manually to look cartoonish. Here's where the shader comes into play: We can use it to get a cartoony look by applying an algorithm to the graphics in realtime during redendering.
Some ingredients for a cartoon shader:
- Edge detection to highlight edges - This is the most important thing for a cartoony look
- Smoothing - In most cases we want to smooth the non edge parts a bit
- Frame type detection - So we can apply the shader to specific things (i.e. we don't want to apply it to UI or text)
All of these steps must be implemented in the fragment shader as we want to apply these to individual fragments / pixels. The following steps will build up from the default shader codes.
Edge detection
You may have heard of the so called
Sobel operator which can be used to calculate edges for any pixel of an image based on its neighboring 8 pixel colors. This is an option, however, we certainly do not want to sample all 8 neighboring pixels because that's quite expensive (in fact, texture lookups are probably the most computationally expensive operation in fragment shaders and you should reduce them as much as possible). Instead, we can e.g. use the left and upper neighbor pixel colors and compare them to the center, only.
Code-wise this may look like this:
Code: Select all
vec4 col = texture2D(texture, texCoord);
vec4 leftCol = texture2D(texture, texCoord + vec2(-dUnit, 0.0));
vec4 upCol = texture2D(texture, texCoord + vec2(0.0, -dUnit));
vec4 diff = abs(col - leftCol) + abs(col - upCol);
float edge = length(diff);
The dUnit variable is a uniform float value that contains the distance between two pixels in the world texture in UV-Coordinates. It's effectively 1 / size of the world texture. We can use it to calculate the locations of the neighboring pixels for the lookups. When we visualize these edges it looks like this:
As you can see this filter also detect edges in regions with noise like on the trees. Furthermore, since we only compare the center pixel to the left and upper neighboring pixels, we cannot show edges to transparency on the right or bottom side of buildings (as these pixels will be transparent as well).
So we
may need to sample two more neighbors after all:
Code: Select all
vec4 center = texture2D(texture, texCoord);
vec4 leftCol = texture2D(texture, texCoord + vec2(-dUnit, 0.0));
vec4 upCol = texture2D(texture, texCoord + vec2(0.0, -dUnit));
vec4 rightCol = texture2D(texture, texCoord + vec2(dUnit, 0.0));
vec4 downCol = texture2D(texture, texCoord + vec2(0.0, dUnit));
vec4 diff = abs(rightCol - leftCol) + abs(downCol - upCol);
diff.a *= 2.0;
float edge = pow(0.4 * length(diff), 2.0);
Here I also did some math on the edge value for better results. I consider this quite usable:
Let's slab that edge onto the pixel color using:
Code: Select all
vec4 col = vec4(center.rgb - vec3(edge * center.a), center.a);
and watch this beauty:
You may notice that some edges like the ones on the ground are way too prominent. We'll fix this later.
Smoothing
TheoTown uses a lot of
noise to convey details in its pixels. This is intended but does not fit with a cartoony look. The easiest way to get a smoothed pixel color is to take the average color of all 5 previously sampled pixels:
Code: Select all
vec4 col = 0.2 * (leftCol + upCol + rightCol + downCol + center);
This way we get this:
There's definitely some smoothing involved here. However, look at these strange lines above the buildings. Since our shader does not care about frame boundaries (i.e. the boundaries of the drawn images) surrounding non transparent areas of the world texture can
bleed in.
So how do we fix this? One approach is to multiply the color of neighboring pixels by the center alpha value and to ensure that the center alpha will also be the overall resulting transparency of the fragment:
Code: Select all
vec4 col = ((leftCol + upCol + rightCol + downCol) * center.a + 2.0 * center) / 6.0;
col = vec4(col.rgb * center.a, center.a);
So now we have some nice smoothing without strange lines:
Now the edges to transparency are especially pronounced. This would even be the case if we'd set the edge value to 0.0. The reason for this is pre-multiplied alpha which we have to consider to smooth somewhat correctly. Although not especially important in this context, as we want to draw edges anyway, a more correct code looks like this:
Code: Select all
col = ((leftCol + upCol + rightCol + downCol) + 2.0 * center) / (2.0 * center.a + leftCol.a + upCol.a + rightCol.a + downCol.a);
col = vec4(col.rgb * center.a, center.a);
Frame type detection
It's time to fix the way too prominent lines on the ground. Not only that, look what the shader in its current form does to our UI.
A bit of a background story: Normally, games are rendered using multiple shaders for different parts of the game. As switching between shaders is usually expensive a depth (or z-)buffer is used to be able to draw things out of order without sorting issues. However, for simplicity only one shader is used in TheoTown to draw everything. Furthermore, there's no depth buffer as it's not needed when being able to just "draw things from back to front". As a result. our shader will not only be used for rendering the city but also for rendering UI, text, overlays etc. This is sufficient for the game's built-in shaders, but as shown above we need a way to detect which type of frame we are rendering right now.
For that, we can ask the game to provide us with a frame type identifier that will be assocaited with each vertex (so it's a vertex attribute). In the vertex shader we have to define it as:
To tell the fragment shader the frame type we have to introduce a varying variable that we fill with the content of vType in the vertex shader's main function:
Code: Select all
...
varying lowp float type;
...
void main() {
...
type = vType;
}
The vType attribute will contain the same integer value for all vertices of a quad. We still have to use the float datatype for it for compatibility reasons.
The next step is to "receive" the frame type in the fragment shader. For that we have to define the same varying variable in the fragment shader:
Now as we have the type variable we can start using it in the main function of the fragment shader. We don't want to apply the cartoony look to UI, text, weather, overlay and background frames. For that we can use a if condition and some built-in constants:
Code: Select all
vec4 center = texture2D(texture, texCoord);
vec4 col = center;
if (type != TYPE_UI
&& type != TYPE_TEXT
&& type != TYPE_WEATHER
&& type != TYPE_OVERLAY
&& type != TYPE_GROUND_BACKGROUND) {
vec4 leftCol = texture2D(texture, texCoord + vec2(-dUnit, 0.0));
vec4 upCol = texture2D(texture, texCoord + vec2(0.0, -dUnit));
vec4 rightCol = texture2D(texture, texCoord + vec2(dUnit, 0.0));
vec4 downCol = texture2D(texture, texCoord + vec2(0.0, dUnit));
vec4 diff = abs(rightCol - leftCol) + abs(downCol - upCol);
diff.a *= 2.0;
float edge = pow(0.4 * length(diff), 2.0);
col = 0.2 * ((leftCol + upCol + rightCol + downCol) * center.a + center);
col = vec4(col.rgb * center.a, center.a);
col = vec4(col.rgb - vec3(edge * col.a), col.a);
}
...
This already fixes most issues:
Let's tweak the edges dependent on frame type:
Code: Select all
if (type >= TYPE_GROUND_LAND && type <= TYPE_REGION_EDGE || type == TYPE_ROAD) {
edge *= 0.01;
}
We can use <= and >= to check the frame type against ranges as they are guaranteed to be defined in the order they'll be listed in the next post. Some more tweaking and we get this:
On a side-note: Branching (i.e. if conditions) in shader code should usually be avoided. In contrast to code running on a CPU all GPU threads within a thread pool share a program counter. This means that even if only a single thread will branch into some extra code, the execution time will behave as if all threads entered that code section. In the context of type dependend shader effects I don't think there's a viable alternative. Just so you know.
I'll call it a day from here. Feel free to tweak the shader yourself. I'd be glad to see your modifications
here in the showcase or in the store.