Alpha Magic - the Art of being Transparent
Newsletters | PD Howler | PD Artist | PD Particles | Tutorials
This is a WIEP - a work in eternal progress
You probably have done this before:
You drew a piece of foliage on a plain white background image, such as a bush or a tree, with the special particle brushes or foliage brushes, and you had the alpha mode enabled. Maybe you also added extra sharpening, or other filter effects to bring out even more details.
Then you picked up the selection into a custom brush, and disabled the particle brushes, pulled in another background image with darker tones, and started stamping the image from the brush or painting with it..... Oh no!? what's that nasty bright glow around the bush, that glowing bright halo around the tree?
In the example above, the helms of grass were using a gradient that got darker and darker towards the end, so the tips should fade to dark, not show a bright lite-grey or white appearance. After storing the brush to manage it and work with it, we can change the Hue of it, and create variants of different coloration but the white glow remains white of course.
Here is the brush image, saved as PNG:
It is not easy to see that there is a white glow in this case because the image is showing against a white background.
If I put it over a background of different color though, something dark, or black:
|
Or you saved the brush in 32-bit mode (Targa, Tiff, PNG) to use it on a billboard polygon in a 3D scene or game engine, and find that again -oh no! - there is a distinct bright glow showing around the bush or tree. What's up with that? How can it be avoided?
Let's discuss that 'glow' issue
It can often be reduced or eliminated.
Not so easily with animated brushes though. But still, though it is tedious. If you work it frame by frame.
So let's focus on a single frame first.
What's that glow, where is it coming from?
It's in the transition zone from transparent (outside) of the desired visible pixels to opaque (inside the bush or tree). The glow is seen along the edge of the everything that is anti-aliased: the trunk, branches, twigs, leaves.... The thinner the items, the more dominant they seem. This is just because of the relative size: the transition zone of that glow is a fixed number of pixels, whereas the various parts of the bush or tree are wide and big at first near the bottom of the trunk, then get more narrow as it evolves towards the branches, thin branches and twigs and eventually finish as leaves or flowers.
If the flowers are bright or white, you won't notice or mind the white glow there. But it's there alright, strictly speaking.
In that transition zone, along the edges from the inside opaque to the outside transparent world, there isn't just either a fully opaque pixel or a fully transparent pixel:
There is a transition of opacity, going gradually from fully opaque (inside) and visible, to fully transparent (outside). Along that transition zone, there are pixels with transparency that is transitioning from on to off in the alpha channel, somewhere perhaps at opacity values of 10%, 30%, 50-70%,.... due mostly to anti-aliasing. The rendering of the particle brushes has anti-aliasing enabled, to make it look better.
Or perhaps you didn't draw the bush or tree with our foliage/particle brushes. Perhaps it was a photo with a bit of blur or glare which does result in a gradual, colored transition. If the background is a clean blue sky, you might be tempted to use chroma-keying, aka blue-screen techniques to isolate the background from the foreground. That will work, but there is a fine zone where we transition the opacity, while the RGB channels also transition to or from blue. Thus, you'll be expecting a blue'ish tint around the edges, a blue glow.
Not as dramatic as the white glow scenario, but still the same.
If you render a paint stroke with alpha enabled, using for example a foliage brush or particle brush (in Style = color + alpha), there are not just the 3 RGB channels that will need to be anti-aliased: there are 4 channels: the red, green abd blue (RGB), PLUS the alpha channel, which sets the opacity or transparency of the pixel.
The colors are antialiased, just as much as the alpha value is antialiased.
So while you see a change in opacity while looking from inside to outside the edge of the leaves, you also see a change in its coloring. If the outside was plain white, the glow will change to a brighter white'ish appearance. It will eventually disappear as the alpha value turns fully transparent. But before that happens, you stand a good chance of seeing a pixel or two that carries a mid-grey or light almost white appearance. That's your glow.
The eye is sensitive to changes. If it changes slightly in that transition zone, you will notice it.
To what color does it change? Is it always to white? No, of course not. So what color does it transition to? Well, to whatever the colors are in the background before you paint there.
If the entire image was white at first, then you will get a white'ish glow.
If the background was black initially, then the transition turns dark along the edges.
The white glow case won't be noticed if you stamp the brush over a very bright background, until you try to use the image against a dark scene, or overlapping on itself and its dark foliage parts from a prior brush stroke.
Likewise, the case with dark glow will not be noticed much if used over a dark scene. In fact, it may be best to start with a dark background. But still, it's not perfect. If you use that bush or tree image against a bright background sky, it will show a dark halo, at times.
So, what can you do? Is it possible to change the coloring without changing the alpha transition?
Of course it is. And there are some tools that will automate it for some common scenarios, and there are some more and other tools, tricks and techniques you'll want to learn, practice and master for other cases.
Premultiply-Correction
If you have picked up the image into a custom brush, there's an option in the Brush menu to do what's called pre-Multiply correction. Before you use it, we recommend that you store a copy of the current custom brush. Use menu: Brush > Store and manage copy... That way you can easily switch back to this original brush.
Then use menu: Brush > Premultiply correction...
When you want to use this technique, you need to know what the background color was before the foliage was painted. If it was white, tell it that it was white. etc... then let it fix the glow by correcting the transition, removing the white coloring in the anti-aliased zones.
It can often work perfectly. Here's the result when using the improved brush: no more glowing
In the above example, white was the background. You can choose white or black or either primary or secondary color. So if it's anything other than white or black, set the color as the primary or secondary, and choose accordingly.
Here is the improved version of the brush image, again as PNG, saved from the Brush menu after applying pre-multiply correction:
Side-by-side, against white background, and against dark background, and even mid-grey background:
| Without pre-multiply correction against white background: |
After pre-multiply correction against white background: |
|
|
against black background: |
against black background: |
|
|
against grey background: |
against grey background: |
|
|
Ok, so that was the example where we already had an alpha channel containing the selected pixels that should be opaque, while the rest around it is to be shown transparent.
What if you draw something or start with a photo?
If you want it done right...
It can also often be too little too late.
Especially if the glow is not just in the narrow range of pixels that is coming just from anti-aliasing, which is around 1-2 pixels wide, 3 tops.
If instead of painting the brush of a tree you are looking at working from a photo, it is very likely that you have the glow across many more pixels. That can still be dealt with, but is more challenging.
This calls for the saying: If you want it done right,... then you've got to do it yourself."
Something like this, for example: working on the image (not the brush):
- You could invert the selection, then
- Clear the selection (the outside parts) in multiply mode with a color that is the complementary color of the current background that's showing as the glow,
- Then re-invert the alpha back.
That's more or less what happens in pre-multiply correction. We might be off some details but it can be a start. When you see a bright white glow, and it is not what you want, you'll obviously want to make that glow darker. So, reverse the alpha, clear the outside selection to black or something dark, then reverse alpha back.
Then pick up the current selection as the new custom brush.
You might be happy with the results with just one clear operation (try using the keyboard shortcut 'K'). Or you might need to do the clearing twice or more. Witch each iteration, it darkens deeper. The clearing is reduced by the amount of alpha values. So if you need more of it in the transition zone, apply the clear again.
If you have gotten too much darkening in the edges zones by now, perhaps you can tighten or adjust alpha too, or even shrink the alpha. Look for these options in the Selection menu. Adjust alpha is similar to the Adjust color filter, and includes a contrast adjustment. But it won't affect the contrast of the RGB channels. It will work on the alpha channel instead, of course.