AuthorTopic: Anti-aliasing explained  (Read 10088 times)

Offline QuickSilva

  • 0010
  • *
  • Posts: 125
  • Karma: +0/-0
    • View Profile

Anti-aliasing explained

on: January 24, 2009, 08:17:54 am
One thing that has always troubled me with pixel art is achieving decent looking anti-aliasing. What I`m hoping for is a proven method that will always get me good results as some people seem to do it well and others not.  Is there a methodical approach that works or do you guys just apply it by eye?

Does anyone have any good tips that they have picked up over the years? For example do you anti-alias only around your original lines or do you replace some of them with anti-aliasing? Do you use only one colour or several to smooth out your lines? At which point does it go from looking good to becoming a blurry mess?

I`ve never really found a good tutorial for this so any help would be much appreciated, by lots of people I expect. I`m not after a quick fix but want a solid foundation to build on after many failed attempts myself.

Like I have said, a proven technique that always works would be good but I`m under no illusions as to it being quick and easy to do, good work of any type requires effort, time and practice.

Thanks for any help,
Jason.

Offline Helm

  • Moderator
  • 0110
  • *
  • Posts: 5159
  • Karma: +0/-0
    • View Profile
    • Asides-Bsides

Re: Anti-aliasing explained

Reply #1 on: January 24, 2009, 08:47:50 am
I'd say half is replacing, half is adding for me at least.

It becomes a blurry mess when the antialias starts banding around the line.

Sometimes no-aa is the way to go on specific edges, the sharpest you want it to look, the less aa you should apply.

The real issue is banding and how to deal with that, AA is more a matter of taste and application of an effect once you realize how the computer monitor works, what pixel rows do and what banding does.

Offline Ai

  • 0100
  • ***
  • Posts: 1057
  • Karma: +2/-0
  • finti
    • http://pixeljoint.com/pixels/profile.asp?id=1996
    • finticemo
    • View Profile

Re: Anti-aliasing explained

Reply #2 on: January 24, 2009, 09:53:41 am
see my tutorial at:
http://www.pixeljoint.com/forum/forum_posts.asp?TID=706&PID=74417#74417
and the few posts preceding and following it.

It provides a sound basic method, along with the underlying maths. This method, unmodified, antialiases evenly inside and outside the shape, to maintain the apparent size of the object.  Of course depending on which part is 'over' the other part, you may find it more effective to antialias mostly outwards or mostly inwards. The amount of colors -- pretty much whatever you can afford; sometimes you will want to use existing colors for to unify things more, other times adding.
There are various tricks to avoid banding and discontinuity in antialiasing, which I've covered here:
http://neota.castleparadox.com/aa_tutorial.html

How I learnt lighting and shading, aside from studying real life lighting situations, was to play with POVRay, a photorealistic raytracer.
I learnt antialiasing in a similar way, partly from others' AA and partly from looking at how, say, GIMP renders a vector shape.
You need to be sure that it renders it gamma-correct in that case, or you will get a distorted idea of antialiasing, as I did.
You may be better off using a 3d renderer (eg blender, using a flat 'no-lighting' material) to model your antialiasing off of, since it's guaranteed to use correct gamma..

Helm has covered pretty much everything else.
If you insist on being pessimistic about your own abilities, consider also being pessimistic about the accuracy of that pessimistic judgement.

Offline happymonster

  • 0010
  • *
  • Posts: 455
  • Karma: +0/-0
    • View Profile

Re: Anti-aliasing explained

Reply #3 on: January 24, 2009, 10:57:38 am
That's one thing I really hate with low-resolution anti-aliased fonts in operating systems / programs. They add a grey horizontal or vertical line next to a full 100% colour one, which makes it look too blurred to me.

Maybe I'm just sensitive to it, ClearType looks a bit off to me as well..  :-\

Offline Ai

  • 0100
  • ***
  • Posts: 1057
  • Karma: +2/-0
  • finti
    • http://pixeljoint.com/pixels/profile.asp?id=1996
    • finticemo
    • View Profile

Re: Anti-aliasing explained

Reply #4 on: January 24, 2009, 12:51:50 pm
That's one thing I really hate with low-resolution anti-aliased fonts in operating systems / programs. They add a grey horizontal or vertical line next to a full 100% colour one, which makes it look too blurred to me.

Maybe I'm just sensitive to it, ClearType looks a bit off to me as well..  :-\
All font antialiasing looks crap to me, personally. I know in Linux you can disable font antialiasing (as I have) by creating a personal fonts.conf file with some appropriate rules, and IMO it is infinitely more readable on LCD displays. Have you investigated doing the same with whatever OS you're talking about?
If you insist on being pessimistic about your own abilities, consider also being pessimistic about the accuracy of that pessimistic judgement.

Offline happymonster

  • 0010
  • *
  • Posts: 455
  • Karma: +0/-0
    • View Profile

Re: Anti-aliasing explained

Reply #5 on: January 25, 2009, 03:05:39 pm
I don't have anti-aliasing on in Windows by default anyway. Hand-anti-aliased pixel fonts look fine to me though.

Cleartype uses the RGB allignment in LCD displays to provide smoother anti-aliasing, at the expense of chroma fringing. See Wikipedia for more details. It still irritates me though on some level..

Offline ilkke

  • 0010
  • *
  • Posts: 233
  • Karma: +0/-0
  • pix off
    • iLkKke
    • https://pixeljoint.com/p/9270.htm
    • View Profile
    • portfolio

Re: Anti-aliasing explained

Reply #6 on: January 31, 2009, 01:15:25 pm
As a general rule try to use as few pixels of AA as possible. The best way to practice is to make, say, and 8 color white to black spread and practice AAing white shapes on black and vice versa. Try organic shapes, circles and ellipses and straight edges at various degrees. Start with organic then move on to the more precise shapes.

There is one method that is purely mechanical that I use when making AA but it's very hard to explain. I'll try to make a picture to illustrate it, but it boils down to observing distribution of color in pixels diagonal to each other. It's pretty easy to start AAing, but it gets hard to see your errors and to see how to fix them. This is a method to speed up that process.

Here's an example:

Not an awesomely AAed thing, in fact it's a rushed piece of crap, but it will work for this purpose.
Now lookie here:

Pay attention to diagonal strips of pixels between markers of same color. (You will have to zoom of course)
Notice how they are pretty regular and symmetrical gradients, even if the symmetry is often offset. Please note that black and white are also important when looking at the strip, as marked in the lower right corner.
When you want to rapidly AA while zoomed you can just observe the diagonals and see if a gradient is regular or if a pixel or two stands out. Of course this is not enough in itself or otherwise it would be possible to make post-processing antialias filter. In the end you always have to pay attention to your shape and how it looks on a sub-pixel level. This has made AA much easier for me and I was amused to find out that Jamon came up with the very same thing, so I guess someone else may find it useful as well.

i

Offline vierbit

  • 0010
  • *
  • Posts: 158
  • Karma: +0/-0
  • 0.09 posts per day
    • View Profile

Re: Anti-aliasing explained

Reply #7 on: January 31, 2009, 02:48:49 pm
I personally think stuff like AA comes with practice and the experience you get out of it.
And AA is probably a bit hard to get with just a tutorial, even if ilkke explained it quit good.

If you have good shapes, you also have spend less time in AA.

Offline ptoing

  • 0101
  • ****
  • Posts: 3063
  • Karma: +0/-0
  • variegated quadrangle arranger
    • the_ptoing
    • http://pixeljoint.com/p/2191.htm
    • View Profile
    • Perpetually inactive website

Re: Anti-aliasing explained

Reply #8 on: January 31, 2009, 03:11:58 pm
I dunno about that, vierbit. At least for me it clicked one day. AA really is just math, if you understand the principle behind it, it's easy (I think) It's just approximation, tho stuff like aaing with limited colours and having good judgement where to aa a bit more and where a bit less comes with practise.
There are no ugly colours, only ugly combinations of colours.

Offline Shrike

  • 0010
  • *
  • Posts: 412
  • Karma: +1/-0
    • View Profile

Re: Anti-aliasing explained

Reply #9 on: January 31, 2009, 03:58:23 pm
I agree with both Vierbit and Ptoing,  tutorials don't give you the right idea, at least for me.  I just had to experiment with it until I got it, and it did click one day.  I like the explanation of it being just math, because that's how I look at it too.  Only real trick I have is not to AA 90degree lines, as in

 .
    .
       .
          .
              .    :D

Good question though.