AuthorTopic: Anti-aliasing explained  (Read 10087 times)

Offline QuickSilva

  • 0010
  • *
  • Posts: 125
  • Karma: +0/-0
    • View Profile

Anti-aliasing explained

on: January 24, 2009, 08:17:54 am
One thing that has always troubled me with pixel art is achieving decent looking anti-aliasing. What I`m hoping for is a proven method that will always get me good results as some people seem to do it well and others not.  Is there a methodical approach that works or do you guys just apply it by eye?

Does anyone have any good tips that they have picked up over the years? For example do you anti-alias only around your original lines or do you replace some of them with anti-aliasing? Do you use only one colour or several to smooth out your lines? At which point does it go from looking good to becoming a blurry mess?

I`ve never really found a good tutorial for this so any help would be much appreciated, by lots of people I expect. I`m not after a quick fix but want a solid foundation to build on after many failed attempts myself.

Like I have said, a proven technique that always works would be good but I`m under no illusions as to it being quick and easy to do, good work of any type requires effort, time and practice.

Thanks for any help,
Jason.

Offline Helm

  • Moderator
  • 0110
  • *
  • Posts: 5159
  • Karma: +0/-0
    • View Profile
    • Asides-Bsides

Re: Anti-aliasing explained

Reply #1 on: January 24, 2009, 08:47:50 am
I'd say half is replacing, half is adding for me at least.

It becomes a blurry mess when the antialias starts banding around the line.

Sometimes no-aa is the way to go on specific edges, the sharpest you want it to look, the less aa you should apply.

The real issue is banding and how to deal with that, AA is more a matter of taste and application of an effect once you realize how the computer monitor works, what pixel rows do and what banding does.

Offline Ai

  • 0100
  • ***
  • Posts: 1057
  • Karma: +2/-0
  • finti
    • http://pixeljoint.com/pixels/profile.asp?id=1996
    • finticemo
    • View Profile

Re: Anti-aliasing explained

Reply #2 on: January 24, 2009, 09:53:41 am
see my tutorial at:
http://www.pixeljoint.com/forum/forum_posts.asp?TID=706&PID=74417#74417
and the few posts preceding and following it.

It provides a sound basic method, along with the underlying maths. This method, unmodified, antialiases evenly inside and outside the shape, to maintain the apparent size of the object.  Of course depending on which part is 'over' the other part, you may find it more effective to antialias mostly outwards or mostly inwards. The amount of colors -- pretty much whatever you can afford; sometimes you will want to use existing colors for to unify things more, other times adding.
There are various tricks to avoid banding and discontinuity in antialiasing, which I've covered here:
http://neota.castleparadox.com/aa_tutorial.html

How I learnt lighting and shading, aside from studying real life lighting situations, was to play with POVRay, a photorealistic raytracer.
I learnt antialiasing in a similar way, partly from others' AA and partly from looking at how, say, GIMP renders a vector shape.
You need to be sure that it renders it gamma-correct in that case, or you will get a distorted idea of antialiasing, as I did.
You may be better off using a 3d renderer (eg blender, using a flat 'no-lighting' material) to model your antialiasing off of, since it's guaranteed to use correct gamma..

Helm has covered pretty much everything else.
If you insist on being pessimistic about your own abilities, consider also being pessimistic about the accuracy of that pessimistic judgement.

Offline happymonster

  • 0010
  • *
  • Posts: 455
  • Karma: +0/-0
    • View Profile

Re: Anti-aliasing explained

Reply #3 on: January 24, 2009, 10:57:38 am
That's one thing I really hate with low-resolution anti-aliased fonts in operating systems / programs. They add a grey horizontal or vertical line next to a full 100% colour one, which makes it look too blurred to me.

Maybe I'm just sensitive to it, ClearType looks a bit off to me as well..  :-\

Offline Ai

  • 0100
  • ***
  • Posts: 1057
  • Karma: +2/-0
  • finti
    • http://pixeljoint.com/pixels/profile.asp?id=1996
    • finticemo
    • View Profile

Re: Anti-aliasing explained

Reply #4 on: January 24, 2009, 12:51:50 pm
That's one thing I really hate with low-resolution anti-aliased fonts in operating systems / programs. They add a grey horizontal or vertical line next to a full 100% colour one, which makes it look too blurred to me.

Maybe I'm just sensitive to it, ClearType looks a bit off to me as well..  :-\
All font antialiasing looks crap to me, personally. I know in Linux you can disable font antialiasing (as I have) by creating a personal fonts.conf file with some appropriate rules, and IMO it is infinitely more readable on LCD displays. Have you investigated doing the same with whatever OS you're talking about?
If you insist on being pessimistic about your own abilities, consider also being pessimistic about the accuracy of that pessimistic judgement.

Offline happymonster

  • 0010
  • *
  • Posts: 455
  • Karma: +0/-0
    • View Profile

Re: Anti-aliasing explained

Reply #5 on: January 25, 2009, 03:05:39 pm
I don't have anti-aliasing on in Windows by default anyway. Hand-anti-aliased pixel fonts look fine to me though.

Cleartype uses the RGB allignment in LCD displays to provide smoother anti-aliasing, at the expense of chroma fringing. See Wikipedia for more details. It still irritates me though on some level..

Offline ilkke

  • 0010
  • *
  • Posts: 233
  • Karma: +0/-0
  • pix off
    • iLkKke
    • https://pixeljoint.com/p/9270.htm
    • View Profile
    • portfolio

Re: Anti-aliasing explained

Reply #6 on: January 31, 2009, 01:15:25 pm
As a general rule try to use as few pixels of AA as possible. The best way to practice is to make, say, and 8 color white to black spread and practice AAing white shapes on black and vice versa. Try organic shapes, circles and ellipses and straight edges at various degrees. Start with organic then move on to the more precise shapes.

There is one method that is purely mechanical that I use when making AA but it's very hard to explain. I'll try to make a picture to illustrate it, but it boils down to observing distribution of color in pixels diagonal to each other. It's pretty easy to start AAing, but it gets hard to see your errors and to see how to fix them. This is a method to speed up that process.

Here's an example:

Not an awesomely AAed thing, in fact it's a rushed piece of crap, but it will work for this purpose.
Now lookie here:

Pay attention to diagonal strips of pixels between markers of same color. (You will have to zoom of course)
Notice how they are pretty regular and symmetrical gradients, even if the symmetry is often offset. Please note that black and white are also important when looking at the strip, as marked in the lower right corner.
When you want to rapidly AA while zoomed you can just observe the diagonals and see if a gradient is regular or if a pixel or two stands out. Of course this is not enough in itself or otherwise it would be possible to make post-processing antialias filter. In the end you always have to pay attention to your shape and how it looks on a sub-pixel level. This has made AA much easier for me and I was amused to find out that Jamon came up with the very same thing, so I guess someone else may find it useful as well.

i

Offline vierbit

  • 0010
  • *
  • Posts: 158
  • Karma: +0/-0
  • 0.09 posts per day
    • View Profile

Re: Anti-aliasing explained

Reply #7 on: January 31, 2009, 02:48:49 pm
I personally think stuff like AA comes with practice and the experience you get out of it.
And AA is probably a bit hard to get with just a tutorial, even if ilkke explained it quit good.

If you have good shapes, you also have spend less time in AA.

Offline ptoing

  • 0101
  • ****
  • Posts: 3063
  • Karma: +0/-0
  • variegated quadrangle arranger
    • the_ptoing
    • http://pixeljoint.com/p/2191.htm
    • View Profile
    • Perpetually inactive website

Re: Anti-aliasing explained

Reply #8 on: January 31, 2009, 03:11:58 pm
I dunno about that, vierbit. At least for me it clicked one day. AA really is just math, if you understand the principle behind it, it's easy (I think) It's just approximation, tho stuff like aaing with limited colours and having good judgement where to aa a bit more and where a bit less comes with practise.
There are no ugly colours, only ugly combinations of colours.

Offline Shrike

  • 0010
  • *
  • Posts: 412
  • Karma: +1/-0
    • View Profile

Re: Anti-aliasing explained

Reply #9 on: January 31, 2009, 03:58:23 pm
I agree with both Vierbit and Ptoing,  tutorials don't give you the right idea, at least for me.  I just had to experiment with it until I got it, and it did click one day.  I like the explanation of it being just math, because that's how I look at it too.  Only real trick I have is not to AA 90degree lines, as in

 .
    .
       .
          .
              .    :D

Good question though.

Offline QuickSilva

  • 0010
  • *
  • Posts: 125
  • Karma: +0/-0
    • View Profile

Re: Anti-aliasing explained

Reply #10 on: January 31, 2009, 04:46:42 pm
Thanks for the replies, sorry I`m a bit late to respond but reading this has helped to clear a few things up. I`m slowly getting there I think. It`s also interesting to see some artists work that still looks great yet hardly uses any AA, Kenneth Fejer springs to mind as does Konjak. I actually really like some none AA work as it retains a sharp clean look. Some people seem to use it well though and maintain that clean look. It`s a good skill to know and one well worth mastering.

Jason.
« Last Edit: February 01, 2009, 12:21:21 pm by QuickSilva »

Offline ptoing

  • 0101
  • ****
  • Posts: 3063
  • Karma: +0/-0
  • variegated quadrangle arranger
    • the_ptoing
    • http://pixeljoint.com/p/2191.htm
    • View Profile
    • Perpetually inactive website

Re: Anti-aliasing explained

Reply #11 on: January 31, 2009, 06:41:28 pm
Speaking of no AA stuff. Metal Slug has hardly any AA. Nor has Dodonpachi (well at least not in the sprites much). Many games have not really much AA and still look good. It's not like if you have shit art and you AA it, it will get better. If you know how to draw you should be able to make stuff good without AA.
There are no ugly colours, only ugly combinations of colours.

Offline .TakaM

  • 0100
  • ***
  • Posts: 1178
  • Karma: +1/-0
    • View Profile
    • Fetch Quest

Re: Anti-aliasing explained

Reply #12 on: January 31, 2009, 10:19:11 pm
Only real trick I have is not to AA 90degree lines, as in

 .
    .
       .
          .
              .    :D
Assuming you meant 45 lines, as in

I've always disagreed with not AAing these lines, at least under all circumstances.
As most of us will agree, AAing is mostly math, and using simple math it's easy to see that a 45 line is not actually "1 pixel thick":

If you were to move the edges of the diagonal line to fill in the gaps- it becomes much clearer

You can see that a diagonal line only occupies 60% the width of a straight line. (as in working with the physical orientation of pixels)
This is why over long stretches of a 45 line I think it's appropriate to AA, preferably 40% of the colour of the line itself.
However, it's not really necessary over short lines, or when lines are several pixels thick
Life without knowledge is death in disguise

Offline Indigo

  • Administrator
  • 0011
  • *
  • Posts: 945
  • Karma: +0/-0
  • Artist, Indie Game Dev
    • DanFessler
    • DanFessler
    • http://pixeljoint.com/p/849.htm
    • DanFessler
    • DanFessler
    • View Profile
    • Portfolio

Re: Anti-aliasing explained

Reply #13 on: February 01, 2009, 12:38:36 am
takaM - I couldn't agree with you more.  In fact I was going to write a quick article type thing about it.  Mathematically 45 degree lines should be AA'd.  The AA is very light, so if your color ramp doesn't support it, it's probably a closer approximation to not AA them at all.  But if your ramp does allow for it, it definitely makes sense.

Offline QuickSilva

  • 0010
  • *
  • Posts: 125
  • Karma: +0/-0
    • View Profile

Re: Anti-aliasing explained

Reply #14 on: February 01, 2009, 12:25:07 pm
I completely agree with ptoing that AA-ing will not save poor artwork it can however lift already good artwork to a higher level if done well. It certainly has its uses.

Jason.

Offline Jad

  • 0100
  • ***
  • Posts: 1048
  • Karma: +0/-0
    • View Profile

Re: Anti-aliasing explained

Reply #15 on: February 02, 2009, 04:31:00 pm
Super quickie:

If you want equal line weight on a single line, AA everything

If you're speaking of the edges of big shapes, AAing diagonal lines can be redundant.
' _ '

Offline Gil

  • 0100
  • ***
  • Posts: 1543
  • Karma: +1/-0
  • Too square to be hip
    • http://pixeljoint.com/p/475.htm
    • View Profile
    • My Portfolio

Re: Anti-aliasing explained

Reply #16 on: February 02, 2009, 09:25:16 pm
All this theory... I just push pixels 'til it looks good.

Offline Jad

  • 0100
  • ***
  • Posts: 1048
  • Karma: +0/-0
    • View Profile

Re: Anti-aliasing explained

Reply #17 on: February 03, 2009, 12:58:38 am
Actually I do too.

Then I formulate my ideas about techniques into words O:! And then we discuss it when we feel like it.

Actually I had a two-year long break from pixel art and when I started again I was on a level I could not have imagined myself to be on because of:

traditional practise (me sketching things with pencil; playing with other mediums in high-school, etc)

lurking at pixelation (reading threads like this, which made me do things in a completely different way once I started pushing pixels)

So yes. Theory is fucking awesome goddamnit. !yus!

I guess that because kind of the whole purpose of this forum is discussing and discovering the theory of pixel art your post just seems .. completely off to me. >:
« Last Edit: February 03, 2009, 01:02:56 am by Jad »
' _ '

Offline Gil

  • 0100
  • ***
  • Posts: 1543
  • Karma: +1/-0
  • Too square to be hip
    • http://pixeljoint.com/p/475.htm
    • View Profile
    • My Portfolio

Re: Anti-aliasing explained

Reply #18 on: February 03, 2009, 01:02:50 am
Of course it is, but only when abstracted. You need to use theory so you can guide your thoughts later, not just looking stuff up you need at that point. Art is all about studying and a lot of theory goes into that.

Offline Shrike

  • 0010
  • *
  • Posts: 412
  • Karma: +1/-0
    • View Profile

Re: Anti-aliasing explained

Reply #19 on: February 03, 2009, 08:59:02 pm
Only real trick I have is not to AA 90degree lines, as in

 .
    .
       .
          .
              .    :D
Assuming you meant 45 lines, as in

I've always disagreed with not AAing these lines, at least under all circumstances.
As most of us will agree, AAing is mostly math, and using simple math it's easy to see that a 45 line is not actually "1 pixel thick":

If you were to move the edges of the diagonal line to fill in the gaps- it becomes much clearer

You can see that a diagonal line only occupies 60% the width of a straight line. (as in working with the physical orientation of pixels)
This is why over long stretches of a 45 line I think it's appropriate to AA, preferably 40% of the colour of the line itself.
However, it's not really necessary over short lines, or when lines are several pixels thick

xD
Yeah, I meant 45 degree lines, and my periods were totally inaccurate.  Sorry about that, but still I generally don't AA those, unless for a very gradual curve.  I guess I should try!  Thanks for catching me up, I'm still a newb at heart.

Also, I agree with Jad.  Like always.   ;)
Toodles!
Shrike