Show Posts
21
General Discussion / PixelLOD
« on: December 26, 2010, 04:36:27 am »
Hello.
I am learning OpenGL and as a way to help the concepts stick I am trying to take the boring academic examples and extend them into things that interest me. I am currently learning about mipmaps and as I like to work in 2D with pixel art I was about ready to dismiss them and move on as doing any scaling with pixel art besides nearest neighbor up scaling is usually evil.
Then I started to consider why this is and does it need to be. The following series of posts is the results of that musing. Discussion, critique, and toying with the demos are welcome.
Miplods pixel whats?
The concept of mipmap generation is simple. Take a texture and create smaller textures by powers of two. The methods for doing this programmatically have been pretty well figured out by graphical programming wizards spewing mumbo jumbo like tri-linear bicubic nearest pixel bumping and that is all well and good when you are dealing with high resolution textures.
When it comes to pixel art the importance of a single pixel means that handing this over to a robot, even a startlingly intelligent one, will not give ideal results. It turns out we humans are good for something once in a while and this is one of those times.
The decision that needs to be made, many times, is simple. Take four pixels, a 2x2 block, and turn them into one pixel. If the block of pixels that you are deciding on are solid black pixels then the decision is easy. Replace it with a single black pixel.
If two pixels are black and two are white then a more difficult decision must be made. Do you use solid black, solid white, or do you average and introduce a grey color? The possibility that you will use a different color entirely also exists in scenarios where a limited palette is concerned and using yellow may make the most sense. Unfortunately, or fortunately depending on how you look at it, there is not a clear right answer in these scenarios.
The human part comes in by examining the piece as a whole and looking at what these 4 pixels are part of representing. I have read Helm’s analogy of pixel art with the game Go, and this is particularly apt when it comes to pixel mipmap generation.
Maybe the black pixels are part of text and getting the idea that text is here, even if you can’t read it, is what’s important so solid black is used. Alternatively, maybe those white dots represent specular lighting and white would be preferred while grey would destroy what is being represented. These are simplified examples for demonstrative purposes and in practice the decision is usually much more complex and difficult.
Mipmap Generation Example
For this example we are going to take a look at a boring square shape that uses pillow shading to simplify the decisions that need to be made during generation.
The first block of the square texture is the most difficult of the entire piece. You have 3 black pixels making up the corner of the outline and the specular dot making up the highlight corner. Removing either means changing what is being portrayed.
It turns out there is another possibility. When you run into a situation where there are two or more pixels that are equally important and you cannot rightly dismiss one over the other then you need to start to look outward. In the vicinity of the bottom-right of the block is a vast sea of plain blue pixels. Sacrificing one of those blue pixels to save the specular is an easy trade off.
The impact of this choice should not be taken lightly. You are not eliminating one seemingly unimportant blue pixel, you are eliminating four. This single step can lead to a series of decisions that seems to spiral out of control. In this case all four are blue pixels that can be easily eliminated.
This process continues around the entire border effectively compressing in on the solid blue area. We end up leaving the vital border, highlight, and shadow intact while simplifying the amount of surface represented in blue. Once you have the first scale figured out you can often glean what you learned and apply it down as you make the rest. This is not always the case.
Another way you can look at this process is taking whatever it is that is being represented and working under more and more restricted resolutions until you get to a single dot. The more I have been working under these conditions the more I am starting to find it to be the exact opposite of where I started this experiment. The exercise of taking a piece and simplifying it until the decision comes down to a single pixel, deciding what single color best represents this thing, is a beautiful, challenging process.
The Demo
The demo is available here.
Two instances of the blue square texture are loaded. Texture 0 [T0] creates a generated mipmap where the computer does a decent, albeit flawed, job. Texture 1 [T1] uses the handmade individual textures that are in the folder.
When you launch the demo by default it is using T0, the computer generated one, with nearest neighbor filtering. If you zoom in and out (up/down arrow. Shift for finer control.) you can see the bang up job that is done. If you carefully observe you will notice that as it gets further away it does a cycle of dropping the highlight/shadow, outline, and then both. If you switch to T1 (left/right arrow) you will see that they look identical at all z levels. This is because by default the mipmaps are not being used and essentially what you are looking at is identical.
Flip on and off the mipmaps (F1/F2) and at this point the difference when zooming should be noticeable. Toggle between the settings at various z levels to see how they look. T0 does a much better job over nearest neighbor until you get a good distance from the camera where it turns into a smudgy blue block. With T1 and mipmaps enabled at nearly every z depth you can see the solid black outline, specular, highlight and shading. Some levels are particularly troublesome where the outline may be too thick or the highlight is nearly invisible, but the first step in not losing visual data is accomplished.
To solve these problems a couple techniques can be used and we will explore them in the next post. Hopefully this has given some food for thought and a neat demo to play around with. Feel free to try making your own mipmaps by replacing the textures that are in the folder. They need to be saved as uncompressed TGA files, which I was unfortunately not able to locate a way to do in Promotion so had to use another application to convert them. If you get a black box then the most likely problem is the TGA file is using compression.
I am learning OpenGL and as a way to help the concepts stick I am trying to take the boring academic examples and extend them into things that interest me. I am currently learning about mipmaps and as I like to work in 2D with pixel art I was about ready to dismiss them and move on as doing any scaling with pixel art besides nearest neighbor up scaling is usually evil.
Then I started to consider why this is and does it need to be. The following series of posts is the results of that musing. Discussion, critique, and toying with the demos are welcome.
Miplods pixel whats?
The concept of mipmap generation is simple. Take a texture and create smaller textures by powers of two. The methods for doing this programmatically have been pretty well figured out by graphical programming wizards spewing mumbo jumbo like tri-linear bicubic nearest pixel bumping and that is all well and good when you are dealing with high resolution textures.
When it comes to pixel art the importance of a single pixel means that handing this over to a robot, even a startlingly intelligent one, will not give ideal results. It turns out we humans are good for something once in a while and this is one of those times.
The decision that needs to be made, many times, is simple. Take four pixels, a 2x2 block, and turn them into one pixel. If the block of pixels that you are deciding on are solid black pixels then the decision is easy. Replace it with a single black pixel.
If two pixels are black and two are white then a more difficult decision must be made. Do you use solid black, solid white, or do you average and introduce a grey color? The possibility that you will use a different color entirely also exists in scenarios where a limited palette is concerned and using yellow may make the most sense. Unfortunately, or fortunately depending on how you look at it, there is not a clear right answer in these scenarios.
The human part comes in by examining the piece as a whole and looking at what these 4 pixels are part of representing. I have read Helm’s analogy of pixel art with the game Go, and this is particularly apt when it comes to pixel mipmap generation.
Maybe the black pixels are part of text and getting the idea that text is here, even if you can’t read it, is what’s important so solid black is used. Alternatively, maybe those white dots represent specular lighting and white would be preferred while grey would destroy what is being represented. These are simplified examples for demonstrative purposes and in practice the decision is usually much more complex and difficult.
Mipmap Generation Example
For this example we are going to take a look at a boring square shape that uses pillow shading to simplify the decisions that need to be made during generation.
The first block of the square texture is the most difficult of the entire piece. You have 3 black pixels making up the corner of the outline and the specular dot making up the highlight corner. Removing either means changing what is being portrayed.
It turns out there is another possibility. When you run into a situation where there are two or more pixels that are equally important and you cannot rightly dismiss one over the other then you need to start to look outward. In the vicinity of the bottom-right of the block is a vast sea of plain blue pixels. Sacrificing one of those blue pixels to save the specular is an easy trade off.
The impact of this choice should not be taken lightly. You are not eliminating one seemingly unimportant blue pixel, you are eliminating four. This single step can lead to a series of decisions that seems to spiral out of control. In this case all four are blue pixels that can be easily eliminated.
This process continues around the entire border effectively compressing in on the solid blue area. We end up leaving the vital border, highlight, and shadow intact while simplifying the amount of surface represented in blue. Once you have the first scale figured out you can often glean what you learned and apply it down as you make the rest. This is not always the case.
Another way you can look at this process is taking whatever it is that is being represented and working under more and more restricted resolutions until you get to a single dot. The more I have been working under these conditions the more I am starting to find it to be the exact opposite of where I started this experiment. The exercise of taking a piece and simplifying it until the decision comes down to a single pixel, deciding what single color best represents this thing, is a beautiful, challenging process.
The Demo
The demo is available here.
Two instances of the blue square texture are loaded. Texture 0 [T0] creates a generated mipmap where the computer does a decent, albeit flawed, job. Texture 1 [T1] uses the handmade individual textures that are in the folder.
When you launch the demo by default it is using T0, the computer generated one, with nearest neighbor filtering. If you zoom in and out (up/down arrow. Shift for finer control.) you can see the bang up job that is done. If you carefully observe you will notice that as it gets further away it does a cycle of dropping the highlight/shadow, outline, and then both. If you switch to T1 (left/right arrow) you will see that they look identical at all z levels. This is because by default the mipmaps are not being used and essentially what you are looking at is identical.
Flip on and off the mipmaps (F1/F2) and at this point the difference when zooming should be noticeable. Toggle between the settings at various z levels to see how they look. T0 does a much better job over nearest neighbor until you get a good distance from the camera where it turns into a smudgy blue block. With T1 and mipmaps enabled at nearly every z depth you can see the solid black outline, specular, highlight and shading. Some levels are particularly troublesome where the outline may be too thick or the highlight is nearly invisible, but the first step in not losing visual data is accomplished.
To solve these problems a couple techniques can be used and we will explore them in the next post. Hopefully this has given some food for thought and a neat demo to play around with. Feel free to try making your own mipmaps by replacing the textures that are in the folder. They need to be saved as uncompressed TGA files, which I was unfortunately not able to locate a way to do in Promotion so had to use another application to convert them. If you get a black box then the most likely problem is the TGA file is using compression.