AuthorTopic: [WIP] Rescale&Rotate Tool  (Read 19020 times)

Offline yrizoud

  • 0010
  • *
  • Posts: 330
  • Karma: +0/-0
    • View Profile

Re: [WIP] Rescale&Rotate Tool

Reply #20 on: June 06, 2011, 06:51:17 pm
Ok so I've put a similar algorithm as rotsprite (but simplified) in Grafx2's latest WIP build :
http://code.google.com/p/grafx2/downloads/detail?name=grafx2-2.4wip1790-win32.zip
It's... rather disappointing.
 Now I understand better what it's good at: it's good at avoiding some patterns where a long straight edge becomes exactly horizontal or vertical and shows "noise", pixels that are randomly of the color of either side.
Here's an explanation of the phenomenon:

On the upper image, the grid intersections show which pixels will be picked by a standard roto(zoom) algorithm. I've put some "zoom balloons" at the bottom to show that  a noisy pattern will emerge from a line that should be straight : black*1, white*3, black*1, white*2, black*1.

Using scale2x to soften the edges statistically reduces the times when a newly-horizontal line has the grid intersect on the parts where the pixels are "spikey". (I should have highlighted... in the picture above with a 45° rotation, there is a 10/16 risk of noise pattern using standard algorithm, reduced to 3/16 with scale2x 3 times)

But it's the only case where I see an improvement... For example with pure line art (single-pixel lines), it still produces a mess of lines that are randomly thicker or interrupted, and it absolutely can't improve the rotation of dithered surfaces, antialiased lines etc... Any case where the pixels are in small cluster, ie. most of the detailed PA.

So... no miracle there, still requires finely tweaking the rotation angle until you find one exact angle the produces the least number of artifacts. Grafx2 has a problem here, as it computes the final coordinates of the 4 new angles (rounded to integer), so you can't make sub-pixel adjustments... and it's some part of code that I have never understood, so I won't be able to fix it.

Offline lithander

  • 0001
  • *
  • Posts: 43
  • Karma: +0/-0
    • View Profile
    • www.pixelpracht.net

Re: [WIP] Rescale&Rotate Tool

Reply #21 on: June 06, 2011, 09:50:14 pm
yrizoud, that visualization you posted is quite nice. Makes discussing the matter a lot easier! :)

But, first here's one of mine:



As you can see one of the properties of the upscaling algorithm is that they "rounden" the image. Without upscaling diagonals would appear jagged (due to the low resolution) and the artist relies on the eye of the viewer to reconstruct the image. With the right upscaling algorithm the "real" shape of the depicted thing is approximated. And this improves the quality when rotating, transforming and rescaling the image. A chunk of pixels that is meant to have a certain shape is more likely to end up in that shape after the transformation!

My progress is a lot slower then I hoped for (due to a lot of distractions, not because I don't believe in it's doability) but in my most recent build you can freely transform (left mouse button) and rotate (right mouse button) the image allready, and I think that it allready domonstrates that using an upsaled source image does improve the final quality.

PixelPunch003.rar



It's far from perfect but there's still one step in the transformation process that I didn't adress yet. Sampling. Your illustration and explaination show perfectly what causes the artefacts. Now let's say for every green square you count the number of white and black sub-pixels to make a decision. That would get rid of most of the artefacts except where there are diagonals in the source image that introduce jaggedness because of the low resolution in which case (hopefully) the upscaling will kick in.

P.S.: None of the artworks shown in the screenshots are done by me. But they all have been posted on the forums so I hope it's okay for me to use them in the screenshots.
« Last Edit: June 06, 2011, 09:59:28 pm by lithander »

Offline yrizoud

  • 0010
  • *
  • Posts: 330
  • Karma: +0/-0
    • View Profile

Re: [WIP] Rescale&Rotate Tool

Reply #22 on: June 06, 2011, 11:15:02 pm
> Now let's say for every green square you count the number of white and black sub-pixels to make a decision
This would introduce a bias 'majoritarian color wins'. Maybe it can be countered by choosing a pre-filter that favors (bloats) the smaller pixel clusters.. Like in your screenshot, where a single pixel doubled becomes 5 pixels instead of the theorical 4.

Offline questseeker

  • 0010
  • *
  • Posts: 112
  • Karma: +0/-0
    • View Profile

Re: [WIP] Rescale&Rotate Tool

Reply #23 on: June 08, 2011, 08:06:03 am
An idea I'm considering for when I have enough time to write a Gimp plugin is automatically sampling different subpixel offsets and different angles and picking the most common output out of hundreds of samples as the result (the second most common result for each pixel could be put in a separate "second opinion" layer as a reference for handmade corrections).
Hopefully jagged edges would be smoothed out, but maybe to the point of annihilating dithering and antialiasing pixels and altering line thickness.

Offline lithander

  • 0001
  • *
  • Posts: 43
  • Karma: +0/-0
    • View Profile
    • www.pixelpracht.net

Re: [WIP] Rescale&Rotate Tool

Reply #24 on: June 10, 2011, 01:31:15 am
@questseeker: given that for every pixel in the destination only 4 pixels in the source are close enough to be considered (because when you are not scaling, only rotating all others are more then a pixel away) you shouldn't need hundreds of subsamples to evaluate which one fits best. Whatever result you could achieve with a lot of subsamples you could achieve with sampling only the 4 pixels and some calculations based on their exact position (the fractional part of the coordinate matters now). But I fear that finding the best match out of the 4 possible values is not possible with only considering those 4 pixels. Imagine a single white pixel on a patch of black. In some cases only a small part of the white pixel (roughly a quarter) is overlapping with four destination pixels and when you decide (for each destination pixel individually) wether to write white or black all will become black in all solutions I can come up with right now. :/ How would your plugin solve that?

Upscaling the source should help a little (that's why it's part of my test app) but it's not perfect...

Offline questseeker

  • 0010
  • *
  • Posts: 112
  • Karma: +0/-0
    • View Profile

Re: [WIP] Rescale&Rotate Tool

Reply #25 on: June 10, 2011, 08:49:05 am
@lithander:
  • I'm considering combined upscaling and rotation, like Rotsprite.
  • Upscaling (with arbitrary fancy algorithms) then casting hundreds of "votes" is an elegant and general brute force approach to perform what you gloss over as "some calculations"; more source pixels than the 4 nearest neighbours of the output pixel might have some influence. The appropriate amount of gratuitous upscaling, the pattern of sample positions, and how many samples are enough, are of course three open questions.
  • In the isolated white pixel example, it's indeed likely to be obliterated because all output pixels have less than half white in their region of interest, but I don't see it as much of a problem: hand retouching is expected, and by choosing the center of rotation (and thus the alignment between the original and rotated grid) the user has significant control over what "falls through the cracks".
    An upscaling algorithm that inflates isolated pixels at the expense of background ones would distort other features and make clusters of 2 or more white pixels possible, which might or might not be a good tradeoff depending on what is being rescaled.
If preventing dropouts is important, 1:1 scale rotation can be implemented by two successive skew operations: if you shift each column and row a whole number of pixels you get exactly the same pixels that were in the source (plus transparent padding on the sides). This jaggy image could be used as a reference for postprocessing (what source pixels disappeared? Where do I want to restore them?) rather than as an output.

Offline lithander

  • 0001
  • *
  • Posts: 43
  • Karma: +0/-0
    • View Profile
    • www.pixelpracht.net

Re: [WIP] Rescale&Rotate Tool

Reply #26 on: June 10, 2011, 04:56:08 pm
@questseeker:
Interesting ideas! I didn't know that you can achieve a rotation by just shifting rows and columns. That sounds incredible efficient! Do you happen to know any piece of software that implements it so I could see how it looks? Wonder how it compares to nearest neighbour sampling... same goes for your "hundreds of votes" approach: I still think that I could achieve the same effect cheaper but if you'd have an example where this technique is used we could just compare the results?

Also, while I don't expect perfect results I hope to end up with a set of algorithms that produce acceptable results where hand retouching is not necessary. In the scenario I want to use it that would just not be possible. An individual pixel that flickers between visibility and invisibilty depending on the current rotation would be a pretty huge flaw.
But your idea of using a reference to compare against might help here. How about also interpolating the image with a standard approach (that actually mixes close colors together) then, by comparing the local neighbourhood in that smooth-filtered image with the result where we don't mix numbers and based on the amount of difference (error) we reconsider our choice of pixels to reduce the summed local difference in comparision with the reference. Hmmm...

Offline lithander

  • 0001
  • *
  • Posts: 43
  • Karma: +0/-0
    • View Profile
    • www.pixelpracht.net

Re: [WIP] Rescale&Rotate Tool

Reply #27 on: June 16, 2011, 07:55:01 pm
I've updated the first post with a new version of the tool. I added a rather detailed explaination of what the different options do. Also some usage guidelines. So please give it a try and tell me what combination yields the best possible results in your opinion. I find it very hard to judge that. I think that some combinations (Scale3xHQ -> Bilinear Transformation -> Best Fit Wide (or on some images Any)) result in transformed images that preserve the original pretty well but none are without loss. Makes me anxious if this will suffice for the big plan: animated items/characters based on static elements in a source texture.

What do you think? At the moment I don't have any new features planned but if you got ideas let me hear! :)

Offline lithander

  • 0001
  • *
  • Posts: 43
  • Karma: +0/-0
    • View Profile
    • www.pixelpracht.net

Re: [WIP] Rescale&Rotate Tool

Reply #28 on: July 09, 2011, 12:30:46 pm
Found an interesting paper about vectorization of pixel art. Could be useful as an intermediate format for rotation & scaling, I guess.
http://research.microsoft.com/en-us/um/people/kopf/pixelart/paper/pixel.pdf

Offline ui

  • 0001
  • *
  • Posts: 59
  • Karma: +0/-0
  • Venezuela
    • View Profile
    • http://www.iloveui.com

Re: [WIP] Rescale&Rotate Tool

Reply #29 on: August 05, 2011, 11:01:03 pm
zooommg....so you did it!! JAJAJA... amazing man! :DDD i will test this like NOW!!! :DDD let us know about any update and thx for your work