The Pixel Aesthetic

[AlexHW]

Thanks!
I've added the new mode to the online visualizer for anyone that wants to mess with it.
Choose mode1 from the select menu, and then choose an image.
It might not work in chrome (I think chrome has some default settings that conflict with the shader, or maybe it's my graphics card), but works fine for me in firefox.

[0xDB]

I also took note of spots where the directions appear to cross (represented by the dots in the image). These occur when two linked connections compete with each other. I want to add this detail into the document (linked pixels that cross), since it is something I think is important.

Does not the importance vary depending on values and formations of the involved pixels? e.g. it appears to me in the case of the eyebrow/bulge in the bird, the lower value cluster is meant to represent a continuous line which is fully surrounded by a big higher value cluster which itself at the crossing which gets "pointed" out does not produce any perceptual line direction there, does not "cross" the line-clusters direction perceptually.

In any case, the new visualization already appears to be infinitely more useful.

I suggest blending the visualization directly with the scaled source like this:
(the layer Blending mode used here is called "Allanon" in Krita)

[Ai]

Thanks for the feedback, Ai.
The banding occurs with thin lines rather than lines that are of different thickness.

How so? My 'bad example' was done by imagining myself as a naive pixeler: 'Okay, 1 px line is not thick enough, 2px is too thick..I'll just use 3px bands but overlap them a little'. (ie. an attempt at creating 1.3-1.5px width without any AA available). The result being similar to fig.23, which I consider a thickened line.
But it also applies to faces of any non-huge size AFAICS.

This is due to the looser connections of linked pixels. So adding weight would be one way to prevent it, or by using less linked pixels.

Thanks, this helps me think about linking and binding more..
To me, it looks like the 'e' pixels are binding vertically, taking priority over the diagonal linking. Maybe that's what you mean above. The linking/binding principle doesn't appear to apply directly to larger shapes such as

Code: [Select]

Key:
 e: non-axisbound edge pixel
 E: axisbound edge pixel
 #: shape impression created by axisbound edge pixels.

Bad: (4 edge pixels with 2-axis alignment)

..E#E.
e*###e
.e###e
..E#E.

Better: (2 edge pixels with 2-axis alignment)

..e*E.
e***#e
.e**#e
...eE.

Better, different solution: (2 edge pixels with 2-axis alignment)

..e**E.
e****#e
.e***E
...ee.


However, in my experience, the 2-axis-aligned pixels do visually 'bind' across any amount of space found in typical pixel image sizes, creating impressions of rectangles or lines. This is not necessarily "bad" (it's one of the major factors involved in an impression of symmetry imo), but is generally avoided in any strong flow-focused design. (Loomis calls these arrangements 'informal' [flow-focused] and 'formal' [uniformity-focused] respectively).

So thats what I was trying to get at -- how alignments of opposite vertices bind through the shape, not only with immediate neighbours, and this effect is especially hard to avoid because of pixel art's low resolutions.

Like the new visualization, it's very clear. I agree with 0xDB on the value of blending it with the original. Being able to cycle through display modes 'original -> blended -> diagram-only' would probably be instructive.

[AlexHW]

Does not the importance vary depending on values and formations of the involved pixels?

Yeah, you're right. There is some sort of influential factor that makes it appear whether one way or another has priority. I'm not sure what that is. It could be due to a developed understanding of patterns that we acquire in relation to complex formations. Because from the standpoint of single pixels, it is difficult to determine the priority without a greater understanding of what is being conveyed.
From an algorithmic standpoint it makes sense to treat these occurrences as a special case. I'd love to find a way to treat them more accurately to how we would expect them to be, but I haven't found a way yet, if there is a way.
And yea, I want to work on including a way to overlay the visualizations onto the original image.

How so? My 'bad example' was done by imagining myself as a naive pixeler: 'Okay, 1 px line is not thick enough, 2px is too thick..I'll just use 3px bands but overlap them a little'. (ie. an attempt at creating 1.3-1.5px width without any AA available). The result being similar to fig.23, which I consider a thickened line.
But it also applies to faces of any non-huge size AFAICS.

Oh! I misunderstood your diagrams before. Now I understand what you were saying.
hm.. Yeah, I could see how the patterns from one side of a face could emphasize similar patterns across from it.

I had thought banding was the case of thin lines, and the looser linked pixels, but I can see now that the same issue with loose connections are present in thicker lines:

If you look at the blue box, there are only 3 connections (two linked connections and one bound connection), whereas the red boxes have 4 connections (the connections are tighter bound).

I'll have to make sure to include this sort of stuff in the document regarding banding, thanks for pointing that out!

[Ai]

I was also referring to single-color shapes that by definition have "no" banding (other than with the background, perhaps)

But I just wanted to say, the left diagram illustrates really clearly why banding looks weird on a purely technical level -- it creates a uniform pattern that clearly isn't a straight line, and that non-straightness is reinforced further with each level of repetition. This should also indicate why you can't just add levels of dithering to unband something -- the underlying line shape is still being reinforced. You have to use a pattern that disguises the underlying jagged line shape.

Interesting that you hadn't spotted that larger bands was also banding. To me, your example #3 represents one of the most apparent and prevalent forms of banding, though it is somewhat easier to look at than banding with thinner layers.

[AlexHW]

I'm thinking that banding occurs in other situations that aren't as rectangular. It looks like the banding is related to the sub-ordering of pixels. Where the diagonal orders connect with the horizontal/vertical orders.


The red and blue lines illustrate where you might see the illusion of a line. There are more than just the red/blue lines, but all sorts of angles connecting where the orders change, but I think where the changes of order are closer in proximity with each other are where they are more easily seen.

[Ai]

Interesting. I mostly agree. I think for your cases #2 #3 #4, I would rather think of the loosely-banding points as 'vertices where segment length changes'. But I agree case #1 (and any other cases dealing with multiple non-1px thicknesses) is best described by "Where the diagonal orders connect with the horizontal/vertical orders.".

[Helm]

I'm still digesting all of this but it seems to me like a valuable contribution, thank you for your hard work, Alex!

[0xDB]

Does not the importance vary depending on values and formations of the involved pixels?

Yeah, you're right. There is some sort of influential factor that makes it appear whether one way or another has priority. I'm not sure what that is. It could be due to a developed understanding of patterns that we acquire in relation to complex formations. Because from the standpoint of single pixels, it is difficult to determine the priority without a greater understanding of what is being conveyed.
From an algorithmic standpoint it makes sense to treat these occurrences as a special case. I'd love to find a way to treat them more accurately to how we would expect them to be, but I haven't found a way yet, if there is a way.
And yea, I want to work on including a way to overlay the visualizations onto the original image.

Very true. Any visualization algorithm only "sees" the final result, the artists rendition of something. A pixel-art analyzation algorithm does not "see" the thought process, does not "grasp" the underlying construction and intent or any larger structures, because that only exists implicitly in the mind of the artist and in the mind of the viewers who interprete what they see based on experience and their individual lifetimes worth of the appearances of things that they internalized in their minds.

An algorithm which only looks at the pixels that are there does not have that experience or library of things to draw from when it is supposed to judge the quality of the "aesthetics".

It is a similar problem that Pixel-Art upscaling algorithms are facing, despite there being some highly sophisticated ones which try to find bigger structures, longer strokes and attempt to figure out the intent of long curves and lines and bigger areas, they are still missing all the information the artists had access to (especially Volumes) when they created the piece and they are also still missing the "experience" a human viewer has, so they can not interprete what they "see", can not reconstruct and judge the process that lead to the result, can not derive Volumes from the result.

But the process or intent, as you say, "a greater understanding of what is being conveyed", is not present in the pixels themselves, so yes, it will be difficult to make a visualization algorithm which can accurately "point out potentional problems in the aesthetics"(which I am assuming is the goal of this study and visualization)...

...but, here is a thought which may or may not lead to a more accurate judgement whether two diagonal connections do form an ambiguous X direction in our perception. I'm pretty sure it has something to do with Contrast between the Pixels looked at and with the possible "continuations" of the "things" which we want to check for whether they create an X:

(link if image is not inlined)

[AlexHW]

But the process or intent, as you say, "a greater understanding of what is being conveyed", is not present in the pixels themselves, so yes, it will be difficult to make a visualization algorithm which can accurately "point out potentional problems in the aesthetics"(which I am assuming is the goal of this study and visualization)...

I prefer thinking about the aesthetics as simply a collection of relationships regarding visual stimuli. There isn't anything wrong or correct with how any relationship acts. The visualization algorithm stuff is a way to try and extract the relationships at a more simplistic/raw level. It can help reveal where ambiguity in the relationships develop if certain conditions are met, etc, which provide areas where the relationships can be explored more. The case where linked pixels cross is a good example, as it isn't necessarily a thing a person would immediately recognize without understanding the linked/bound relationship- because they'd instead already have a higher level understanding of the image as a whole.
 

...but, here is a thought which may or may not lead to a more accurate judgement whether two diagonal connections do form an ambiguous X direction in our perception. I'm pretty sure it has something to do with Contrast between the Pixels looked at and with the possible "continuations" of the "things" which we want to check for whether they create an X:

That's an interesting example. I think you're right, in that contrast plays a part in determining the continuity. Also, I think gradations act as a binder, and also have influence regarding continuity.

Here's another visual experiment I've been working on. It involves relationships between how many connections there are, similar to your idea, 0xDB.. The blue shows darker where there are more connections. The lightest blue correspond to continuous straight single line connections. The green correspond to single unconnected pixels, which are the ambiguous areas in this case, and require more exploration.