Pixelation
General => General Discussion => Topic started by: AlexHW on March 09, 2016, 02:03:12 am
-
The Pixel Aesthetic
(http://www.alexhw.com/pixel_files/image002.png)
(http://www.alexhw.com/pixel_files/image022.png)
(http://alexhw.com/pixelAesthetic.html)
I took some time to put up some writing I did a while back based on a personal study of the pixel. The intent was to explore pixel aesthetics in a black and white sort of way- making clear the differences between specific aspects related to the pixel and how they might interact with each other.
I'm not sure what others will think, which is why I've put it up. It might help me going forward if I get some feedback on it.
---
Working on some visualizers to help illustrate and provide more context.
Select an image to see the linking/binding of the pixels. Requires webgl enabled in your browser.
http://alexhw.com/pav/index.html (http://alexhw.com/pav/index.html)
(http://i.imgur.com/revzfLi.png)
---
-
Repeating a few thoughts on terminology. Please consider:
If you're going to make gem analogies, Face and Facet feel reversed. Face is a flat shape with sides. Facet is "one side of something many-sided" by definition.
Less important, but Laced does not convey as much meaning as Interlaced could. Interlaced is more descriptive because it implies there is more than one influence on the pixel - being bound and linked.
A lot of thought provoking perspectives, regardless.
-
I'll spend more time thinking about those, Atnas.
There are various reasons why I chose those- I need to make those reasons more clear I suppose.
I've started working on a visualization tool to help generate more context and bring a sense of application to these concepts.
http://alexhw.com/pav/index.html (http://alexhw.com/pav/index.html)
Select an image to see the linking/binding of the pixels. Requires webgl enabled in your browser.
(http://i.imgur.com/f1buipt.png)
It's interesting seeing the different dynamics of various sized images/details, etc.
http://i.imgur.com/dCfv929.png (http://i.imgur.com/dCfv929.png)
-
I am wondering about the practical usefulness of those concepts of "linking/binding". It seems to me the output of the visualization of those things is not recognizable as anything, so why even think about it? Or would you say the lack of the visualization looking like anything is a hint that the arrangement of pixels in the source image is flawed and does not do a good job at making use of the expressive possibilities in the arrangement of pixels / hints towards poor aesthetic choices?
-
Looks to me like some really grainy edge detection filter.
-
It's possible the technique involved with visualizing the linking/binding could be improved, 0xDB. That is something I will be experimenting with. I also plan to add other data and controls to the visualizer.
You could be right in that certain patterns may correspond to certain expressive qualities. Where it looks jumbled may hint towards more focused connections, whereas less jumbled areas hint at less focused connections (or maybe the other way around?). It's open to investigation as to the applications, etc. Perhaps such things will be clearer as the visualizer becomes more developed.
-
Had a thought about banding, and wanted to analyze it a bit.
Left side of image illustrates the issue with banding pretty well.
(http://i.imgur.com/cI3iHIn.png)
-
Just took the time to read through this. Some very interesting thoughts. I suspect that the terminology could be made a bit more approachable, though I'd like to note specifically that the face/facet distinction seems straightforward and uncomplicated to me. I suspect that both you and I have a more technical bent than the average pixeller, though
Especially liked the coverage of Value: this shows a simple, quick technique that was used in demoscene graphics to good effect (altering line color in relation to the line's angle, to maintain correct looking thickness), but doesn't appear to be used much in modern pixel art.
I'm not sure I understand your earlier diagrams very well (ie. I know what they are supposed to depict but they don't 'read'), but it seems to work very well for detecting banding.
Does the potential for a thickened line to 'band' with itself need to be mentioned in Weight section? Here is what I mean:
Good example - each edge pixel(e) aligns with exactly one other edge pixel.
***e
e****e
..e****e
....e****e
Bad examples: each/most edge pixel(s) aligns with 2 other edge pixels
e*e
..e*e
....e*e
.....e*e
.......e*e
-
Thanks for the feedback, Ai.
The way of visualizing the concepts could be improved I think. At the time, it was all I could think of to help illustrate things.
The banding occurs with thin lines rather than lines that are of different thickness. This is due to the looser connections of linked pixels. So adding weight would be one way to prevent it, or by using less linked pixels.
I'm experimenting with another way of visualizing the connections/directions involved with pixels:
(http://i.imgur.com/oELu5ks.png)
With this, it is easier to see the laced pixels- how they are bound/linked, creating a latticed pattern.
The bound and linked pixels also read better (horizontal/vertical lines for the bound connections, and diagonal for the linked connections).
I also took note of spots where the directions appear to cross (represented by the dots in the image). These occur when two linked connections compete with each other. I want to add this detail into the document (linked pixels that cross), since it is something I think is important.
-
this latest representation is superb. definitely suggests your on to something. It detects edge flow really really well.
-
Thanks!
I've added the new mode to the online visualizer for anyone that wants to mess with it.
Choose mode1 from the select menu, and then choose an image.
It might not work in chrome (I think chrome has some default settings that conflict with the shader, or maybe it's my graphics card), but works fine for me in firefox.
-
I also took note of spots where the directions appear to cross (represented by the dots in the image). These occur when two linked connections compete with each other. I want to add this detail into the document (linked pixels that cross), since it is something I think is important.
Does not the importance vary depending on values and formations of the involved pixels? e.g. it appears to me in the case of the eyebrow/bulge in the bird, the lower value cluster is meant to represent a continuous line which is fully surrounded by a big higher value cluster which itself at the crossing which gets "pointed" out does not produce any perceptual line direction there, does not "cross" the line-clusters direction perceptually.
In any case, the new visualization already appears to be infinitely more useful.
I suggest blending the visualization directly with the scaled source like this:
(the layer Blending mode used here is called "Allanon (https://userbase.kde.org/Krita/Manual/Blendingmodes#Allanon)" in Krita)
(http://abload.de/img/pixel_aesthetics01stbv7.png) (http://abload.de/img/pixel_aesthetics021azt6.png)
-
Thanks for the feedback, Ai.
The banding occurs with thin lines rather than lines that are of different thickness.
How so? My 'bad example' was done by imagining myself as a naive pixeler: 'Okay, 1 px line is not thick enough, 2px is too thick..I'll just use 3px bands but overlap them a little'. (ie. an attempt at creating 1.3-1.5px width without any AA available). The result being similar to fig.23, which I consider a thickened line.
But it also applies to faces of any non-huge size AFAICS.
This is due to the looser connections of linked pixels. So adding weight would be one way to prevent it, or by using less linked pixels.
Thanks, this helps me think about linking and binding more..
To me, it looks like the 'e' pixels are binding vertically, taking priority over the diagonal linking. Maybe that's what you mean above. The linking/binding principle doesn't appear to apply directly to larger shapes such as
Key:
e: non-axisbound edge pixel
E: axisbound edge pixel
#: shape impression created by axisbound edge pixels.
Bad: (4 edge pixels with 2-axis alignment)
..E#E.
e*###e
.e###e
..E#E.
Better: (2 edge pixels with 2-axis alignment)
..e*E.
e***#e
.e**#e
...eE.
Better, different solution: (2 edge pixels with 2-axis alignment)
..e**E.
e****#e
.e***E
...ee.
However, in my experience, the 2-axis-aligned pixels do visually 'bind' across any amount of space found in typical pixel image sizes, creating impressions of rectangles or lines. This is not necessarily "bad" (it's one of the major factors involved in an impression of symmetry imo), but is generally avoided in any strong flow-focused design. (Loomis calls these arrangements 'informal' [flow-focused] and 'formal' [uniformity-focused] respectively).
So thats what I was trying to get at -- how alignments of opposite vertices bind through the shape, not only with immediate neighbours, and this effect is especially hard to avoid because of pixel art's low resolutions.
Like the new visualization, it's very clear. I agree with 0xDB on the value of blending it with the original. Being able to cycle through display modes 'original -> blended -> diagram-only' would probably be instructive.
-
Does not the importance vary depending on values and formations of the involved pixels?
Yeah, you're right. There is some sort of influential factor that makes it appear whether one way or another has priority. I'm not sure what that is. It could be due to a developed understanding of patterns that we acquire in relation to complex formations. Because from the standpoint of single pixels, it is difficult to determine the priority without a greater understanding of what is being conveyed.
From an algorithmic standpoint it makes sense to treat these occurrences as a special case. I'd love to find a way to treat them more accurately to how we would expect them to be, but I haven't found a way yet, if there is a way.
And yea, I want to work on including a way to overlay the visualizations onto the original image.
How so? My 'bad example' was done by imagining myself as a naive pixeler: 'Okay, 1 px line is not thick enough, 2px is too thick..I'll just use 3px bands but overlap them a little'. (ie. an attempt at creating 1.3-1.5px width without any AA available). The result being similar to fig.23, which I consider a thickened line.
But it also applies to faces of any non-huge size AFAICS.
Oh! I misunderstood your diagrams before. Now I understand what you were saying.
hm.. Yeah, I could see how the patterns from one side of a face could emphasize similar patterns across from it.
I had thought banding was the case of thin lines, and the looser linked pixels, but I can see now that the same issue with loose connections are present in thicker lines:
(http://i.imgur.com/EXdn1Hm.png)
If you look at the blue box, there are only 3 connections (two linked connections and one bound connection), whereas the red boxes have 4 connections (the connections are tighter bound).
I'll have to make sure to include this sort of stuff in the document regarding banding, thanks for pointing that out!
-
I was also referring to single-color shapes that by definition have "no" banding (other than with the background, perhaps)
But I just wanted to say, the left diagram illustrates really clearly why banding looks weird on a purely technical level -- it creates a uniform pattern that clearly isn't a straight line, and that non-straightness is reinforced further with each level of repetition. This should also indicate why you can't just add levels of dithering to unband something -- the underlying line shape is still being reinforced. You have to use a pattern that disguises the underlying jagged line shape.
Interesting that you hadn't spotted that larger bands was also banding. To me, your example #3 represents one of the most apparent and prevalent forms of banding, though it is somewhat easier to look at than banding with thinner layers.
-
I'm thinking that banding occurs in other situations that aren't as rectangular. It looks like the banding is related to the sub-ordering of pixels. Where the diagonal orders connect with the horizontal/vertical orders.
(http://i.imgur.com/DHvKFpY.png)
(http://i.imgur.com/mjhmY9M.png)
The red and blue lines illustrate where you might see the illusion of a line. There are more than just the red/blue lines, but all sorts of angles connecting where the orders change, but I think where the changes of order are closer in proximity with each other are where they are more easily seen.
-
Interesting. I mostly agree. I think for your cases #2 #3 #4, I would rather think of the loosely-banding points as 'vertices where segment length changes'. But I agree case #1 (and any other cases dealing with multiple non-1px thicknesses) is best described by "Where the diagonal orders connect with the horizontal/vertical orders.".
-
I'm still digesting all of this but it seems to me like a valuable contribution, thank you for your hard work, Alex!
-
Does not the importance vary depending on values and formations of the involved pixels?
Yeah, you're right. There is some sort of influential factor that makes it appear whether one way or another has priority. I'm not sure what that is. It could be due to a developed understanding of patterns that we acquire in relation to complex formations. Because from the standpoint of single pixels, it is difficult to determine the priority without a greater understanding of what is being conveyed.
From an algorithmic standpoint it makes sense to treat these occurrences as a special case. I'd love to find a way to treat them more accurately to how we would expect them to be, but I haven't found a way yet, if there is a way.
And yea, I want to work on including a way to overlay the visualizations onto the original image.
Very true. Any visualization algorithm only "sees" the final result, the artists rendition of something. A pixel-art analyzation algorithm does not "see" the thought process, does not "grasp" the underlying construction and intent or any larger structures, because that only exists implicitly in the mind of the artist and in the mind of the viewers who interprete what they see based on experience and their individual lifetimes worth of the appearances of things that they internalized in their minds.
An algorithm which only looks at the pixels that are there does not have that experience or library of things to draw from when it is supposed to judge the quality of the "aesthetics".
It is a similar problem that Pixel-Art upscaling algorithms are facing, despite there being some highly sophisticated ones which try to find bigger structures, longer strokes and attempt to figure out the intent of long curves and lines and bigger areas, they are still missing all the information the artists had access to (especially Volumes) when they created the piece and they are also still missing the "experience" a human viewer has, so they can not interprete what they "see", can not reconstruct and judge the process that lead to the result, can not derive Volumes from the result.
But the process or intent, as you say, "a greater understanding of what is being conveyed", is not present in the pixels themselves, so yes, it will be difficult to make a visualization algorithm which can accurately "point out potentional problems in the aesthetics"(which I am assuming is the goal of this study and visualization)...
...but, here is a thought which may or may not lead to a more accurate judgement whether two diagonal connections do form an ambiguous X direction in our perception. I'm pretty sure it has something to do with Contrast between the Pixels looked at and with the possible "continuations" of the "things" which we want to check for whether they create an X:
(link if image is not inlined) (http://abload.de/image.php?img=2016_03_15_pixelaesthrgrn2.png)
(http://abload.de/img/2016_03_15_pixelaesthrgrn2.png)
-
But the process or intent, as you say, "a greater understanding of what is being conveyed", is not present in the pixels themselves, so yes, it will be difficult to make a visualization algorithm which can accurately "point out potentional problems in the aesthetics"(which I am assuming is the goal of this study and visualization)...
I prefer thinking about the aesthetics as simply a collection of relationships regarding visual stimuli. There isn't anything wrong or correct with how any relationship acts. The visualization algorithm stuff is a way to try and extract the relationships at a more simplistic/raw level. It can help reveal where ambiguity in the relationships develop if certain conditions are met, etc, which provide areas where the relationships can be explored more. The case where linked pixels cross is a good example, as it isn't necessarily a thing a person would immediately recognize without understanding the linked/bound relationship- because they'd instead already have a higher level understanding of the image as a whole.
...but, here is a thought which may or may not lead to a more accurate judgement whether two diagonal connections do form an ambiguous X direction in our perception. I'm pretty sure it has something to do with Contrast between the Pixels looked at and with the possible "continuations" of the "things" which we want to check for whether they create an X:
That's an interesting example. I think you're right, in that contrast plays a part in determining the continuity. Also, I think gradations act as a binder, and also have influence regarding continuity.
Here's another visual experiment I've been working on. It involves relationships between how many connections there are, similar to your idea, 0xDB.. The blue shows darker where there are more connections. The lightest blue correspond to continuous straight single line connections. The green correspond to single unconnected pixels, which are the ambiguous areas in this case, and require more exploration.
(http://i.imgur.com/4CpdcEq.png)