Pixelation

Critique => Devlogs & Projects => Topic started by: rizer on May 27, 2016, 10:30:23 am

Title: Pixaki
Post by: rizer on May 27, 2016, 10:30:23 am
Hi, I'm the creator of a pixel art app called Pixaki. I'm working on a new version, and I have some questions about how things in the app should behave, so I thought "those Pixelation people seem nice, and clearly know a lot about pixel art, maybe I should ask them"  :)

A big new change I'm making is the option to have indexed colour mode documents. The app supports layers, and at the moment it's possible to adjust the opacity of a layer, and then merge it down, which obviously has the potential to produce a whole load more colours. So I thought that maybe in indexed colour mode there should be no semi-transparency in the layers or the selected colours, but in RGBA mode people can do what they like. Does that make sense? Would it be annoying to not be able to adjust a layer's opacity when working with indexed colours, or is that what you'd expect?

I'd love to hear what you think. Thanks for your time,

Luke
Title: Re: Pixaki
Post by: eishiya on May 27, 2016, 05:21:21 pm
When working in indexed/palette mode, I would expect it to be index painting, where the colours "snap" to the closest colour that is in the palette. This would apply not just to layer opacity, but "fuzzy" tools like gradients, soft brushes, etc.
Title: Re: Pixaki
Post by: PixelPiledriver on May 27, 2016, 11:44:01 pm
Yo, welcome to Pixelation.  :)

I was just looking at Pixaki a few days ago.
Nice to hear you are still supporting it.  :y:
Will give it a try.

Quote
Would it be annoying to not be able to adjust a layer's opacity when working with indexed colours, or is that what you'd expect?
What works the best for me is to have RGB and A as separate elements with their own index tables.
That way alpha can be changed and controlled globally just like colors.
Title: Re: Pixaki
Post by: Ai on May 28, 2016, 09:43:06 am

Quote
Would it be annoying to not be able to adjust a layer's opacity when working with indexed colours, or is that what you'd expect?
What works the best for me is to have RGB and A as separate elements with their own index tables.
That way alpha can be changed and controlled globally just like colors.
That looks to me like you answered a different question than he asked? You can have indexed color with opacity and layer modes (GIMP 2.9 does this), or without; and you can also quantize the result to existing palette, or add new colors. Your comment appears to address *document-wide* color / alpha rather than layer blending, or am I mistaken?

(Personally I'm a bit confused as how indexed alpha channel is supposed to work anyway, once you have a layer stack. Blend normally and quantize the result to the 'alpha palette'? Take the maximum of the lower and upper channel alpha? Something else?)
Title: Re: Pixaki
Post by: Probo on May 28, 2016, 12:39:26 pm
Would it be annoying to not be able to adjust a layer's opacity when working with indexed colours

I would absolutely, massively like the choice to change a layers opacity when working with indexed colours! I use Graphics Gale most of the time, and that doesn't let you control the alpha of a layer in indexed colour mode. Or rather it still lets you move the layer's opacity slider, but it dithers the layer by deleting pixels rather than changing opacity. Which would also be a nice option to still have I guess, but I don't use this dithering transparency often at all. Its not a particularly clever dithering algorithm either, maybe if it was id use it more.
Title: Re: Pixaki
Post by: 32 on May 28, 2016, 01:09:27 pm
I don't think I'd use opacity much as an artistic tool when I'm in indexed mode but it would definitely be nice to have available for the sake of being able to see through layers. Definitely get hamstrung by GraphicsGale sometimes because of the dithering system Probo mentioned. I think if you just had an option when merging layers to disregard opacity that would be ideal.

Behaviour like Eishiya mentioned would also probably come in handy now and then artistically.
Title: Re: Pixaki
Post by: eishiya on May 28, 2016, 03:50:47 pm
32's suggestion reminds me of onion-skinning, and that is a feature I'd like to see in more programs too. It would be very cool to be able to set a layer's true opacity (with the colour snapping like I described), as well as its apparent/onion skin opacity, in which case it's purely a UI thing like selections (marching ants are displayed on the image but aren't actually a part of it), except for opacity. When flattening, picking colours, etc, the onion skin opacity would be ignored, while the true opacity would be taken into account. This is something that would be useful both for RGBA and indexed modes.
Title: Re: Pixaki
Post by: PixelPiledriver on May 28, 2016, 11:16:51 pm
Quote
That looks to me like you answered a different question than he asked?
Yep, my bad.
Def agree with everyone else's suggestions.

Quote
Personally I'm a bit confused as how indexed alpha channel is supposed to work anyway
It's a palette of black to white blocks.
Just like colors except they represent alpha steps instead.
So you have a restricted set of alpha values available.
Title: Re: Pixaki
Post by: Ai on May 29, 2016, 03:28:41 am
Quote
Personally I'm a bit confused as how indexed alpha channel is supposed to work anyway
It's a palette of black to white blocks.
Just like colors except they represent alpha steps instead.
So you have a restricted set of alpha values available.
Yes, I understand the concept, but not how it's supposed to work in context of a multi layer document. That was what this bunch of questions: "Blend normally and quantize the result to the 'alpha palette'? Take the maximum of the lower and upper channel alpha? Something else?" was addressed at). So basically I'm having difficulty seeing how you're supposed to combine [indexed alpha] + [layering] in an intuitive/consistent way -- how compositing should interact with the alpha palette.

@rizer:
Getting back to answering OP's question, personally I have a use for a) onionskinning, b) layer opacity with quantization to palette, c) layer opacity without quantization to palette.
In that order of priority (but I think b might depend on c to work sensibly? GIMP does c and then applies the quantization only when you merge down the layer, so what you get when you merge isn't always exactly what you expected.)

I can't comment on experiences with your app cause I don't have a iPad; keep that in mind if I say something ignorant.

If you don't have something like GraFX2's QShade mode, then some limited selection of layer modes (eg. add, sub, grain merge, grain extract) would be appreciated. If you do have a QShade-alike, then it's cool (qshade covers most of the same use cases)

To address what Probo said about alpha dither: I don't think it's particularly important, but would be neat to have -as an option- (like a variant of 'Dissolve' blend mode that's not randomized, instead based on a standard 4x4 Bayer matrix.)
Title: Re: Pixaki
Post by: rizer on May 30, 2016, 10:34:25 am
Wow, amazing, thanks so much everyone! This is a real treasure trove  :)

So putting everything together, here's what I'm thinking I'll do:

Keep layer opacity as an option, just like in RGBA mode. If you choose to export the document and you have layers with semi-transparency, then you'll just get an image with more colours than you have in your palette — I don't think I need to tell the user that they've done this, I think it should be pretty obvious.

If you select to merge a layer at 100% or 0% opacity, it will just merge down (the latter will just delete the layer).

If you select to merge a layer with any other opacity, you'll be presented with some sort of dialogue asking if you'd like to add the colours to the palette or quantise to existing colours.

Onion skinning will be available if you have more than one frame of animation.

Which leads me on to a few more questions:


Thanks again!
Title: Re: Pixaki
Post by: 32 on May 30, 2016, 11:24:55 am
1. Definitely no expert on this but that's generally how I would expect it to work, whether there's more effective ways to do it I don't know.

2. You're definitely right that losing colour detail in this process makes it basically useless. I think if the new palette has fewer entries I would match up closest colours and keep the extra entries. Options make a happy artist though so an interactive colour merge sounds great ;D

3: In addition to what you've mentioned an option to colour the front and back frames is extremely useful. So say the forward frame is tinted green and the back frame red. Otherwise it's basically impossible to distinguish between the two frames if you have them both turned on. Being able to see multiple frames forward and backward with diminishing opacity is also nice but there are fewer instances where I feel like I need that functionality in normal pixel art animating.
Title: Re: Pixaki
Post by: Ai on May 30, 2016, 12:22:33 pm
Does anyone have any thoughts on how the colour quantisation algorithm should work? My first thought was maybe to calculate the difference in hue, saturation, and brightness for each colour in the palette and the target colour, then take the palette colour with the smallest ΔH + ΔS + ΔB?
No. I mean, it's an understandable first thought, and the basic method is sound, but it would give crap results because HS* are pretty bad for measuring difference, so please don't do that. I'd also suggest staying away from standard -- non-linear -- RGB for this purpose.

Ideally: see https://en.wikipedia.org/wiki/Color_difference . Delta-E, which is described in this article, does a pretty good job, even the simpler versions of it. It uses LAB or LCH colorspace, depending on exactly which version you are using. GIMP's 'convert to indexed' quantizes using LAB colorspace, so it can serve as a demo of the results.

However, that is relatively computation-intensive, and also tricky to implement in a genuinely correct way. So if you can't do that, use linear rgb differences (see https://en.wikipedia.org/wiki/SRGB#The_reverse_transformation , and ignore the part that turns linear rgb into XYZ). Linearization of RGB can be done with a 256-entry lookup table, so it's fast, and mathematically it's pretty easy to do right.
Title: Re: Pixaki
Post by: Probo on May 31, 2016, 12:49:24 pm

1. I don't have a suggestion for an algorithm, but if its not too much work to integrate the kind of existing algorithms ai mentioned, why not add multiple algorithms and give us some choice in the UI? that would be pretty cool ive got to say

2. totally agree with what 32 said. A detailed palette merger where you can either manually decide on a per-colour basis whether it comes over (and choose which colour it replaces etc), or just apply the automatic colour-matching algorithm that does the work for you and stores the unused new colours in the palette, would be good. A 'remove unused colours' option like GG would be good, just in general palette options or a button.

3.
yeah that sounds good, Id like the option to show only the frame behind or only the frame in front though, it can sometimes be a bit confusing to see both.

Title: Re: Pixaki
Post by: Ai on June 02, 2016, 09:08:50 am

1. I don't have a suggestion for an algorithm, but if its not too much work to integrate the kind of existing algorithms ai mentioned, why not add multiple algorithms and give us some choice in the UI? that would be pretty cool ive got to say
MTPaint is a good(?) example of a program that does this.

It offers the following options;

* Number of colours
* Color matching:
  * Colorspace:
    * RGB
    * sRGB
    * LXN [note: similar to LAB. I have no idea why they felt they had to implement this instead.]
  * Difference measure:
    * Largest (Linf)
    * Sum (L1)
    * Euclidean (L2)
  * Reduce color bleed:
    * Gamut
    * Weakly
    * Strongly
  * Serpentine scan
  * Error propagation % [note: only applies to FS dithers AFAIK; used to reduce 'noisiness']
  * Selective error propagation: Off | Separate/Split | Separate/Sum | Length/Sum | Length/Split
  * [X] Full Error precision
* Method of deriving palette:
  * Exact
  * Current palette
  * PNN Quantize (slow, better quality)
  * Wu Quantize (fast)
  * Min-Max quantize (best for small pictures and dithering)
* Dithering:
 * None
 * Floyd-steinberg
 * Floyd-steinberg (quick)
 * Stucki
 * Dithered (effect)
 * Scattered (effect)

To me, this is an example of what not to do -- overwhelm the user with a huge array of options, rather than hardcoding many settings to generally-sensible values. However, it does illustrate the breadth of the problem domain, and the program itself could also serve as a tool for you, @rizer, to figure out exactly what color reduction algorithm is best for you to implement in Pixaki.
Title: Re: Pixaki
Post by: rizer on June 02, 2016, 10:28:40 am
That's a lot of options!

Thanks for the feedback, especially @Ai for pointing me to Delta-E  ;D

I've been busy writing the code over the last couple of days, and I've just finished my Delta-E implementation, which seems to be working. I went with the '94 formula, which seems to be a nice middle ground between accuracy and speed. Quantising an image is my next step… maybe I'll post in the results.

@32 The shift in colours for the onion skin sounds like a good idea, and shouldn't be too hard to implement.

Thanks everyone!

Title: Re: Pixaki
Post by: rizer on June 02, 2016, 05:45:38 pm
And here are some results…

(http://cl.ly/3z2h44041d2R/SkirrelSmall.png) (http://cl.ly/373Q3G070Z29/SkirreSmallQuantised.png)
Original on the left, quantised to the Android Arts palette (http://androidarts.com/palette/16pal.htm) on the right.

Full size original (http://cl.ly/2P2j1M1U1Q3W/Skirrel.png) | Full size quantised (http://cl.ly/031A230p2T2C/SkirrelQuantised.png)

It's pretty slow at the moment, particularly for such a large image, but the results seem good to me. Any thoughts?

A couple of sites that have been really useful for this are ColorMine (http://colormine.org) and EasyRGB (http://www.easyrgb.com) if anyone's interested in this sort of thing (and Wikipedia of course!).
Title: Re: Pixaki
Post by: Ai on June 03, 2016, 07:15:05 am
Nominally good, yes.

By which I mean, that sample picture is pretty big by pixel art standards. So while your results look pretty good -- color matching is quite accurate -- , I don't think they really represent the scale or style of picture that people will be likely to reduce.

I filtered your sample a bit (GMIC Anisotropic smooth + local normalization + gamma correct downscale to 25% the size) to arrive at something that IMO is more representative:

(http://i.imgur.com/TbVq5ao.png)

GIMP reduction of this image to Android Arts palette:
(http://i.imgur.com/XOMi3yn.png)

(tell me if the above images don't show up. They work for me but I'm getting suspicious of Imgur recently)

EDIT: To clarify, a wide variety of images might be used. But I don't really see why any of them would be hi res or contain many fine details, given the context.
Title: Re: Pixaki
Post by: rizer on June 04, 2016, 01:23:12 pm
Thanks @Ai — those images seem to be working fine. Yeah, I agree it's good to test results with images that are closer to what people are likely to be using. Here are my results with the image you posted:

(http://cl.ly/3v3M1v2w061T/download/SquirrelNotGIMP.png)

It's certainly different. Do you think the GIMP results are better? I wonder whether they're either using a different Delta-E algorithm, or performing some other steps (or maybe both).

Thanks!
Title: Re: Pixaki
Post by: rizer on June 04, 2016, 01:31:37 pm
Thought I'd quickly implement the Delta-E 76 algorithm to see what the results are:

(http://cl.ly/0L2z0v0m1E37/Squirrel76.png)

Certainly closer to my previous results than to GIMP's. I would guess that GIMP is doing some sort of smoothing, which is probably desirable for photos, but less so for pixel art.
Title: Re: Pixaki
Post by: Ai on June 04, 2016, 02:23:21 pm
I think your results are better. The '76 algorithm seems better, all other things being equal.

FWIW, GIMP doesn't do "smoothing" per se. It builds a LAB-colorspace-based histogram of the image and divides this histogram up into chunks ("bins"), each of which it assigns a color. If a color is inside that bin, it gets that color. Naturally, this is faster and somewhat less accurate than doing color difference calculations for each unique input color (which I guess is what you are currently doing. Anyway, AFAIK the algo I described for gimp is known as octree cut or median cut - pretty standard.)
Title: Re: Pixaki
Post by: rizer on June 07, 2016, 02:05:09 pm
Thank @Ai  :) Yeah, I came across median cut when I was reading about colour quantisation  — could be useful if I ever want to create a palette from an image.

I have some more user interface questions if anyone would like to help:

Firstly, with an indexed colour palette, what should happen if the user deletes a colour from the palette? Aseprite shifts all subsequent colours back and recolours the artwork so that the new indexes match, but I think that would be really confusing in Pixaki. The other options I thought of were to erase all pixels of the deleted colour, or to not allow the colour to be deleted if it's in use. Any thoughts?

My second question is to do with blend modes for layers. Would they be useful or is it not a high priority for most people?

Thanks!
Title: Re: Pixaki
Post by: Ai on June 07, 2016, 02:30:36 pm
Firstly, with an indexed colour palette, what should happen if the user deletes a colour from the palette? Aseprite shifts all subsequent colours back and recolours the artwork so that the new indexes match, but I think that would be really confusing in Pixaki. The other options I thought of were to erase all pixels of the deleted colour, or to not allow the colour to be deleted if it's in use. Any thoughts?
Well, Grafx2 simply ignores the idea of deleting a color from the palette. The palette is always 256 colors, even if most of those are duplicate blanks. Pretty sure other '256color only' paint programs (eg. DeluxePaint) do this too. You use the swap or sort functions if you need to reorder the palette, and that pretty much covers it.

Don't really have time to say more but I think it's a pretty good + easy to understand approach.