32/64 colour generic palette

[Arne]

It would be interesting to tackle a 32 / 64 color palette sometime. I remember Ptoing and I musing about it. It's almost daunting though, seems like a lot of work getting the most out of ramps, 'cross ramps' while covering the most useful part of the spectrum (like I histogram'ed). However, while the 'generic' 16 color palette was kind of geared towards being useful for games with smaller sprites, a palette with more colors could also be used for demo art (i.e. traced Boris Vallejo paintings ), photos, etc. The, uh, color space weight field of useful colors would be different. I started a bit on a tool for visualizing how ramps relate in color space, but didn't get very far. Just imagine little pearl necklaces jumbled into a cube, cone or cylinder. with R,G,B or H,S,V tied to any of the dimensions (Radius, Angle, Height)
http://androidarts.com/palette/

A warning: Don't screencap the Color Table in PS (5.5 atleast) and rescale down to 16*16 pixels to make a palette image. I'm guessing that color profiles or something affects those colors, so they won't be the same as on the canvas.

( I may have mentioned this before, but my CRT monitor was slightly mis-calibrated when I/we did the 16 color palette. Then I re-calibrated the monitor, and it got a little better. However, now I have an iMac with 2 flat screens, but of course, there are still calibration issues. Also, one of my eyes has a red hue and the other a green. Maybe the amount of color receptors varies. Anyways, before I had calibrated at all I couldn't see any color below 20-30 (grey) well. This meant that I was mostly working in a shorter range, making my stuff too bright and grey. I have to curve a lot of my older stuff. I'm not sure if I've calibrated the palette for the iMac LCD yet, it seems the latest version is 20 and it was made on the old PC. I might need to adjust it for brighter back-lit LCD screens. )

[ptoing]

A 64 colour generic palette would in deed be daunting to do, but I guess overall not as hard as sorting out a very good 16 colour palette.

Could be worth giving a shot at some point

[Gil]

We need to create a VM emulating an old system running arne's palette and cool restrictions on programmers. That'd be awesome.

Also, I worked on converting Arne's to 32 colors for a project I was doing with someone else (don't remember who)

[Mathias]

A 64 colour generic palette would in deed be daunting to do, but I guess overall not as hard as sorting out a very good 16 colour palette.

How would you do that?

[Ai]

A 64 colour generic palette would in deed be daunting to do, but I guess overall not as hard as sorting out a very good 16 colour palette.

How would you do that?

Basically the same as Arne did:
http://www.wayofthepixel.net/pixelation/index.php?topic=4306.0
Some reference, some thought, some drawing; trying to maximize the number and length of ramps that can be made, etc.

Just with far more checking. With a 16-color palette, you want to check each color's relation with each of the 15 other colors, at least; and you also want the palette to have a good distribution (covering the hue and lightness gamut well rather than clustering in some areas of it). Basically just amounts to: 4x the colors, 16x the amount of checking required. A semi-automated tool would probably help to cut this down.
Another approach would be to start with a larger colorcube (eg 7*7*7 == 343 colors) and automatically remove the 'least different' colors until the count of colors reaches 64. There's definitely room for some automation as a basis with 64 colors.

[Arne]

I don't think it can be easily automated. If you do 7^3 colors you end up with a lot of bad unusable colors, in particular greens, purples and neons, missing a lot of usable colors, and also make the ramps jaggy. I once made a 3D RGB histogram by sampling photos, game screenshots etc to see what colors are the most common. Perhaps that data can be used to distort the 7^3 grid, then ramp recognition routine could align the colors into lines/curves... but I'm not sure if writing such a thing is possible. Maybe something... Gaussian. There'd still be a lot of (for the computer) arbitrary parameters though, and I think an artist would have to be involved.

A while back I wrote a sodaplay'esque thing. It's easy to write in a way. The points just tug each other, trying to stay at certain distances. My 3D histogram was extremely populated in the grey areas. By setting a minimum distance for the points, but making the populated areas magnetic, it might be possible to have the points form a cluster with varied density, and also have a 3D hexagonal pattern... which may or may not be useful. If you do the same thing in 2D the points will settle into a hexagonal pattern.

I think it's easier to use something like my color space visualizer and just manually place the colors, while making sure to space and cross the ramps properly.

I discovered too late with the 256 color Cortex Command palette that I had neglected ramps (as well as monitor calibration). Most of that palette was made with automated processes (Partly Photoshop which I think uses a low res RGB histogram cube with thresholds of some sort). I went in and removed unneeded and too close colors with a program I made, but as it turned out, you can't neglect hand tuning ramps even with as many as 256 colors. I'll have to redo that palette some time :/

[Mathias]

Sounds like a colossal task, and one that would be very difficult to finally determine as complete and finished. Seems like a person would want to always tweak it from time to time. So many colors, how could you not.

Does it have to be a binary friendly number? Why not 47, for example? I too, default to numbers like 8, 16, 32, 64, etc when thinking about such things, but are these rules applicable in this situation?; Why an arbitrary limit?

And isn't 64 a bit much? What low-color games use 64 colors at once? That's enough for full gradation in smaller more monochromatic/less colorful sprites/objects, maybe not a big gradiently shaded sunset bg for instance, but certainly smaller things.

[Ai]

I don't think it can be easily automated. If you do 7^3 colors you end up with a lot of bad unusable colors, in particular greens, purples and neons, missing a lot of usable colors, and also make the ramps jaggy.

..If you are calculating the colorcube in sRGB, that's true. I find L*a*b colorcubes (as used in eg. http://pixeljoint.com/pixelart/13313.htm) more usable. L*c*h might be even better, I'll see. Anyway it's true that a plain colorcube doesn't work particularly well. More like a 14-shade L ramp applied to seven 7*7 squares of a*b* or circles of c*h*, with a little foolery to avoid duplicating grays+whites.
Like a better done version of the palette used by http://pixeljoint.com/pixelart/13313.htm#

I once made a 3D RGB histogram by sampling photos, game screenshots etc to see what colors are the most common. Perhaps that data can be used to distort the 7^3 grid, then ramp recognition routine could align the colors into lines/curves... but I'm not sure if writing such a thing is possible. Maybe something... Gaussian. There'd still be a lot of (for the computer) arbitrary parameters though, and I think an artist would have to be involved.

Yes of course. As a basis, as I said.
It is possible. IMO it would make more sense to convert that RGB histogram into LAB or LCH, then automatic fitting would be pretty simple and reasonably reliable. Might also be interesting to see what GIMP makes of your test data, since it does color reduction in LAB colorspace.

What it came up with is pretty similar to c64 palette. If I apply unsharp mask radius = 2 amount = .25 beforehand, even more so.
Additionally applying auto-whitebalance, I'm pretty happy with the resulting 16color palette.


Not so happy with the 64-color palette it picked. (I suspect the result would be better if using the fullsize test image though)

I think it's easier to use something like my color space visualizer and just manually place the colors, while making sure to space and cross the ramps properly.

I hope that's visualized in linear RGB rather than standard sRGB.. otherwise it'll suffer major distortion (eg a perceptual curve can appear straight, and a perceptual line will appear curved.) Even more so for HSV/HSL visualization.

I discovered too late with the 256 color Cortex Command palette that I had neglected ramps (as well as monitor calibration). Most of that palette was made with automated processes (Partly Photoshop which I think uses a low res RGB histogram cube with thresholds of some sort). I went in and removed unneeded and too close colors with a program I made, but as it turned out, you can't neglect hand tuning ramps even with as many as 256 colors. I'll have to redo that palette some time :/

One of the criteria I found was to maximize vividness at about 75% intensity so that you have a ramp somewhat-gray .. bright clear color.. whitish. Most programs I have found don't address that. GIMP does address that somewhat.

[Ai]

Sounds like a colossal task, and one that would be very difficult to finally determine as complete and finished. Seems like a person would want to always tweak it from time to time. So many colors, how could you not.

Hence, some automation tool is needed. Surt had some idea about fitting colors to splines to determine 'good' ramps, and I have an algorithym which finds all reasonable available ramps; these are the main things that would be desirable in a tool: tunable subjective judgement of color/palette 'kwalitee' in conjunction with objective judgement of color/palette 'kwalitee'[1]

Does it have to be a binary friendly number? Why not 47, for example? I too, default to numbers like 8, 16, 32, 64, etc when thinking about such things, but are these rules applicable in this situation?; Why an arbitrary limit?

Because of premature optimization.

Binary-friendly means no values are wasted.
Say you use a 48-color palette. You are then obligated to store your sprites in at least 6bpp format,  and 6bpp provides 2^6 == 64 different values, why not use those extra values then?
Storing the palette itself can be more convenient with a power-of-2 sized palette, too, because 3*NCOLORS*BITSPERCHANNEL always fits into an exact number of bytes when NCOLORS is a power of 2  >= 8

Of course you can use a 48 color palette, there is no good reason not to. Just, on current systems, the 'perfect fit' of power-of-2 sized palettes is seductive.

And isn't 64 a bit much? What low-color games use 64 colors at once? That's enough for full gradation in smaller more monochromatic/less colorful sprites/objects, maybe not a big gradiently shaded sunset bg for instance, but certainly smaller things.

SEGA Genesis / Megadrive uses 62 colors at once (two layers with 15c+transparency, two layers with 16c).

[1] kwalitee: "It looks like quality, it sounds like quality, but it's not quite quality."
http://pycheesecake.org/

[Arne]

Yeah, I noticed when doing my HSV, HSL, etc color space plots that you get distortions on straights and curves. But I think it's actually not a huge deal, because it's only the relations of the colors you're after. You only really need to see if the curve/line is consistent and how the ramp relates to the other ramps, and what part of the color space you've left untouched. In my program I have the ability to swap... for example HSV to HVS, or change formula for calculating intensity/ grey value. I just want to give the user the ability to see the color space in many different ways to get a feel for things.

Another issue is that of halfbrites. If we're talking old hardware, halfbrites could be useful when doing shade and light effects, like shadow and spotlights. The indexes of the different colors would have to be layed out so a shl/shr (?) and other operators would do... neat things. Yet another thing to think about is reserved colors for flashing, copper, masks, etc.

Then there's the grey value approach that the C64 palette uses to... bridge colors. With a 64 color palette, it may or may not be as useful.

Of course, the palette could actually just be a help or starting point, and not have anything to do with hardware. Most games are 24/32bit these days. In my latest project I made a separate masked image to allow me to flash a color...

[Mathias]

As a suggestion, if there's to be official community development of a 64 color palette, it probably constitutes it's own feature, so it doesn't get lost in this thread.

And Ai has already started. Way to go, Ai. I can't really decipher whether a palette is quality by seeing it so jumbled up, so I did a real quick job organizing it a little :



I still can't really comment on it until I use it for something, but it seems useful, despite some redundant shades.


**edit This palette has no pure black or white. At this point in my pixel "career" I'm still not sure if this is a good or bad thing. Black is nice for voids, though. I tend to really value shades I can use for dark subtle detail in backgrounds, etc. In my current Meteor Boy project, I don't have this and it bugs me.

[Gil]

This is my latest 32 color one, based on Arne V20. Very early try, it needs a lot of calibrating to get anywhere near Arne's level, but that's it so far.

Comments are welcome.

[Shrike]

Good, but it needs greys!  And maybe a little less pink.

[TrevoriuS]

I see 3 shades that are by far too similar, and 1 or 2 I'd never ever use on absolute level.
Perhaps fiddle around with it to make space for greys, and then try to apply it for a bit

[Arne]

Thread split?

I had some old ones laying around, so I put them on a sheet. I had one with lots of kind of monotone ramps and it didn't cover much of the color space. Then I had a few others that I've made from my histogram stuff. Today I made a 4bit per channel (4^3=64) and it left out too many grays. Finally I just made 3 of my palettes into a single palette using PS, then I noodled on the ramps. It covers the color spectrum pretty well. I put some special colors in there like duck egg blue, skin colors, etc. Sky gradient got a lot of colors, maybe too many. I generally avoid magenta hues in sky colors, and teal-green. I haven't paid any attention to brightness based cross ramping, but I threw out some colors (seen under the final top right chunk) to get a longer plain grey ramp in.

Follow the arrows. Top right is final. Top left is some old thing I made ages ago, but it's too ramp oriented and kinda dull, and it doesn't cross ramp much.


I save the 4*16 chunk as raw and open as binary in my programs. This is a HSL cylinder I believe. I can slide the discs up and down to crop out colors. It rotates too. The ramps don't really show though (with so few colors and equally spaced neighbours). I'm not sure if the palette should spend a lot of indexes on ramps though... It's a balance between coverage versus gradients. I tried a palette with more focus on ramps and it fails to explore a lot of the color space. I'm very partial to flat looking gfx with mostly light and shadow. I don't like gradient / 'chrome' crazy games like we saw back in the Amiga days.


For reference, the 4 step generated palette with maximum coverage.


Here I tried unwrapping the colors onto a 2D plane, which is impossible of course. Basically they just try to keep their ideal RGB Pythagoras distance. It looks a bit like people pushing around in a crowd before they find their places. (The dots aren't round because the iMac is immensely slow at screencapping, it takes several frames so there's frame tearing of sorts with the dots jittering about. I shake them into place.)


Here's 256 colors. They're kind of sorted I guess. One solution could do some kind of regional bias so colors will prefer certain places based on one of their dimensions.

[Ai]

thinking of that kind of approach -- I made a very carefully optimized 256-color palette for OHRRPGCE some time ago, specifically designed to work well on a wide range of pictures, to be 'game-art'-ish, and to provide good crossramping.

so, color reducing that to 64 (with 3 different variants):

The original:


no, no plain grays. IMO they're overrated (and the hue shift on the gray ramp allows for better crossramping that produces longer, smoother ramps.)

First variant (no tweaking)



Second variant (contrast bumped up)



Third variant (auto white-balance)



If pure black and white are crucial, of course you can replace the lightest and darkest color with them.
I think all of the above are pretty good and flexible, I prefer v3 myself.

[Arne]

On your 3rd version, some of those bright colors looks a bit close, perhaps. Of course, I'm more into games with a black BG, so bright colors are a bit lost in that case, just like dark colors are lost against a bright BG.

Here's mine, same as before, I just organized the colors after hue and brightness. Need to do something about the two mid blues.

[happymonster]

Looks yummy Arne!!

Are you going to put this together with your 16 colour palette in one new thread?

[Gil]

This is less spaced out ramp-based palette I did some months ago, based on Arne's. 32 colors again.

Would this work better or worse than the other palette? I think my other one has more future.

[Lazycow]

Quite nice, but 2 of the grey's are not "clean" greys, and there are only 2 browns. (but 4x lila purple, which is probably not needed)

[Shrike]

I'd take out some of the shades of purple, and add in 2 greys and have a black-grey-grey-white ramp.  If you like purple too much try getting rid of the two bottom ramps and make a grey there, and up the contrast of the others, since there aren't really any darks.

[Gil]

Quite nice, but 2 of the grey's are not "clean" greys

Why would you need clean grays guys? I provided slightly tinted grays, which should work just fine. I think any straight gray ramp looks boring and I never use it in a piece. Slightly tinted, they still provide as buffer colors, but the ramp doesn't look boring.

You are probably right about the browns, there can be an extra one there to bridge the orange.

The purple and blue ramp are godawful right now, same with the dark colors, I guess I've got some more work to do.

[Arne]

I'm tempted to skip plain greys and go with one warm and one cool grey ramp. When you draw with markers, it looks good if you use different temperatures.

Looking at my histogram, grey gets a huge amount of hits, but with games / art it's more useful to explore the more saturated regions of space. Still, I could probably convert a few of my more grayish colors and the plain grey ramp into two warm and cool ramps.

On the other hand... my plain greys can be made to look cool and warm when placed adjacent to other colors, and as neutral grays they bridge better to any given color. But with cool and warm ramps I could also bridge, with more fitting hues to choose from. I guess someone who has used the C64 palette is more suitable than me to be a judge on this matter.

I should probably take a look at the Chaos Engine and Speedball palettes. They have a lot of nice hues in the greys, iirc.

[Mathias]

Any chance of a thread resurrection here, Arne? Anything further to add? Highly interested.
Note my current 'Expand C64 pal to 32 colors' thread.




*EDIT   Here's this thread's results organized into one bar and sorted by luminosity, as per ProMotion's greyscale conversion method.



Note the defined "fourths" in the luma ramp.