AuthorTopic: Realtime Rendering of 3d Meshes as Pixelart  (Read 128127 times)

Offline Howard Day

  • 0010
  • *
  • Posts: 222
  • Karma: +0/-0
  • Hey, I'm a doofus.
    • View Profile

Re: Realtime Rendering of 3d Meshes as Pixelart

Reply #80 on: October 01, 2015, 09:51:55 pm
Gil: ..Uh, wow - so, I'm a little surprised you're taking this to the "this can't possibly ever work" level... To be honest, I thought we'd been working pretty well together, and I thought I'd been making great progress. Honestly, I and you both know this will never replace pixel art - that's not the point - the point is to get as close as we can, and I don't think we're there yet. I've reversed the dithering back to a simple 2x2 checkerboard for all further main builds, but when it's released people can change that to whatever. I realize that you're just trying to be a hardass - but try working with me. :)
Ai: Getting exact Bayer dithering patterns is more complex than you'd think. You have to put all the info into a single texture.
Seiseki: Thanks - and yeah, each model has a special shader on it - right now all the objects are using the SAME special shader/material - but this will mean you can mix and match these assets with other objects that aren't pixeled.
Cherno: Oh, okay - well in that case, it's as simple as providing the grayscale image you want to dither with. You can do some pretty cool stuff:

which used this image to dither:


More to come.

Offline Ambivorous

  • 0010
  • *
  • Posts: 365
  • Karma: +2/-0
  • If you can't do; inspire.
    • ambivorous
    • ambivorous
    • http://pixeljoint.com/p/47949.htm
    • ambivorous
    • View Profile

Re: Realtime Rendering of 3d Meshes as Pixelart

Reply #81 on: October 01, 2015, 10:07:14 pm
Oh right, you're the Wings of Saint Nazaire guy!
Keep up the good work man, your stuff is awesome. And this thing is awesome.  :y:

I actually don't have anything of value to add though, so just carry on.
* may contain misinformation

Offline Ai

  • 0100
  • ***
  • Posts: 1057
  • Karma: +2/-0
  • finti
    • http://pixeljoint.com/pixels/profile.asp?id=1996
    • finticemo
    • View Profile

Re: Realtime Rendering of 3d Meshes as Pixelart

Reply #82 on: October 01, 2015, 11:59:11 pm
TBH I prefer anything that doesn't produce those weirdly-clustering isolated 'bits of checkerboard' we were getting before.  Increasing dither levels to 4 (2x2 bayer matrix) or more would do that IME. No dither at all would also do that.
But I specifically enjoyed those bits of checkerboard :(

The same ones? I had no issue with the chunks of checkerboard that were connected to solid pixel regions of the same color. Ones that are isolated 'pop out' visually to me, looking like a small bland separate object, which I think is not generally the desired result. That's the parts I'm talking about.
I would say that generally, pixel art that uses dithering avoids such things (usually by providing a mass for the dithering to 'attach' to).

Ai: Getting exact Bayer dithering patterns is more complex than you'd think. You have to put all the info into a single texture.
I don't understand what the difficulty is. Bayer matrices are exactly small 2d textures that you threshold; which as I understood is exactly what you are already doing.
For example, what would be the problem with using this:


(2x2 bayer matrix tiled 128x128 times, intensities used are 25% 50% 75% 100%)
« Last Edit: October 02, 2015, 10:29:23 pm by Ai »
If you insist on being pessimistic about your own abilities, consider also being pessimistic about the accuracy of that pessimistic judgement.

Offline Gil

  • 0100
  • ***
  • Posts: 1543
  • Karma: +1/-0
  • Too square to be hip
    • http://pixeljoint.com/p/475.htm
    • View Profile
    • My Portfolio

Re: Realtime Rendering of 3d Meshes as Pixelart

Reply #83 on: October 02, 2015, 07:45:30 am
Gil: ..Uh, wow - so, I'm a little surprised you're taking this to the "this can't possibly ever work" level... To be honest, I thought we'd been working pretty well together, and I thought I'd been making great progress. Honestly, I and you both know this will never replace pixel art - that's not the point - the point is to get as close as we can, and I don't think we're there yet.
The reason I took that route is because I went in with graphicsgale, ready to provide to edits that could help you out and found there's very few pixels there that felt placed. I do a lot of cleanups of processed stuff and that's what it felt like, like I'd had to clean the whole image.

I'm a bit skeptic that you'll ever get rid of most of those, but hey, if you're optimistic, I'm all for it. I personally would cop-out at this point and go for a simpler mechanical style that I can get away with more easily. So, sure, let's see how far we can get first, pretty sure a lot of work can still be done.

So, that latest version is wonderful, I feel like the earlier quality is back again. What's more is that the lady looks pretty good for the first time. Not sure why she's behaving so well right now, but I'd try to hang on to that. The dither pattern shows off that feature really well and I like it.

What I'm wondering right now is if there's a way to cluster stuff more. Basically, I see clusters as a bias to join the surrounding colors. In pseudo-algorithm something like: "The closer a pixel is in color to neighboring pixels, the more likely it is to change its color to those pixels". For example, a single grey pixel in a field of white would probably change color to white. A black pixel wouldn't though, it's too far removed from the surrounding white to feel bias. The result would have to be that areas of one color clump together more. This bias would be introduced before dithering, or it will try to "fix" the dither pattern.

Offline Howard Day

  • 0010
  • *
  • Posts: 222
  • Karma: +0/-0
  • Hey, I'm a doofus.
    • View Profile

Re: Realtime Rendering of 3d Meshes as Pixelart

Reply #84 on: October 02, 2015, 10:11:41 pm
Ambivorous: Yo. :)

Ai: Oh, well the issue is that I'm not smart. I didn't understand what you were saying, and was going about it a totally wrong way. Sorry. I've put your dithering matrix, as is, into the build - and it looks pretty cool!

Gil: Hmm. pixel clustering is likely something I could write a shader for, but only on DXT11 platforms... I'm looking at ideas on how to push it more - like the 2-pixel minimum clustering thread - but forcing the shader to do things the way I want is turning a bit awkward.
Still working on it, but I think my next task is to make a smoke and background shader. those two things are bugging me. :D

And the build's been updated.
http://www.hedfiles.net/PixelShader/index.html

Offline ptoing

  • 0101
  • ****
  • Posts: 3063
  • Karma: +0/-0
  • variegated quadrangle arranger
    • the_ptoing
    • http://pixeljoint.com/p/2191.htm
    • View Profile
    • Perpetually inactive website

Re: Realtime Rendering of 3d Meshes as Pixelart

Reply #85 on: October 02, 2015, 11:20:34 pm
This is pretty cool. I was wondering if you could perhaps use a depth-buffer to determine a shadow threshold for things like the legs and such, to get a similar effect like in Capcom fighting games, where far limbs will be darker and the like.
There are no ugly colours, only ugly combinations of colours.

Offline Ai

  • 0100
  • ***
  • Posts: 1057
  • Karma: +2/-0
  • finti
    • http://pixeljoint.com/pixels/profile.asp?id=1996
    • finticemo
    • View Profile

Re: Realtime Rendering of 3d Meshes as Pixelart

Reply #86 on: October 02, 2015, 11:53:15 pm
Ambivorous: Yo. :)

Ai: Oh, well the issue is that I'm not smart. I didn't understand what you were saying, and was going about it a totally wrong way. Sorry. I've put your dithering matrix, as is, into the build - and it looks pretty cool!
Yeah, that seems to fix the issue with clusters of checkerboard without looking overly un-pixely. Looks nice, especially the napalm on the woman is closely approaching my idea of how it should ideally look.

On a slightly different but related topic..
I'm not necessarily saying this is a good idea, just putting it out there because the texture you posted earlier inspired it:

Have you considered the possibility of a bias value for the dither matrix? A value that you add to the texture value before thresholding it? That would allow tuning 'ditheredness' up or down (hopefully on a per-object basis, so that you can dither your landscapey stuff and VFX, and have no dither on your characters).

Quote
Gil: Hmm. pixel clustering is likely something I could write a shader for, but only on DXT11 platforms... I'm looking at ideas on how to push it more - like the 2-pixel minimum clustering thread - but forcing the shader to do things the way I want is turning a bit awkward.
2pix minimum would be nice as an option in a similar vein to the option I suggest above (that is, as a way of emphasizing the different roles of visual elements by differentiating them stylistically). It is surprisingly hard to solve well, though, IME -- a fair bit of planning goes into getting a final result that is both reasonably clean and attractive.

« Last Edit: October 03, 2015, 12:04:26 am by Ai »
If you insist on being pessimistic about your own abilities, consider also being pessimistic about the accuracy of that pessimistic judgement.

Offline Howard Day

  • 0010
  • *
  • Posts: 222
  • Karma: +0/-0
  • Hey, I'm a doofus.
    • View Profile

Re: Realtime Rendering of 3d Meshes as Pixelart

Reply #87 on: October 03, 2015, 12:19:03 am
ptoing: Hmm - but one that only affects the shadowed areas? Yeah, I can do that. :D

Ai: Oh, good. Glad it fixes it, sorry I was an idiot. In better news, the Dither bias already exists! It's settable on the objects shader generally, and within each object as well, as the vertex color of green drives how much dither the surface gets. In some cases, mostly the transparent alpha ones, color dithering intensity is separately settable from alpha dithering. It also provides and overall alpha bias offset - the flame on the lady is the same as the explosion - with the alpha bias dropped down a bit to make the flames less billowy. Still mulling over pixel clustering ideas - there might not be a good solution.

Offline Cyangmou

  • 0011
  • **
  • Posts: 929
  • Karma: +3/-0
    • cyangmou
    • http://pixeljoint.com/p/32234.htm
    • cyangmou
    • View Profile
    • Pixwerk Homepage

Re: Realtime Rendering of 3d Meshes as Pixelart

Reply #88 on: October 03, 2015, 01:35:10 am
while it is impressive as it's own style I won't necessarily call the current result that it's anywhere close to pixel art, except being "made out of pixels".

There are some disagreements what and what is not pixel art and I am pretty open for everything stylewise, however for myself pixel art is the very controlled use of pixels to make an image.

Pixel art is also restricted heavily by the grid and one of the main things one have to keep in mind next to angles is readability.
If I check the current artwork here, it looks like a color reduced 3D model with a dither filter.

Why?
It's not clean in terms of outlines
there is a lot of half blurring and half assed AA going on where the underlying lines of the 3d model fall through the grid.
A pixel artist always would draw very clean outlines and AA them with not more than 4 colors

The model is not a great design to be realized in pixel-art
This might sound stupid, but there are limits what one can do with pixel art - and those limits don't apply to the drawing level, rather down to the design level.
Pixel art is definitely not about drawing stuff with fat pixels - it's about using a grid and the suggestion of clear cut blobs of colors to illustrate something - clean and readable.
Your model has by far more subtle planes than a pixelartist would ever use in his work and the shading of the planes happens via dither.
If I'd do something like this per hand I wouldn't use any dither. I would clearly seperate the planes with different color values on a palette.
In your model there are some planes (especially round ones) which have 5 colors plus dither, which is an effect no pixel artist would ever go for. It takes to long and works against having clear and strong clusters.

If you get the outlines cleaner and can reduce the colors per plane used to a clear value you will already be much closer to the looks of actual pixel art. If that's possible you maybe want to try to get this working.



The points where I think a shader like this won't work, is that with pixel art the artist liberally can break tons of stuff a 3d program would handle differently, in terms of angles used, shading enhancement like all 2d does. Pixel art is a lot further away from realism just because the pixel artist always considers readability as credo.

Since I haven't worked with a shader like that and the only thing which is close to it is actually Guilty Gear XRD's anime filter. I think it will be kinda similar to that if you get it working - tons of additional work and information to add to models, to maintain what one would do right away with 2D techniques.
"Because the beauty of the human body is that it hasn't a single muscle which doesn't serve its purpose; that there's not a line wasted; that every detail of it fits one idea, the idea of a man and the life of a man."

Dev-Art
Twitter

Offline Ai

  • 0100
  • ***
  • Posts: 1057
  • Karma: +2/-0
  • finti
    • http://pixeljoint.com/pixels/profile.asp?id=1996
    • finticemo
    • View Profile

Re: Realtime Rendering of 3d Meshes as Pixelart

Reply #89 on: October 03, 2015, 02:23:31 am
A pixel artist always would draw very clean outlines and AA them with not more than 4 colors

In your model there are some planes (especially round ones) which have 5 colors plus dither, which is an effect no pixel artist would ever go for. It takes to long and works against having clear and strong clusters.
That seems slightly problematic:


(remade gfx for Bub-n-Bros. Lots of colors (24 - 266 per icon), no dirty tools used (not even lowered opacity), coherent on a pixel level)

Either the originals are pixel art and these are not -- purely through color count increase -- or color count is not a suitable metric.

I do agree with all the other points you made though, especially about model design.
« Last Edit: October 03, 2015, 02:29:01 am by Ai »
If you insist on being pessimistic about your own abilities, consider also being pessimistic about the accuracy of that pessimistic judgement.