Realtime Rendering of 3d Meshes as Pixelart

[RAV]

Looks nice. but I also think you traded problems here. not to bum you out with being nit-picky, but consider this going further:

If pixel art is not pixel perfect it just doesn't work. To me it seems, the less it looked like pixel art, the better it looked overall.
The more it restricts itself now, the more glaring the issues, which make looking any more like pixel art not necessarily more desirable.

When we look at the gun tips of the rotating white gunship for example, on each rotation it's a wiggly waggly mess of clusters that are barely recognizable as barrels. Keep in mind, the exact perspectives and angles of lines in pixel art are not by chance. Pixel artists know which ones make for good clean art and which ones make it too tough to get right in a prominent grid aesthetic restricting it. The freedom of 3d now breaks all that, and with it the pixel art breaks on its usual breaking points, all over the place, randomly.

[Gil]

Daaamn, that's a massive improvement. I'm starting to get really excited

RAV: I don't think you can judge these by themselves, as pixel art.

1) They are not and not meant to be pixel art
2) You'd probably use this shader in a scenario where it makes sense, like combining it with pixel art for certain characters that would otherwise prove impossible to animate

For that second point, I'm wondering if the shader performs better than usual at portraying small objects.

[Howard Day]

Rav: That's an issue with the model, not the shader. Any attempt to fit more shape information into a subpixel area would end up messy, no matter the medium. Here's a cleaned up version:

The shader and procedure have improved. The model just needed to improve a bit with it.

Gil: Thanks! They are meant to emulate pixel art - and I'm game to keep pushing till it's looking even better.

[32]

This is absolutely lovely  I've often wanted to see this done but never really expected to. I think this definitely gives a good enough impression that you could use it alongside hand made pixels for more complicated stuff. Though the character also look pretty cool. I'd love to see how this looks on more traditionally pixel art styled and sized characters.

I think you should leave the dither off. Looks nice on some of the smaller panels on the walker but the big solid areas of dither and dither on the outlines is nothing a person would do. Maybe that's something you can fix but dither is pretty out of fashion anyway.

Can't wait to see where this goes

[RAV]

: I don't think you can judge these by themselves, as pixel art.

1) They are not and not meant to be pixel art
2) You'd probably use this shader in a scenario where it makes sense, like combining it with pixel art for certain characters that would otherwise prove impossible to animate

What I point out also are aesthetic problems regardless of what art it is. Yet the topic is "3d meshes as pixel art", and Howard stated specifically it is his interest to find out how close he can get to look like it, and asked for what problems are in his way. Actually, trying to integrate it with other real pixel art makes judging and discussing the problems as pixel art even more important, since you would want to avoid glaring aesthetic breaks between the assets sitting next to each other. Maybe I come across too critical though. And after all, you are an actual pixel artist on professional level, so I'd say Howard should rather listen to your opinion on what you like about it.

That's an issue with the model, not the shader. Any attempt to fit more shape information into a subpixel area would end up messy, no matter the medium. Here's a cleaned up version:

It is an issue with computed rendering at large. And that's what I'm pointing to: it isn't an issue in pixel art, because the pixel artist knows and controls exactly the conditions of display that don't change, and tailors the art perfectly to fit that, understanding what detail fits what degree, distance and sub-pixeling, for example.

The more flexible you make the conditions of display, the more the problems in the rendering as pixel art. There are little to no margins for mistakes in real pixel art. The adaptions you made to the model are mostly good for the conditions you adapted them for. Change distance, and they come back the same, among the issues it still has either way compared next to manually polished pieces. Curiously though, micro-managing models like that to fixed angles and distances, starts to eat up some of the advantages of going 3d.

There are still some good reasons one might want to opt for that regardless though, and you can search for recommended sweet-spots of modelling to alleviate some of the problems in some range and degree. You can try cheat a bit the pixels, and a sort of predefined Level of Detail system might help too in extremer cases of use. Just keep an eye on what expenses are worth what result.

[Gil]

What I point out also are aesthetic problems regardless of what art it is. Yet the topic is "3d meshes as pixel art", and Howard stated specifically it is his interest to find out how close he can get to look like it, and asked for what problems are in his way. Actually, trying to integrate it with other real pixel art makes judging and discussing the problems as pixel art even more important, since you would want to avoid glaring aesthetic breaks between the assets sitting next to each other. Maybe I come across too critical though. And after all, you are an actual pixel artist on professional level, so I'd say Howard should rather listen to your opinion on what you like about it.

Oh, I would hardly value my opinion above anyone else on this forum. I do get your point. I think the point is this: I think he can still get a fair bit closer to pixel art, but he'll never get close enough to be able to call it pixel art. The simplest definition of pixel art is art where you have control over individual pixels. I don't feel like any of the renders have enough control to call it pixel-level control. I do think you can get away with combining stuff like this with pixel art though, and that would be my aim if I were to make something like this. Also, the fact that you can make procedural animations is HUGE. The only way to have procedural pixel animation is by using a combination of lots of keyframes and paper-dolling, which doesn't provide you with even a sliver of what's possible with 3D models.

[Ai]

The more flexible you make the conditions of display, the more the problems in the rendering as pixel art. There are little to no margins for mistakes in real pixel art. The adaptions you made to the model are mostly good for the conditions you adapted them for. Change distance, and they come back the same, among the issues it still has either way compared next to manually polished pieces. Curiously though, micro-managing models like that to fixed angles and distances, starts to eat up some of the advantages of going 3d.

That reminds me of that article explaining GGXrd's art style, Conceit posted it awhile back in the OT thread IIRC.
this is a related link, but not it. Anyway they were doing just that kind of stuff -- they had a bunch of layers of stuff (textures, normals, etc) to control all that stuff, and even loads of kinematics just for distorting the model to look maximally awesome in each frame. Getting kind of crazy..

Howard:


That is so much better it's ridiculous. I think when you release it, if it's popular you'll probably find the community arrives at the idea that there is a kind of .. different way of designing your model for each broad level of scale (so that it reads well, that is.). But even so, getting the flexibility of 3d with that limitation, still has a huge scope for time saving and exploring options that otherwise couldn't be tried.

[RAV]

I agree to that. Yet keep in mind to be honest in your judgement of use when the time comes, for all the details of the issue I pointed out before for this reason. Especially when it comes to integrating these two arts, the power you feel in the possibilities might blind you for the aesthetic overall costs your project could suffer for it, or that it could lock you down visually as well in other ways. Whether it is a difference in polish between assets, whether 3d can't always keep up with the pixel art, or the pixel art can't always keep up with 3d, in the total impression of the scene. It might well be somewhat jarring aesthetically, similar to mixed resolutions, compared to a clean craft in acceptance of its strengths and limitations. It's the little things. And the attempt of figuring it all appropriately, might turn out its own little rabbit hole of hidden costs in development you dig yourself in, in ways you didn't expect, compared to a simply more modest approach to the problems of pixel art. And new possibilities not only can save work, they can also create more work with your rising expectations of blockbuster development in an almost vicious cycle of what more you want to achieve, or need to achieve to maintain balance of qualities between things. Meanwhile, a pixel art project very much conscious of its limitations, is planned such to live well with it on an appropriate level of needs, so it isn't actually in conflict with itself, but a well rolling development. That's the real question everyone has to answer themselves, what's the ceiling of their production at the end of the day.

I think though, that it is good to have creative options to try out for various problems of development. You never know. The more the merrier. So it's good Howard keeps investigating this direction, someone should, try and find out the merits of it, it's the time for it. It's kinda amusing that I, of all people, would assume a sort of "warning voice of nay-saying" here, if you want to read it like that, considering I'm usually more on the front of a liberal and practical approach to things. I felt though someone should assume that role still, if not you, as you usually would, then I as the most unlikely person for that, since all the critique I have given here, is also a critique on myself, you see. I'm starting to feel somewhat schizophrenic now.

[Conzeit]

Conceit: Jesus, sorry.  I try and answer with all the information!

ok I am now confused as hell. What on earth r u apologizing for? XD are you using some weird internet intelligible sarcasm or did I somehow say exactly the oppposite of what I meant?...or was that meant for RAV or smth?

in any case just to make sure

I THINK THIS IS  GREAT THING AND I AM GLAD YOU MADE IT HAPPEN, PLEASE KEEP MAKING IT HAPPEN! XD

EDIT:

Thanks! On the "too realistic" front, I'm curious precisely what you mean. Texture? Details? Clean lines?

I think most of all it means the calculated correctness of whatever it applies universally. Be it perspectives, shadows, shading, whatever not done manually. it is all physically correct or solves numeric otherwise, even when it is "toonish". Since this kind of correctness in everything across all frames this smooth is not tradition to pixel art, it can look uncanny.

That reminds me of that article explaining GGXrd's art style, Conceit posted it awhile back in the OT thread IIRC.

yup. (btw Ai nice link Imma watch it)

here's what I had posted before
the english transcription of the GGXRD behind the scenes interview thingy.
http://www.polycount.com/forum/showpost.php?p=2099538&postcount=229
http://www.polycount.com/forum/showpost.php?p=2107579&postcount=241

original article
http://www.4gamer.net/games/216/G021678/20140703095/
http://www.4gamer.net/games/216/G021678/20140703095/index_2.html
(I think I might reconstruct the original article+images...wanna get into lowpoly and trying to replicate this would be good motivation)

Xrd pulled perfect anime aesthethic in realitme, and I know Cel shading is not pixels, but anime and pixels share the emphasis on that "manual" aspect.

There's a bazillion interesting tricks going on so read the whole thing Howard it'll give you lots of ideas, but in regards to adding flaws intentionally  look at this correction in XRD

the bone themselves have a higher degree of freedom than usual, so the characters can fake 2D flaws. For example facial features can move around the face to better match drawings (Bedman in his instant kill cinematic as he'd appear without
deformation
,
the suggested 2D corrections

and the final result


SNIP!

Characters also needed to fake or exaggerate perspective in ways changing the camera's field of view couldn't achieve so this was done through allowing bone scaling on all three axes. UE3's animation system can't do that out of the box, they had to code it themselves and cite it as a great advantage to having access to the source code. And once they had that in, that meant they'd also opened the door to all kinds of muscle motions, cartoony squash'n'stretch animations and deformations. A big punch will get a big fist and so on. Bone scaling is now a feature in UE4 and they like to think it's because of Xrd.

May's victory pose


the actual deformation


SNIP

Sol in Softimage with his bone scale settings

I know this kind of stuff is like the opposite of a cheat, tweaking everything to fit one speficic occasion and never use it again, it's almost anti productivity anti fast workflow =/ but some other techniques do allow for artistic direction without constant tweaking.

I personally dont get anything out of of the bone scalings image of sol but you might

This trick with lighting is a good example of increased artistic direction without constant tweaking

R channel of the vertex color is a shadow bias, ie it's like ambient occlusion except artist-directed. Pixels that have a lower value will be more easily in shadow
(
occlusion term,




model without it,




model with it).

They do something crazy with each one of those channels which I dont really know what they're for. There was a whole topic talking about this stuff where people broke it down for me but I cant seem to find it anymore ( if any mod reads this, did it get wiped off automatically or something?)

They have a separate one for highlights as well. imagine if each shade could have it's own profile for "shadow bias", you could pull off almost any aesthetic

Specular highlights are controlled by the R and B channels of the ILM texture. R is specular intensity, B is specular size.

See this close-up of I-no where the left side and right side of her top have different specular size

They do a lot of other cool stuff. off the top of my head
-they swap the face models to have biger chibified features when it zooms away, much like what we do when we are making small characters and we make their faces more simplified with bigger face features
-for certain effects like dust and millia's hair shapes or exagerated takes they have separate flippen models for each  goddamn frame.
-using normals from different objects to simplify certain parts of the model, so that what needed to have clean geometric shapes didnt get cluttered with useless information
-another channel that indicated the line thickness of the outlines
-another channel that tweaks how much the lines should scale as distance changes
-another channel that hides certain lines when they clutter up the image too much.

this is all stuff you could use =)

[Howard Day]

32: Thank you! Can you give me an example of something you'd like me to ape? I also just fixed the dither - it wasn't being calculated properly, and as an end result was...Overzealous.

RAV: "There are little to no margins for mistakes in real pixel art." That is...not actually a viewpoint I hold with. At all. In fact, I have no idea how you'd ever quantify that! What's a "mistake", and what's a thinking artistic choice? Dithering is a choice - how much it's dithered is a choice - color ramp hue shifting is a choice...in fact, I'd be shocked if there's pixel art anywhere that would completely fall under the "no mistakes" label. Everything someone does as an artistic choice may be a different choice than another artist would make. 
As for models being too detailed for their final use, well - that's just something people making the artwork will need to consider. same as anything - you're not going to pixel a sprite at 32x32, then later decide to make it 48x48.. and just scale it up expecting things to look right.

Gil: Heh, you might be surprised. I'm actually exerting a lot of control on everything visible. The only thing that falls outside of my direct influence is the edges of shadows. Those are...flickery, and there's not much I can do to fix it. Thanks, shadow maps!

Ai: Ahh, yes. I'm very familiar with the GGXrd's art style and methodology. I'm actually directly using their Vertex AO > driving lighting ramps. Its a great idea, and I'm a huge fan of everything they did. I'm glad you like the improvements - I'm still pushing it more! Anything you'd like to see?

Conceit: I'm being incredibly sarcastic. I knew what you meant, and I was just trying to be humorous. I'm already doing a ton of the stuff they're talking about in the GGXrd technical papers - but I'm doing them with mobile devices and sheer speed in mind. Things like the specular levels and material presets - all of that's done with the ramps. Need a large specular hit? Push the lightest bar in on the side. It really is quite versatile.