Pixelation

Critique => 2D & 3D => Topic started by: Howard Day on September 18, 2015, 07:48:41 pm

Title: Realtime Rendering of 3d Meshes as Pixelart
Post by: Howard Day on September 18, 2015, 07:48:41 pm
Working on custom shaders and rendering in unity to make 3d objects render like classic pixel art. Some success so far! The web build is here: http://www.hedfiles.net/PixelShader/PixelShader.html (http://www.hedfiles.net/PixelShader/PixelShader.html)

(http://www.hedfiles.net/PixelShader/pixart_01.png)
(http://www.hedfiles.net/PixelShader/pixart_02.png)
(http://www.hedfiles.net/PixelShader/pixart_03.gif)
(http://www.hedfiles.net/PixelShader/pixart_05.gif)
(http://www.hedfiles.net/PixelShader/pixart_06.gif)
(http://www.hedfiles.net/PixelShader/pixart_07.gif)

I'm going to put together a tutorial and release the source when it's closer to done-ish.
Enjoy!
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Gil on September 18, 2015, 08:03:48 pm
Oh hell yes. I was looking for something like this a while back. Combined with normal mapped pixel-art, we should be able to pull off some great stuff. I should get back into 3D.

Seems like there's still a fair amount of pixel noise. How much leeway is there left to eliminate that in a real-time situation? I imagine there must be a way to put basic cluster theory into a shader to get it just a bit better, but I can't imagine putting that into a pixel shader.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Howard Day on September 18, 2015, 08:25:28 pm
Well, you can increase the AA quality. 2X AA was simply done for speed on mobile devices (though, honestly shadows are the slowest part right now).
Here it is with 8X AA:
(http://www.hedfiles.net/pixart_04.png)
It's pretty smooth.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Conzeit on September 18, 2015, 09:31:23 pm
 :o Howard you did it again. that's pretty effn good. are you trying to Force me to learn 3D?  >:( :P

it's looking pretty nice so far. I saw something like this on tweeeeeter a while ago but it didnt look nearly as good, that example pretty much looked just like low res cel-shading, are you doing anything new here that you hadnt explained in that Irkalla mech topic?

Also, one of the most appealing things of pixelart is controling shade by shade how the hue and saturation changes with the luminosity ("hueshifting") and I do see that the ramps on these do not stay on the same hue or saturation, but they seem to bounce around without much reason. Are you tweaking the values of the ramps by choosing them manually or is it some other more automated way?
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: lachrymose on September 18, 2015, 09:56:55 pm
Pretty sweet. I know of a japanese program that does something similar. Doesn't work with ATI cards though.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Howard Day on September 18, 2015, 10:15:41 pm
Conceit: Thanks! So, I am controlling the ramps directly. Here's the texture.
(http://www.hedfiles.net/PixelShader/MultiRamp_Test.png)
The way the shader works is that it only lets you set the vertical (v) position of the UV map - the horizontal position is skewed side to side based on the angle of the surface to the scene light, the in/out shadow value, and vertex ambient occlusion.
You can see in that map that there's some hueshifting going on - not a huge amount, but some.
If you're looking at the build (or those images I posted) the palette being used isn't exactly accurate. I've added options to palettize to classic videogames. The examples I posted are being indexed to Rise of The Triad, Arne's 64 color palette, and Duke Nukem 3d. The only one using the actual palette I made for this project is the image on the light background with no AA.
http://www.hedfiles.net/PixelShader/pixart_01.png
The difference from the IRKALLA mechs is that those were pre-rendered in Max, using a procedural max material. This is 100% realtime, designed for use in games.

lachrymose: Thanks! This actually works on everything, including phones and tablets.


Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Conzeit on September 19, 2015, 02:35:46 am
what a fucking answer  ;D Thanks for explaining it to the 3D illiterate layman. godspeed! :y:
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: RAV on September 19, 2015, 03:09:06 am
Good work, Howard. This is the best attempt I've seen of trying to turn pixel art into a mere shader-option of standard 3d. It looks awesome.
Scrutinizing every frame of the .gif, surely a pixel artist will find plenty flaws in the pixel level rendering, after all pixel art is very much about the details of artistic rendering.
However, upwards HD resolutions exact pixelation matters less and less to the commoner's eye, and what remains as obvious hallmark to identify it as pixel art is the controlled palette.

We have this ideological spectrum of how to make pixel art transit into the next dimension, with two ends in seeming opposition:

Focusing on the result or focusing on the process. You have made a very strong and appealing case for the result, to make Pixel Art a sort of aesthetic choice in a different artform.
Taking advantage of industry standard 3d mesh modelling especially for animation and dynamic scenes can have astounding effects.

My own work is all about the process: preserving, refining and focusing the traditional method of pixel art on the transit.
This is so important to me that I might even be willing to skip on a result output that would try to mimic as close as possible true classic retro titles in a literal sense.
That is because I am most interested in finding ways and cases the pixel art method of creation is not a cumbersome hurdle we would want to skip on,
but its own creative advantages we can have productive interest in.

However, for the assets you showcase here as an example, I see your side in greatest favour. Good job on finding ways to keep your end of the spectrum so interesting.

Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Seiseki on September 19, 2015, 03:39:19 pm
This looks so damn cool!  :'(

edit: I wonder if it's too realistic for hand drawn pixel art, I'm curious how it would look without shadows and flatter shading.

what a fucking answer  ;D Thanks for explaining it to the 3D illiterate layman. godspeed! :y:

Go download Blender (https://www.blender.org/download/), watch some tutorials (https://www.youtube.com/watch?v=zOvawDOWqC4) and learn 3D already! :P
Seriously though, learning 3D was the best thing I ever did, second only to signing up to this forum..
I always imagined it would be hard, but it's really not and there are tutorials for everything on youtube.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Joe on September 19, 2015, 03:50:08 pm
Yea, you just created the future. Brilliant.  :y:
Another application I see for this is as a reference for manual work, showing oneself how it would most realistically look with a given palette. Cause this is about as realistic as it can get.

I too am interested to see how this would be refined to emulate all the pixel-level manipulations on the fly. I don't see why it couldn't be done.
Can't wait for the day I have time to finally dive into 3D. A year or so perhaps.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: PixelPiledriver on September 20, 2015, 04:58:44 am
Gratz, I love it.  :)
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Howard Day on September 20, 2015, 08:50:00 am
Conceit: Jesus, sorry.  I try and answer with all the information! :P

RAV: Um, wow. Thanks? I'm not sure what exactly your view on this actually is - good or bad - but I can definitely tell you I'm not trying to be the next evolution of pixel art. Pixel art is a well-defined, well-understood genre of artwork, and I really cannot see anything I'm doing ever replacing that. The idea is frankly ludicrous! Everything I've done so far has been as a result of three key things:
A): Can a 3d renderer produce convincing pseudo-pixelized artwork?
B): I love solving technical puzzles!
C): I am incredibly lazy.
(C) is why I rarely do actual, you know, Pixel Art. I can - and I have in the past. I'm a huge fan of pixel art animation. I just know how to do 3d stuff too - and if I can find a lazy shortcut, I will. :)
Finally, "Surely a pixel artist will find plenty flaws in the pixel level rendering" is exactly why I posted this here. I want to know anything that stands out as needing improvement. Anything that looks off. Indigo has been assisting me with that sort of feedback from the beginning (I'm lucky enough to work with the gentleman), but I'd love any insight this wider net catches.

Seiseki: Thanks! On the "too realistic" front, I'm curious precisely what you mean. Texture? Details? Clean lines? All of this can be... tweaked. For instance, here's a shot with some noise introduced and no shadows.
(http://www.hedfiles.net/PixelShader/pixart_08.png)
...and even more noise, to ridiculous levels:
(http://www.hedfiles.net/PixelShader/pixart_09.png)

Joe: Thank you, sir! Though, as I said above, probably not the future, just another lazy shortcut. :D What sort of pixel level manipulations would you be interested in seeing?

PixelPileDriver: Thanks!

Okay, took the opportunity to put together a little kitbashed space fighter from bits. It looks pretty neat! First is in the native, intentional palette, the second is indexed to the Raptor Call of the Shadows palette. :D
(http://www.hedfiles.net/PixelShader/pixart_10.gif)
(http://www.hedfiles.net/PixelShader/pixart_11.gif)
..and a angled verticle SHMUP version!
(http://www.hedfiles.net/PixelShader/pixart_12.gif)

Enjoy!


Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: RAV on September 20, 2015, 10:06:00 am
Don't worry about it, Howard. I think it's pretty cool and a good thing. It was just a general musing on it "philosophically."
I wasn't talking about it in terms of replacing pixel art either, but the two different approaches of how 3d can function and integrate for pixel art.

Quote
I want to know anything that stands out as needing improvement. Anything that looks off.
Baring some minor adjustment, I think you got it pretty much as good as it can get -- which is good enough for your purpose. It looks pretty cool.

If you don't mind me musing some more, about the fundamental problems of rendering to pixel art, which may focus more on what improvement is reasonable to expect and search for:

What you do is realistic rendering of pixel art. This can be pretty convenient to short cut some of the more tedious aspects of pixel art in a certain style.
There are simple basic rules to realistic rendering, which is applied wholesome to the image by the code.
In addition this can be somewhat adapted partially on a more or less random basis.

What gives a piece an impresssion of pixel art most however is, as I called it, artistic rendering.
And that means a conscious aesthetic decision on each pixel, that may derive from different rules, or even spite rules.
This is the heart of the problem. The human sense, reasonable or emotional, of the pixel.

Whatever further correction you attempt to do programmatically, will look okay in some place, and senseless in another, because it doesn't actually understand what it tries to render, and what it tries to convey in that to the audience. The pixel pattern rendering problem is somewhat akin to the "what's the best palette" discussion, even much worse a problem actually.

In addition, this pixel art is produced in a very typical fashion, to be this smooth. Which will inspire pixel artists to create counter-trends of increasingly ideosyncratic and unrealistic rendering techniques that have their own appeal. This means a further divergence in pixel design that increases the differentiation in impression from 3d application, making that look ever so more different from pixel art, uncanny pixel art, and in its own way limited as pixel art, even though paradoxically it was made in an attempt to push certain limits of pixel art in its strife to look like pixel art. That too is part of the art, finding new ways of expression and differentiating itself from other artforms, searching for a recognition, coherence and self-reliance as its own artform. Interesting enough, your success in automating palette application, rather makes me question further if colour limit palette orientation really is at the relevant heart of pixel art, and so as a pixel artist I'd be tempted to depart from that in response, to find my own way, maybe accelerated by how popular your approach turns out to be.

Still, despite these musings, what you do is very much interesting and worthwhile, and pixel artists will find ways to take assisted advantage of it for their own practical purpose in some form or another as suits them. I very much look forward seeing your game looking like this.

Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Gil on September 20, 2015, 10:14:19 am
I think you need to create some scenes where pixel art is combined with this. I don't see a huge value in making an entire 3D game like this, the value is in making a pixel art game, and having certain parts be 3D I think. Then the question is if you can make your shader fit that particular game, to a point that it doesn't break the immersion.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Ai on September 20, 2015, 11:48:11 am
I'm sort of on the same page with Gil, I guess.. I think that your algorithm performs well for these mecha / space fighters, especially considering it's realtime, but we won't really have an idea whether it performs well in general unless we can see it in action on a range of stuff -- landscape/tiles, less realistic people; interiors; much more toony exaggerated color ramps;even bigger (in #pixels of area terms) objects and significantly smaller objects. Things that live in very different design spaces. Some of that might seem perverse given what you are trying to do here, but IMO it's needed as a basis for comparison, to get a realistic picture of its overall performance.

I guess the overall message there is, hurry up and release it, and we can get to trying it out on that different stuff;)

...

The only crit I would make of your most recent renders is that the guns seem a bit soft. Since I'm familiar with 3d I'm aware this can arise when you increase AA levels, but maybe there is some way to counteract that selectively so you can make the important parts as sharp as they ought to be?

If I think more idealistically I would say that there are some things that you just don't want to be AAed at all. If you look at Metal Slug, for example, their strategic use of hard edges contributes a lot to the readability of the art, which IMO should be a primary concern of any game developer.

Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: lachrymose on September 20, 2015, 12:05:35 pm
I think you need to create some scenes where pixel art is combined with this. I don't see a huge value in making an entire 3D game like this, the value is in making a pixel art game, and having certain parts be 3D I think. Then the question is if you can make your shader fit that particular game, to a point that it doesn't break the immersion.

I see the value of it in decreasing work; for large sprites or for things that require a lot of sprite animation. You could render out/animate a 3D image, use the pixel shader on it.
Then take from that the frames you need for pixel animation, afterwards its just a matter of going though and cleaning up...Certainly sounds faster then starting each frame from scratch like normal. Especially for something like 8-directional movement sprites, or sprites for fighting games.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: RAV on September 20, 2015, 12:59:48 pm
Quote from: AI
If I think more idealistically I would say that there are some things that you just don't want to be AAed at all. If you look at Metal Slug, for example, their strategic use of hard edges contributes a lot to the readability of the art, which IMO should be a primary concern of any game developer.
If we keep looking at it as a realtime application, that's another conundrum we have: pixel art can afford to be very sharp, because it doesn't have to hide mistakes. And thus sharpness is a common trait of pixel art. The more we try to just sweepingly blur the errors out here, it will tend to look less convincing as pixel art and more like just another form of cellshading, as well as make a carefully managed colour palette less and less meaningful in the end. A very narrow line to walk in optimizing it. So that's something to keep in mind.

It also highlights something I've pointed to earlier: what does the selectivity of "there is some things you just don't want to" mean programmatically in the appiication of a technique? This is the key problem we're facing. not just from asset to asset, but even within an asset. Learning the good application of a technique is worth years of struggle for any pixel artist. Reading up the principle of AA is a long shot away from using it well to best effect, refined and selectively. I think this is pixel art at its heart, the strategy of techniques, use with care, from case to case. And I think this is very much what makes most the impression of looking pixel art.


Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: RAV on September 20, 2015, 03:53:36 pm
Quote from: Howard Day
Thanks! On the "too realistic" front, I'm curious precisely what you mean. Texture? Details? Clean lines?
I think most of all it means the calculated correctness of whatever it applies universally. Be it perspectives, shadows, shading, whatever not done manually. it is all physically correct or solves numeric otherwise, even when it is "toonish". Since this kind of correctness in everything across all frames this smooth is not tradition to pixel art, it can look uncanny.

When you look at a lot of pixel art, they often are in some way or another decidedly "incorrect". Even a complex mash of whatever seems right.
But the point is, it is not unreasonably incorrect. it's not a matter of noise. The artist tries to achieve something specific that principle correctness doesn't help with.

They do this for various reasons, for example readability. A correct perspective is not always the most advantageous and satisfying in every situation.
And an "incorrect" decision in one part then affects decisions in other parts to suit that, like shading, etc. A whole chain of decisions that make no physical sense, yet are not random, but right.

You can for example try adjust and combine various projection settings across the scene, but there will be difficulties, since it doesn't really understand what it's doing, and it's just the tip of the iceberg of what you need doing, depending on how far you can and want take it.

It's really the exceptionality of pixel art in every other detail that is so hard to capture on whatever aspect you automate for dynamism.


I hope though, you don't read all this as if I were just trying to throw sticks on your legs. Keep up the good work and find ways to make it work, okay. Try take advantage of your methods and make the best of it. I think it's plenty cool as is, I see sense in it. I have my own stake on the topic, had a lot of thought about it, so I'm kinda passionate and talky about it. Not sure that helps though.

Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Cherno on September 20, 2015, 06:44:59 pm
That's a great shader, I love it. I can see it be a moderate success on the Unity marketplace (moderate because it does caters to a niche, after all). On the other hand, I wouldn't mind getting my hands on this shader for free *whistles* ;)
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Howard Day on September 21, 2015, 08:36:38 am
 RAV: Hmmm. Lots to think on there.

Gil: Yes. I indeed want to try that next. Something along these lines: http://41.media.tumblr.com/963cf1361b3d0290e03e876da0efcd1f/tumblr_nj7rg591yt1u54l7wo1_1280.jpg
(Concept by Nurzhan Bekkaliev!)

Ai: Yeah the blurry AA was really, really bothering me. I've since fixed it!

lachrymose: That's also a thought. :D

Cherno: Well, it's going to be released for free. :D


Okay. In new news... I've been an absolute idiot. The Indexing shader that I wrote wasn't quite doing what I thought it was. As a result...the indexed colors were being smoothly interpolated. This is, safe to say, not the intended behavior. I've since fixed it, and now, the indexed results are absolutely perfect. Kills me that I missed something this stupid. Ugh. UGH.
Here's a look at the new output:
(http://www.hedfiles.net/PixelShader/pixart_15.png)
(http://www.hedfiles.net/PixelShader/pixart_16.png)

So that's settled, and a weight off my mind. The next thing on my ToFIX list was the AA. It was very blurry - and just..looked artificial, and not pixeled at all. So, I figured out a method around that. The Scene is rendered with AA at 3x the final screen resolution.. then Nearest Neighbor scaled to the final desired resolution. Here's the image with no AA:
(http://www.hedfiles.net/PixelShader/pixart_15.gif)
And with the new AA!
(http://www.hedfiles.net/PixelShader/pixart_16.gif)
I'm stoked. It looks much improved to me!
Here are some more shots - pay attention to the LUT cube in the upper left. :D
(http://www.hedfiles.net/PixelShader/pixart_18.png)
(http://www.hedfiles.net/PixelShader/pixart_19.png)
(http://www.hedfiles.net/PixelShader/pixart_21.png)
(http://www.hedfiles.net/PixelShader/pixart_23.png)

I'd love to hear any and all feed back and thoughts you guys have.

Enjoy!
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: RAV on September 21, 2015, 09:26:50 am
Looks nice. but I also think you traded problems here. not to bum you out with being nit-picky, but consider this going further:

If pixel art is not pixel perfect it just doesn't work. To me it seems, the less it looked like pixel art, the better it looked overall.
The more it restricts itself now, the more glaring the issues, which make looking any more like pixel art not necessarily more desirable.

When we look at the gun tips of the rotating white gunship for example, on each rotation it's a wiggly waggly mess of clusters that are barely recognizable as barrels. Keep in mind, the exact perspectives and angles of lines in pixel art are not by chance. Pixel artists know which ones make for good clean art and which ones make it too tough to get right in a prominent grid aesthetic restricting it. The freedom of 3d now breaks all that, and with it the pixel art breaks on its usual breaking points, all over the place, randomly.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Gil on September 21, 2015, 10:05:49 am
Daaamn, that's a massive improvement. I'm starting to get really excited :)

RAV: I don't think you can judge these by themselves, as pixel art.

1) They are not and not meant to be pixel art
2) You'd probably use this shader in a scenario where it makes sense, like combining it with pixel art for certain characters that would otherwise prove impossible to animate

For that second point, I'm wondering if the shader performs better than usual at portraying small objects.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Howard Day on September 21, 2015, 10:39:01 am
Rav: That's an issue with the model, not the shader. Any attempt to fit more shape information into a subpixel area would end up messy, no matter the medium. Here's a cleaned up version:
(http://www.hedfiles.net/PixelShader/pixart_27.png)
The shader and procedure have improved. The model just needed to improve a bit with it. :P

Gil: Thanks! They are meant to emulate pixel art - and I'm game to keep pushing till it's looking even better. :)
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: 32 on September 21, 2015, 10:52:31 am
This is absolutely lovely  ;D I've often wanted to see this done but never really expected to. I think this definitely gives a good enough impression that you could use it alongside hand made pixels for more complicated stuff. Though the character also look pretty cool. I'd love to see how this looks on more traditionally pixel art styled and sized characters.

I think you should leave the dither off. Looks nice on some of the smaller panels on the walker but the big solid areas of dither and dither on the outlines is nothing a person would do. Maybe that's something you can fix but dither is pretty out of fashion anyway.

Can't wait to see where this goes :D
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: RAV on September 21, 2015, 10:55:54 am
Quote from: Gil
: I don't think you can judge these by themselves, as pixel art.

1) They are not and not meant to be pixel art
2) You'd probably use this shader in a scenario where it makes sense, like combining it with pixel art for certain characters that would otherwise prove impossible to animate

What I point out also are aesthetic problems regardless of what art it is. Yet the topic is "3d meshes as pixel art", and Howard stated specifically it is his interest to find out how close he can get to look like it, and asked for what problems are in his way. Actually, trying to integrate it with other real pixel art makes judging and discussing the problems as pixel art even more important, since you would want to avoid glaring aesthetic breaks between the assets sitting next to each other. Maybe I come across too critical though. And after all, you are an actual pixel artist on professional level, so I'd say Howard should rather listen to your opinion on what you like about it.


Quote from: Howard
That's an issue with the model, not the shader. Any attempt to fit more shape information into a subpixel area would end up messy, no matter the medium. Here's a cleaned up version:
It is an issue with computed rendering at large. And that's what I'm pointing to: it isn't an issue in pixel art, because the pixel artist knows and controls exactly the conditions of display that don't change, and tailors the art perfectly to fit that, understanding what detail fits what degree, distance and sub-pixeling, for example.

The more flexible you make the conditions of display, the more the problems in the rendering as pixel art. There are little to no margins for mistakes in real pixel art. The adaptions you made to the model are mostly good for the conditions you adapted them for. Change distance, and they come back the same, among the issues it still has either way compared next to manually polished pieces. Curiously though, micro-managing models like that to fixed angles and distances, starts to eat up some of the advantages of going 3d.

There are still some good reasons one might want to opt for that regardless though, and you can search for recommended sweet-spots of modelling to alleviate some of the problems in some range and degree. You can try cheat a bit the pixels, and a sort of predefined Level of Detail system might help too in extremer cases of use. Just keep an eye on what expenses are worth what result.


Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Gil on September 21, 2015, 02:08:13 pm
What I point out also are aesthetic problems regardless of what art it is. Yet the topic is "3d meshes as pixel art", and Howard stated specifically it is his interest to find out how close he can get to look like it, and asked for what problems are in his way. Actually, trying to integrate it with other real pixel art makes judging and discussing the problems as pixel art even more important, since you would want to avoid glaring aesthetic breaks between the assets sitting next to each other. Maybe I come across too critical though. And after all, you are an actual pixel artist on professional level, so I'd say Howard should rather listen to your opinion on what you like about it.
Oh, I would hardly value my opinion above anyone else on this forum. I do get your point. I think the point is this: I think he can still get a fair bit closer to pixel art, but he'll never get close enough to be able to call it pixel art. The simplest definition of pixel art is art where you have control over individual pixels. I don't feel like any of the renders have enough control to call it pixel-level control. I do think you can get away with combining stuff like this with pixel art though, and that would be my aim if I were to make something like this. Also, the fact that you can make procedural animations is HUGE. The only way to have procedural pixel animation is by using a combination of lots of keyframes and paper-dolling, which doesn't provide you with even a sliver of what's possible with 3D models.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Ai on September 21, 2015, 02:15:42 pm

The more flexible you make the conditions of display, the more the problems in the rendering as pixel art. There are little to no margins for mistakes in real pixel art. The adaptions you made to the model are mostly good for the conditions you adapted them for. Change distance, and they come back the same, among the issues it still has either way compared next to manually polished pieces. Curiously though, micro-managing models like that to fixed angles and distances, starts to eat up some of the advantages of going 3d.

That reminds me of that article explaining GGXrd's art style, Conceit posted it awhile back in the OT thread IIRC.
this (http://www.gdcvault.com/play/1022031/GuiltyGearXrd-s-Art-Style-The) is a related link, but not it. Anyway they were doing just that kind of stuff -- they had a bunch of layers of stuff (textures, normals, etc) to control all that stuff, and even loads of kinematics just for distorting the model to look maximally awesome in each frame. Getting kind of crazy..

Howard:


That is so much better it's ridiculous. I think when you release it, if it's popular you'll probably find the community arrives at the idea that there is a kind of .. different way of designing your model for each broad level of scale (so that it reads well, that is.). But even so, getting the flexibility of 3d with that limitation, still has a huge scope for time saving and exploring options that otherwise couldn't be tried.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: RAV on September 21, 2015, 02:44:14 pm
I agree to that. Yet keep in mind to be honest in your judgement of use when the time comes, for all the details of the issue I pointed out before for this reason. Especially when it comes to integrating these two arts, the power you feel in the possibilities might blind you for the aesthetic overall costs your project could suffer for it, or that it could lock you down visually as well in other ways. Whether it is a difference in polish between assets, whether 3d can't always keep up with the pixel art, or the pixel art can't always keep up with 3d, in the total impression of the scene. It might well be somewhat jarring aesthetically, similar to mixed resolutions, compared to a clean craft in acceptance of its strengths and limitations. It's the little things. And the attempt of figuring it all appropriately, might turn out its own little rabbit hole of hidden costs in development you dig yourself in, in ways you didn't expect, compared to a simply more modest approach to the problems of pixel art. And new possibilities not only can save work, they can also create more work with your rising expectations of blockbuster development in an almost vicious cycle of what more you want to achieve, or need to achieve to maintain balance of qualities between things. Meanwhile, a pixel art project very much conscious of its limitations, is planned such to live well with it on an appropriate level of needs, so it isn't actually in conflict with itself, but a well rolling development. That's the real question everyone has to answer themselves, what's the ceiling of their production at the end of the day.

I think though, that it is good to have creative options to try out for various problems of development. You never know. The more the merrier. So it's good Howard keeps investigating this direction, someone should, try and find out the merits of it, it's the time for it. It's kinda amusing that I, of all people, would assume a sort of "warning voice of nay-saying" here, if you want to read it like that, considering I'm usually more on the front of a liberal and practical approach to things. I felt though someone should assume that role still, if not you, as you usually would, then I as the most unlikely person for that, since all the critique I have given here, is also a critique on myself, you see. I'm starting to feel somewhat schizophrenic now.

Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Conzeit on September 21, 2015, 03:38:43 pm
Conceit: Jesus, sorry.  I try and answer with all the information! :P

ok I am now confused as hell. What on earth r u apologizing for? XD are you using some weird internet intelligible sarcasm or did I somehow say exactly the oppposite of what I meant?...or was that meant for RAV or smth?

in any case just to make sure

I THINK THIS IS  GREAT THING AND I AM GLAD YOU MADE IT HAPPEN, PLEASE KEEP MAKING IT HAPPEN! XD

EDIT:
Quote from: Howard Day
Thanks! On the "too realistic" front, I'm curious precisely what you mean. Texture? Details? Clean lines?
I think most of all it means the calculated correctness of whatever it applies universally. Be it perspectives, shadows, shading, whatever not done manually. it is all physically correct or solves numeric otherwise, even when it is "toonish". Since this kind of correctness in everything across all frames this smooth is not tradition to pixel art, it can look uncanny.


That reminds me of that article explaining GGXrd's art style, Conceit posted it awhile back in the OT thread IIRC.
yup. (btw Ai nice link Imma watch it)

here's what I had posted before
the english transcription of the GGXRD behind the scenes interview thingy.
http://www.polycount.com/forum/showpost.php?p=2099538&postcount=229
http://www.polycount.com/forum/showpost.php?p=2107579&postcount=241

original article
http://www.4gamer.net/games/216/G021678/20140703095/
http://www.4gamer.net/games/216/G021678/20140703095/index_2.html
(I think I might reconstruct the original article+images...wanna get into lowpoly and trying to replicate this would be good motivation)

Xrd pulled perfect anime aesthethic in realitme, and I know Cel shading is not pixels, but anime and pixels share the emphasis on that "manual" aspect.

There's a bazillion interesting tricks going on so read the whole thing Howard it'll give you lots of ideas, but in regards to adding flaws intentionally  look at this correction in XRD

Quote from: GuiltyGearXrd
the bone themselves have a higher degree of freedom than usual, so the characters can fake 2D flaws. For example facial features can move around the face to better match drawings (Bedman in his instant kill cinematic as he'd appear without
deformation
(https://i.imgur.com/0Le6f8z.jpg),
the suggested 2D corrections
(https://i.imgur.com/3AlH2Ox.jpg)
and the final result
(https://i.imgur.com/5t0ujza.jpg)

SNIP!

Characters also needed to fake or exaggerate perspective in ways changing the camera's field of view couldn't achieve so this was done through allowing bone scaling on all three axes. UE3's animation system can't do that out of the box, they had to code it themselves and cite it as a great advantage to having access to the source code. And once they had that in, that meant they'd also opened the door to all kinds of muscle motions, cartoony squash'n'stretch animations and deformations. A big punch will get a big fist and so on. Bone scaling is now a feature in UE4 and they like to think it's because of Xrd.

May's victory pose
(https://i.imgur.com/syQBkeG.jpg)

the actual deformation
(https://i.imgur.com/4aemscg.jpg)

SNIP

Sol in Softimage with his bone scale settings
(https://i.imgur.com/nhjo2Ev.jpg)

I know this kind of stuff is like the opposite of a cheat, tweaking everything to fit one speficic occasion and never use it again, it's almost anti productivity anti fast workflow =/ but some other techniques do allow for artistic direction without constant tweaking.

I personally dont get anything out of of the bone scalings image of sol but you might

This trick with lighting is a good example of increased artistic direction without constant tweaking
Quote from: GuiltyGearXrd
R channel of the vertex color is a shadow bias, ie it's like ambient occlusion except artist-directed. Pixels that have a lower value will be more easily in shadow
((https://i.imgur.com/1PPKLRx.jpg)
occlusion term,



(https://i.imgur.com/tVgRjXC.jpg)
model without it,



(https://i.imgur.com/V80qxnA.jpg)
model with it).
They do something crazy with each one of those channels which I dont really know what they're for. There was a whole topic talking about this stuff where people broke it down for me but I cant seem to find it anymore ( if any mod reads this, did it get wiped off automatically or something?)

They have a separate one for highlights as well. imagine if each shade could have it's own profile for "shadow bias", you could pull off almost any aesthetic

Quote from: GuiltyGearXrd
Specular highlights are controlled by the R and B channels of the ILM texture. R is specular intensity, B is specular size.
(https://i.imgur.com/ibZ397m.jpg)
See this close-up of I-no where the left side and right side of her top have different specular size

They do a lot of other cool stuff. off the top of my head
-they swap the face models to have biger chibified features when it zooms away, much like what we do when we are making small characters and we make their faces more simplified with bigger face features
-for certain effects like dust and millia's hair shapes or exagerated takes they have separate flippen models for each  goddamn frame.
-using normals from different objects to simplify certain parts of the model, so that what needed to have clean geometric shapes didnt get cluttered with useless information
-another channel that indicated the line thickness of the outlines
-another channel that tweaks how much the lines should scale as distance changes
-another channel that hides certain lines when they clutter up the image too much.

this is all stuff you could use =)
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Howard Day on September 21, 2015, 07:35:49 pm
32: Thank you! Can you give me an example of something you'd like me to ape? :D I also just fixed the dither - it wasn't being calculated properly, and as an end result was...Overzealous.

RAV: "There are little to no margins for mistakes in real pixel art." That is...not actually a viewpoint I hold with. At all. In fact, I have no idea how you'd ever quantify that! What's a "mistake", and what's a thinking artistic choice? Dithering is a choice - how much it's dithered is a choice - color ramp hue shifting is a choice...in fact, I'd be shocked if there's pixel art anywhere that would completely fall under the "no mistakes" label. Everything someone does as an artistic choice may be a different choice than another artist would make. 
As for models being too detailed for their final use, well - that's just something people making the artwork will need to consider. same as anything - you're not going to pixel a sprite at 32x32, then later decide to make it 48x48.. and just scale it up expecting things to look right.

Gil: Heh, you might be surprised. I'm actually exerting a lot of control on everything visible. The only thing that falls outside of my direct influence is the edges of shadows. Those are...flickery, and there's not much I can do to fix it. Thanks, shadow maps!

Ai: Ahh, yes. I'm very familiar with the GGXrd's art style and methodology. I'm actually directly using their Vertex AO > driving lighting ramps. Its a great idea, and I'm a huge fan of everything they did. I'm glad you like the improvements - I'm still pushing it more! Anything you'd like to see?

Conceit: I'm being incredibly sarcastic. :P I knew what you meant, and I was just trying to be humorous. I'm already doing a ton of the stuff they're talking about in the GGXrd technical papers - but I'm doing them with mobile devices and sheer speed in mind. Things like the specular levels and material presets - all of that's done with the ramps. Need a large specular hit? Push the lightest bar in on the side. It really is quite versatile. :D
 



Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Ai on September 22, 2015, 01:22:26 am

Ai: Ahh, yes. I'm very familiar with the GGXrd's art style and methodology. I'm actually directly using their Vertex AO > driving lighting ramps. Its a great idea, and I'm a huge fan of everything they did. I'm glad you like the improvements - I'm still pushing it more!
Anything you'd like to see?
In technical terms? No, I don't have enough relevant experience to say.
In artistic terms? Something with very thin  (<=1.5px) features.

Can I digress for a little?
Auto hinting. I've recently been investigating pixel font rendering, and found that the autohinting that OTF fonts are normally treated with, totally curbstomps the manual hinting that TTFs have*. There is some loss of identity, especially at smaller sizes,

*using freetype2-infinality, which is a tweaked freetype2 supporting things options like stem->pixel snapping, etc.

I brought that up by way of saying: 'autohinting' of 3d models for the purposes we're discussing here. Has such a thing been tried before? Is it feasible?
If it is, it seems like it would go some way to mitigating the problem of making a model for 64x64px and then finding yourself in a situation where you need a 24x24 render of it that still reads.

(which ties back to 'very thin features', of course.)

Quote
Conceit: I'm being incredibly sarcastic. :P I knew what you meant, and I was just trying to be humorous. I'm already doing a ton of the stuff they're talking about in the GGXrd technical papers - but I'm doing them with mobile devices and sheer speed in mind. Things like the specular levels and material presets - all of that's done with the ramps. Need a large specular hit? Push the lightest bar in on the side. It really is quite versatile. :D
Ohh, dynamic gradient mapping. Now this is more familiar territory. I guess that means that multiple lightsources are a pain to do. I worked out an option for that previously which basically adds a blending factor between two or more ramps (with the caveat that the ramps are ordered and you can only blend between ramps with adjacent indices). Perhaps that's an idea that might be usable here.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Conzeit on September 22, 2015, 02:02:12 am
oh, internet sarcasm, you foiled me again!   *angrily shakes fist*

alright! I just really like the idea of the drawn in shadow bias. I get that may not be practical for your personal uses, but do you think it would be hard to implement? when I get into ( read: if I ever do get around to...:p) messing with lowpoly and try to recreate this stuff, I'd love to do shadow bias, possibly the scaling bones and maps the maps that indicate how to deform the model as the camera moves. do you think it would be hard to implement?

I think if you're familiar with the Xrd stuff, you probably know about this as well, but I just love their style so...
https://vimeo.com/22844101 rigging of Meindbender's Duplicator
https://vimeo.com/7980975   duplicator


as for crits, I think everything is mostly working fine, very stylistically consistent...the one thing that does really bug me is the way the fire's glow is blending on everything else, it's very noisy and messy and it clashes with the rest. if you could somehow handle it by changing the ramps that the lights shine on it would ring a lot more true to pixelart, as that is often how we handle glows.

(https://imgur.com/I5Cy6zQ.gif) this is in concept not that different from what you're doing, but notice the well defined and far less noisy edges, and notice how when it goes over him it just switches his body to one brighter purple ramp. You could even just keep blending but try to make it less noisy, ideally draw it by hand. Could possibly a billboard you know?
(https://i.imgur.com/s54zVK5.gif)
(https://i.imgur.com/KMzzk9f.gif)

(https://i.imgur.com/KmB4hqv.gif) brighter ramp+a little gradient...could work
(https://imgur.com/qRBIntM.gif)kinda hybrid aproach
(https://i.imgur.com/Ni2FSD4.gif) a different lightsource, backlit maybe or even just a black sillouethe for the part being shun on could look awesome
(https://imgur.com/AO8pA7U.gif) sometimes fire with dark edges that doesnt even blend
(https://i.imgur.com/viweim5.gif) or just fire with flashing dark frames can look great
(https://i.imgur.com/vmaDJql.gif) love how lit up her legs are but they dont completely lose definition

oh man, you have to do the explosions like with the ape thing you did before, AND have them fade by generating smaller and smaller ones ad infinitum to give it a real Metal Slug feel. please? :p

Also, dunno how you're doing the windows but if you were to do it like drift stage...damn.
(https://i.imgur.com/rK4t91J.gif) the way you do it where the brightness changes with the angle is great, but I really like the way the lines here are not fixed to the ship but to the screen, makes em feel tied to lightsource. Maybe could work for mapping specular intensity as well? I'd try to do as many versions of fake lighting as I could if I were you, gives a far more 2D feel. Fake second lightsource by way of outlining one side of the model with a bright line for example?

I'm not sure what to say about the models...I avoid AA so I have nothing to say there, and I'd focus more on more general feeling stuff than pixel perfect stuff. I'd add another dark shade to the woman's skin...she doesnt feel...fleshy :p maybe a reddish one. I would also probably exagerate her face features, face feels very very blank right now. Either go total blank like Flashback or do a small pixelly face
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: RAV on September 22, 2015, 04:42:16 am
Quote from: Howard
"There are little to no margins for mistakes in real pixel art." That is...not actually a viewpoint I hold with. At all. In fact, I have no idea how you'd ever quantify that! What's a "mistake", and what's a thinking artistic choice? Dithering is a choice - how much it's dithered is a choice - color ramp hue shifting is a choice...in fact, I'd be shocked if there's pixel art anywhere that would completely fall under the "no mistakes" label. Everything someone does as an artistic choice may be a different choice than another artist would make.
I don't like arguing it too much, but having a better understanding of the problems you're dealing with is important for any success in finding further improvements to the shader. So this is not to stop you, but you need to be aware of these things for your work. That's how Pixelation serves you best in your quest.

While there is some truth to what you say, if "anything goes" were truly the case, there were no need for a critique section on this board. Art is no nilly-willy subject. It's serious as science.

There are clear mistakes in pixel art, as wrongly set pixels can make the object less readable as well as less satisfying. Not all of this the model can solve, as it is the rendering that decides the pixels in the end, and in the wrong conditions gives a less readable and satisfying depiction of the model. So as we found out now, the right conditions are important to consider in making the model. That is why I pointed it out. You now have a clearer idea of use case for it. That might have been forgotten over the awesome power 3d promises otherwise.

Pixel art rendering has so much less margins for mistakes compared to regular 3d, because each pixel is given a more prominent role in visual importance, and thus a wrongly set pixel disturbs more obviously the image. When you had a bit more interpolation earlier, as 3d usually does, it stabilized the picture quality much more in its errors, which is not just a matter of taste. I take a less truthful pixel art that's easier on the eyes, over a "real" one that hurts the eyes, because it's not made well enough given a situation. That is the very reason 3d went away from being pixel art a couple decades ago for FPS, since the drastic scale changes inherent would make for a disturbing image quality. So that's another case to consider in how to use it. It could also be useful to have finer option settings for more choice in the degree of how strictly the shader tries to be pixel art. The ability to adjust the degree of interpolation to make it "more or less pixel art" can stretch the use cases. Don't just run after a holy grail of supposedly faithful pixel art, it holds you back in utility as well. Consider that for the problem you had at the beginning: turns out maybe it wasn't a bug but a feature.

Quote from: Howard
Heh, you might be surprised. I'm actually exerting a lot of control on everything visible.
The measure by which to judge your control on the pixels is not the code, but the image we're looking at. It is obvious, not surprising. The shader controls the pixels, not you. And the shader's use of techniques in this control is very simplistic and often less than ideal to a given need compared to how pixel artists use the same techniques. A pixel artist knows how to get the best out of a technique on a given piece. The most important limitation you should consider to work on as was pointed out, is giving the artist more selectivity options with which a certain technique is applied. That is give back at least some control over the pixels to the artist by having some more control options on the shader, in the ability for more selectivity in application of a technique on the model. An on/off toggle for a technique is far away from pixel art control. Finding out which control sub-options on each pixel technique make sense and are useful could be your next best goal in searching improvements from here on out. The level of control you allow the artist to still maintain over the image manually, despite the dynamics, is in the end most what decides how close you can make it useful as pixel art, less how smart you would try to make the shader by itself. It's important to remember that.

Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Seiseki on September 22, 2015, 09:08:15 am
Seiseki: Thanks! On the "too realistic" front, I'm curious precisely what you mean. Texture? Details? Clean lines? All of this can be... tweaked. For instance, here's a shot with some noise introduced and no shadows.

I think it's the opposite.. It's mostly just about how accurate and smooth the rendering is, which is inherent to 3D rendering.

So I'd rather go with less noise, more simplistic and clean lines. Simplified shadows.
I'm curious how it would look with a more limited palette and no dithering.

I think the biggest hurdle for you to overcome, is that it looks sorta like pre-rendered 3D saved in a low res and limited palette format.
As pointed out, the AA is a double edged sword, it hides some of the jarring edges, but also makes it look less like pixel art.







Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Howard Day on September 22, 2015, 09:32:00 am
Ai: Okay, like this? MADE A DRONE.
(http://www.hedfiles.net/PixelShader/pixart_33.gif)
Thems some thin lines, son! the object line width is totally controlable - you use the vertex alpha value to multiply the width.
...and as for "AutoHinting"... yes, that's completely possible. Even easy! You run into the problem we did last time, though - your assets need to then be built on a grid...and that will only really work at a single zoom level. Zoom in and out, and your exact pixel positioning goes away. :(
And multiple lights with lighting gradient maps... totally possible as well! I know because I've done it. Here's a test I did for a tribute to the Star Trek 25th Anniversary art style. As you can see, the lighting is ramped, there's multiple colors, and they all play nice together.
http://www.hedfiles.net/STMobile/STMobile.html
 
Conceit: Oh. Oh? Okay, so the shadow bias stuff is in there already. :D it's already working - and I'm using the same vertex values to do workable ambient occlusion.
That short is just unbelievably awesome. Great stuff!
On the notes for the skin, and glow...I agree. I wasn't a huge fan of the existing glow, and was looking for a way to fix it. You showed me the way, thank you!
Here you go - I went with the hybrid approach:
(http://www.hedfiles.net/PixelShader/pixart_31.gif)
(http://www.hedfiles.net/PixelShader/pixart_32.gif)
And the skin shader + enhanced facial features:
(http://www.hedfiles.net/PixelShader/pixart_30.gif)

I'm working on explosions and that screen-space Drift Stage cockpit glass. if I recall correctly, I'm one of the guys that showed them how to do that...I think? It was a while ago.

Thank you, Conceit and Ai - your feedback is extremely helpful and I'm loving the challenge to improve that you're setting for me!

RAV: I wasn;t suggesting that in pixelart "anything goes" at all. I was simply pointing out that any pixel art you care to point at, no matter how skilled and polished, would have parts to it that would be reasonably considered to be "wrong" to other pixel artists. I also don't agree with or accept your proposition that "Art is no nilly-willy subject. It's serious as science." That...that statement is fractally wrong. I'm obviously not the end all authority on anything, but stating that the pixel art done by artists is the same level of seriousness as cancer research is utter nonsense. We're just not going to see eye to eye on that one. Sorry.
As for having less room for error in pixel art vs regular 3d artwork...nope, I just can't agree with you there, either. There's plentiful sloppy examples of both, and they both require skill and time to learn how to get right. I'd argue that it's harder to do highly-detailed, high res 3d artwork than pixelart - there's a reason many Indiegames with limited budgets and time/resources choose to use pixelart as their medium. It's not that it's harder than 3d artwork.

"The measure by which to judge your control on the pixels is not the code, but the image we're looking at."
Hmm. No, the measure that I judge my control over what's shown on screen is by how much I actually control it. As I've said, I control all aspects of color, lighting, shape, shading, outline thickness, surface detail, dithering, and indexed palette. The only aspect I'm missing total control over is exactly how the shadow maps fall on the surface of the 3d object. Please don't make the mistake of thinking that since I've exposed several On/Off toggles for various effects that that is the extent of the controls in the system. Those are simply the ones I've shared for demonstrative purposes.
The end goal of this is to deliver a demo project and a tutorial that clearly explains the workings of the system, in a way that can quickly have new assets dropped in, or, alternatively be imported into use in an existing project. I'm still very much on the path to that goal.

Seiseki: Oh, gotcha. Here's the scene with Dawnbringer's 16 color palette with no dithering. :D
(http://www.hedfiles.net/PixelShader/pixart_34.png)
And yes - I still need to tweak AA settings - but again, part of the problem is that the level of AA used by different pixel artists varies wildly. It's a definite headache. :D
Latest build has also been updated!
www.hedfiles.net/PixelShader/PixelShader.html


Thanks, guys!
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: RAV on September 22, 2015, 09:56:40 am
Howard, the problem we have now is I feel that there is too much an impulse to disagree than to be fruitful any further.
The usefulness to you of my participation and critique is questionable if the disagreement is seemingly so fundamental.

I felt that you were trivializing too much the ability of art to be judged for improvement, and that this is relevant to the decisions you are making.
And you feel I am taking art too serious and supposedly absolute like natural sciences, and in that close yourself off to my suggestions.
I am well aware that there is much intangible taste in the arts, but that is not what I was meaning to talk about. It's not just about taste.
Don't forget there exist "softer" sciences, like Psychology as well, which has relevance to the arts, and that allow for certain statements of usefulness in appliance.
The problems I pointed are not just a matter of art aesthetics anyway, but visual processing of the brain, which can be evaluated better than a gut feel.
Art also is methodical, so it has scientific aspects, in its success and popularity, as well as sciences have artistic aspects in inspiration for practicality.

Discussion with me will be unproductive from here on out though, as my critique will not get more useful as it has.
I think then that I overstayed my usefulness to you, as I do often here. What contribution I had to make, I did.

And yet I look very much forward to seeing your progress, on which the others will support you better than I can.
It's your thing, and you got it. It will get somewhere good, and stays an interesting endeavor no doubt.


Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Ai on September 22, 2015, 10:16:20 am
Ai: Okay, like this? MADE A DRONE.
(http://www.hedfiles.net/PixelShader/pixart_33.gif)
Thems some thin lines, son! the object line width is totally controlable - you use the vertex alpha value to multiply the width.
Yeah, that's pretty good. The only issue I can spot seems to be at the ends, where variation in line width due to varying pixel alignment becomes a bit apparent.
Quote
...and as for "AutoHinting"... yes, that's completely possible. Even easy! You run into the problem we did last time, though - your assets need to then be built on a grid...and that will only really work at a single zoom level. Zoom in and out, and your exact pixel positioning goes away. :(
That just sounds like.. manual snapping. Autohinting is more like dynamic snapping, so that whatever scale you're rendering at, it's snapped to those pixels, not whatever pixel scale you started out with.
I'm not super well up on 3d but I think the 3d equivalent would basically involve projecting in advance (that is: calculate where the corners would be in the render. Now snap those to whole pixels in render terms. Now do the actual rendering with the adjusted vertex locations. ) I think it might fix the issue where those aerials are kind of stairstepped towards the ends. Anyway, that's obviously beyond the scope of a shader, so...

Quote
And multiple lights with lighting gradient maps... totally possible as well! I know because I've done it. Here's a test I did for a tribute to the Star Trek 25th Anniversary art style. As you can see, the lighting is ramped, there's multiple colors, and they all play nice together.
http://www.hedfiles.net/STMobile/STMobile.html
Can't see shit, cap'n. Blank white rectangle on top of a blank white page.
(not surprising, Unity has never worked on this system or any other Linux system I've had. Maybe I'm missing some dependency, dunno.)
 
Quote
Seiseki: Oh, gotcha. Here's the scene with Dawnbringer's 16 color palette with no dithering. :D
(http://www.hedfiles.net/PixelShader/pixart_34.png)
Looks pretty clean.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Howard Day on September 22, 2015, 10:41:41 am
Ai: Hahahahahaha! Wow. First test of AutoPixel snapping. It totally works.
(http://www.hedfiles.net/PixelShader/pixart_39.png)
Now I just have to get it to play nice with the AA. :P
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Ai on September 22, 2015, 11:26:47 am
Lol.
I guess the snapping code is using units 8x too big than it should? it seems to be going in 8px chunks rather than.. well.. 1.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Conzeit on September 22, 2015, 01:47:25 pm
ok now you just have to have some kind of hologram glitch level or something just to show that. fukn awesome

Also I kinda like the white mech with the dawnbringer pallete...I was missing those kinds of hues

do you have separate maps for the shadow bias thing that we can see?
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Nirel on September 22, 2015, 02:45:07 pm
Since I don't believe I have much to add to the discussion I'd like to say I think it looks amazing, and I can already see it making a background together with traditional pixel art for a 2.5D pixel art platformer.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Howard Day on September 22, 2015, 05:20:49 pm
Ai: Yep, correct. otherwise, I can't see if the effect is working. :D Also, the effect looks pretty cool on the drive flares.

Conceit: hahah, yep! That's the thought. it's definitely pretty cool. Here're the ambient occlusion/shadow bias values for the fighter.
(http://www.hedfiles.net/PixelShader/VertAOShadowBias.png)

Nirel: Thanks! I really do want to put this on some Env art. Sooon.

Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Cheetah on September 22, 2015, 09:44:19 pm
Great looking thread. I wanted to add that there is also a detailed GDC talk titled "GuiltyGearXrd's Art Style : The X Factor Between 2D and 3D" by one of the lead artists on the game. Sorry if someone else already mentioned it.

Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Conzeit on September 23, 2015, 12:09:46 am
Yep, Ai mentioned it :p nice iniciative tho  :D
EDIT: unless you werent linking to http://www.gdcvault.com/play/1022031/GuiltyGearXrd-s-Art-Style-The ....your link is actually 404ing for me

Howard I shaded over your gal a bit and I'm uploading it in an edit
EDIT:
(https://i.imgur.com/NLGKNyU.png) yep, me no likey AA :p I figured whatever I could say I might find out I dont actually think if I drew over her, so I just drew over her in case it's useful :p


here's an anim with a bunch of refferences and a rough view of my process too.

(https://i.imgur.com/VjgcACC.gif)
references are:
Vice from King of Fighters
Red Arremer Midnight Bliss from Snk vs Capcom
some pervy Mai move from King of Fighters
monster From Shin Megami Tensei 2
another monster From Shin Megami Tensei 2
a shield summon from Castlevania: Symphony of The Night.
"Miko" from Tengai AKA Sengoku Blade

I notice the sprites are about King of Fighters size, so I looked around for different ways of doing eyes, that's what the refferences are about. I did the KOF thing where it's a black pixel next to a white pixel, but that sounds really hard to do right in 3D, one black pixel should be alright. I also have in the process file a test with no face features, kinda going for a Flashbback thing...that looked interesting as well.

Very interesting seeing the Shadow Bias vertex map thing. thanks Howard =) cant wait what else you've been up to
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Gil on September 23, 2015, 07:17:26 am
I think more people should do edits of this stuff, so Howard can see what a pixel artist would do with similar material. Treat it like it's something that someone drew by hand and apply the critique process for pixel art.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Howard Day on September 23, 2015, 09:42:55 am
Conceit: Wow, thank you sir. That's very helpful. I'm refactoring/rewriting the shader to better do that sort of lighting. I did try the eyes, and they totally work - except for straight on, where she looks cross-eyed. I'm going to spend more time working on that in the coming days.

What I've been playing with so far... the start of explosion VFX.
A test for the debris flame...
(http://www.hedfiles.net/PixelShader/pixart_40.gif)
And with bigger explosion blasts...
(http://www.hedfiles.net/PixelShader/pixart_41.gif)

Enjoy, and keep the crits coming!



Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Gil on September 23, 2015, 10:00:28 am
Here's what I feel I would do. That's not necessarily what you should do, just trying to provide input on what my brain tells me.

I would make dithering a material attribute. For example, studying the pictures in detail, the skin and explosions I wouldn't use any dither at all, while I enjoy it immensely on the painted parts of the mech. As an artist, I would probably turn off dither for most things, except for a select few.

For skin, I would try to go towards a cell-shaded style approach (sharp cutoff shadows over uniform medium tones with one color for speculars), using the gradient map. I would probably have a few buffer colors that take up very few horizontal space in the gradient, to act as buffers that create natural AA.

For metal, I immediately wonder if a non-linear gradient map would cause a reflection-like look to emulate shiny. I would try and tackle most problems I see through the gradient maps, as I feel that is the main source of style, pixel art wise. I would design the concept art as pixel art, and reuse the palettes created in those to try and form the correct gradient maps.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Cherno on September 23, 2015, 06:04:10 pm
Are the explosions made with Unity's ParticleSystem?

I think the effect would be enhanced if the frames per image would be increased.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Howard Day on September 23, 2015, 07:32:21 pm
Gil: Hmm - that's actually a great deal trickier than you might expect. :/ I'll see what I can do. Everything is one material right now, and I'm not sure how I'd go about controlling the dither...oh, wait, duh - I use one of the vertex color channels. Okay, Vertex color comes in R G B and A - so far I've been using RGB for AO and A for outline width - I really only need one channel for AO, so...R will be AO, G will be dithering levels, and B will be???  ...Let's make B self-illumination. Yeah, that'll work. :D I'm still re-writing the shader to have better lighting, so this will be a good chance to put that stuff in.
Cherno: Yes! ...I don't exactly know what you mean - frames per image? effectively the particle effects texture is procedural - it uses a complex mask system. I can speed up how fast the frames animate?
(http://www.hedfiles.net/PixelShader/pixart_43.gif)
I can also lengthen the particle's lives as seen here:
(http://www.hedfiles.net/PixelShader/pixart_42.gif)
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Cherno on September 23, 2015, 08:23:31 pm
By frame per pixel, I mean that the number of sprites in the animations would be decreased, to there's a more pronounced step between animation keyframes, not as smooth as it is now (where there is one sprite per frame). I'm using sprite animation lingo here of course to get the point across.

If the animation speed is decreased, it would only make exactly that: The animation goes slower. What would be needed is a decrease in the number of different particle stages that are generated, coupled with an increase of time each stage is drawn.

For example, if the explosion is goes through ten stages, getting bigger each state:

1-2-3-4-5-6-7-8-9-10

Now, it skips a few steps but each step is drawn longer:

1-1-3-3-5-5-7-7-9-9
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Howard Day on September 23, 2015, 08:35:48 pm
AH. Decease the frame rate. Gotcha. I need to write a script for that, since unity by defaults hard codes a smooth linear interpolation to all the particle effect stuff.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Cherno on September 23, 2015, 09:19:19 pm
Yeah, I'm programming in Unity too and it would be trivial to do it if it were a sprite animation, but with Particles, you probably need to edito some curves and make them into "stairs".
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Howard Day on September 23, 2015, 09:52:08 pm
Done. :D
(http://www.hedfiles.net/PixelShader/pixart_44.gif)
Easy to set the frame rate now... Works on any and all particle systems. :D
(http://www.hedfiles.net/PixelShader/pixart_45.gif)
WHOO!
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Seiseki on September 24, 2015, 12:21:41 am
If you want to make it look less like a render and more hand drawn, maybe you could try to make the shading more simplified and based on angles without smoothing.
 A good example would be the Irkalla Mechs, not just the design, but that the shading is very sharp and they have clear highlights for the edges.

Another idea which would work for a fixed perspective camera, would be darkening/simplifying the palette for everything that is further away from the camera.
Like a lot of platforming sprites do, where the leg furthest away from the camera will be dark, sometimes just a silhouette.

Also the dark background makes it hard to judge though because the darker parts blend into it and the bright parts look blown out.


Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Howard Day on September 24, 2015, 12:34:54 am
Seiseki: Unfortunately, hard edge smoothing breaks the outlines. no idea why - I will take a look and see if I can fix it. Doing a darken thing can work with a orthographic camera as well - You just need to know what the distances to fade should be. Should be easy to do! The dark background can be flipped - hit the "Dark" button right above the Play/Pause button. it will set the background to a light color.

Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Seiseki on September 24, 2015, 09:21:58 pm
Btw, if you can, make a WebGL build instead of Webplayer, the Webplayer doesn't support chrome  :'(
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Howard Day on September 24, 2015, 10:38:40 pm
http://www.hedfiles.net/PixelShader/index.html
It's an older code, Sir, but it checks out. I was about to clear it.

Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Seiseki on September 24, 2015, 11:40:29 pm
Nice :)

Wow it's so smooth in full res.. This looks very stylized though and much further from realistic rendering.
(http://share.cherrytree.at/showfile-22150/cellshaded.png)

The shadows are so overpowering, but there isn't much depth without them.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Ai on September 25, 2015, 01:03:00 am
.. Well, that one works for me. Nice to see it semi-interactively. The way you have to toggle off the previous palette option and then toggle on the new one is a little confusing, given that you have a separate option for disabling indexization on the vertical bar.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Howard Day on September 25, 2015, 01:53:17 am
Sorry! I'll make the shadows less than 100% opaque. And I know I need to fix the buttons. :( next one will work!
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Conzeit on September 25, 2015, 06:06:37 am
oh man. that is a lot of fun to mess with =)

I love making it full res disabling the outlines and the AA. I wasnt really able to use the palletes, even if I turned off the previous one for some reason. I wanted to use palletes without pixels, no can do! My favorite palletes were dawnbringer and Raptor, they have the most unexpected colorchoices for the gray ramp, so they convey pixelyness more boldly. Dawnbringer goes HAY LOOK AT ME IM PIXELD...while raptor just says hey, look at these pixels

I think it would be good if some surfaces didnt get projected shadows, a lot of the time, specially when it's on an adjacent surface instead of a surface directly below it takes out beautiful detail and it doesnt add that much depth, so it doesnt seem like something a pixelartist would always choose to do
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Howard Day on September 25, 2015, 07:05:29 pm
Conceit: Yeah, right now the pixels are directly tied to the color indexing. That's not an easy thing to untangle - it will take me a second. I'll look at hooking up the self-illumination to also remove shadows - if there's more than 25% self illumination, disable shadows? Or something.

Okay! So, I re-wrote the dithering to work on the material, not as an overlay. It works a TON better. It's now controlled via vertex colors, which means variable dithering is possible! It's pretty nice, and adds a huge amount of control to the visual result. I also set the shadow density to 50%, and toned the ambient values down as well.
(http://www.hedfiles.net/PixelShader/pixart_46.gif)
(http://www.hedfiles.net/PixelShader/pixart_47.gif)
Enjoy!

Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Cherno on September 25, 2015, 08:19:36 pm
Great work.
I would like as many variables as possible exposed in the material/shader dialog in the inspector.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Seiseki on September 26, 2015, 01:28:36 am
Wow, that's a huge improvement!  :y:
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Ai on September 26, 2015, 03:05:19 am
Yeah, that's getting really clean. Reminds me of a crisper version of OMF in terms of aesthetic. Particle effects especially feel much more balanced.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Gil on September 26, 2015, 10:45:53 am
Okay! So, I re-wrote the dithering to work on the material, not as an overlay. It works a TON better.
Yay! I was helpful :D
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Howard Day on September 27, 2015, 07:37:10 pm
Got dithered Alpha working! Boy, do I like this effect. It's much better than just straight alpha.
(http://www.hedfiles.net/PixelShader/pixart_50.gif)
Enjoy!

Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Gil on September 28, 2015, 07:29:02 am
The edges seem to flicker a lot in this latest one, most prominent on the green mech
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Cherno on September 28, 2015, 09:59:35 am
I guess the next step in improving the dithering would be to include different dithering levels:

(Top of the image)
(https://dl.dropboxusercontent.com/u/13350050/hdindexpaint/dithersampling.gif)

From here:

http://danfessler.com/blog/hd-index-painting-in-photoshop
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Howard Day on September 28, 2015, 07:41:38 pm
Gil: yeah, that'll happen any time the motion of the objects isn't stepped. The Green Hulk mech in the top center is set to have smooth motion, currently - so the crawling pixel artifacts will be particularly visible on it. if the motion is set to snap, even in 1 degree intervals, that goes entirely away. It's pretty cool. :D


Cherno: Done! It was a bit tricky, but I figured it out. Originally I was using this texture to do the dithering:
(http://www.hedfiles.net/PixelShader/ditherTest2.png)
And the solution was to encode multiple levels of dithering into that same texture - in this case that's done by varying the brightness of the pattern in different patterns. IE, this:
(http://www.hedfiles.net/PixelShader/ditherTest.png)

And here's that new pattern in action:
(http://www.hedfiles.net/PixelShader/pixart_60.png)
(http://www.hedfiles.net/PixelShader/pixart_59.png)
(http://www.hedfiles.net/PixelShader/pixart_55.png)
(http://www.hedfiles.net/PixelShader/pixart_52.png)
(http://www.hedfiles.net/PixelShader/pixart_56.png)

Enjoy!
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Cherno on September 28, 2015, 08:40:41 pm
The blue jet flames look kinda bad, though  :(

The last, green one would make an excellent background for a menu or credits screen. Reminds me of the Quake 2 installation screen.

(http://i.ytimg.com/vi/VBA6fdVykE8/maxresdefault.jpg)
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Ai on September 29, 2015, 12:34:03 am
Hm. You only have 16 levels in that dither matrix, but it (approximately) repeats every 14x14 block, which ought to be 196 levels. Any particular reason you are not using a standard Bayer matrix (https://en.wikipedia.org/wiki/Ordered_dithering)? They are generally more or less optimal for this kind of situation (void-and-cluster matrices are considered 'higher quality' but are not at all pixel-arty).
The link I gave shows 2x2, 4x4, and 8x8 standard matrices. TBH any of them would do the job for the rendering scale you are currently using,
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Seiseki on September 29, 2015, 03:25:35 am
How would it look if you added a basic skybox?
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Howard Day on October 01, 2015, 08:08:31 am
Okay, I updated the dithering to be a different pattern, one with more complexity. I also am moving to only support the WebGL build - I've had too many troubles with the Webplayer version, and justabout everyone uses Chrome nowadays. That said, here's a new version.
http://www.hedfiles.net/PixelShader/index.html
I added the Strife color palette, of which I'm a huge fan.
(http://www.hedfiles.net/PixelShader/pixart_83.png)
(http://www.hedfiles.net/PixelShader/pixart_84.png)
(http://www.hedfiles.net/PixelShader/pixart_85.png)
I've also adjusted the AA yet again, to be faster, reduced the rendering to a 16bit backbuffer for a speed increase. It's very hard to test mobile perf right now, because of a unity 5.2 perf bug that makes things way slower. But from the tests I've done, I should be pegging 60fps on almost all android platforms. with the bug it's between 45-50fps, and that number can vary widely with which settings are used. AA+2x+Shadows is always going to be more stressful on a GPU than NoAA+3X-Shadows.
I'm still working on a skybox - like everything else... you can't just drop straight, normal art in there - I need to write a shader that supports simple background objects. :D

Howie
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Gil on October 01, 2015, 01:41:06 pm
I tried editing the gun of the mech, to see why it looks so bad right now. My semi-failed attempt looks like this:

(http://i.imgur.com/OmjXTpI.png)

Some observations I gained while doing this:

- I felt like I was trying to create something out of Flashback. You need to study flashback and compare to your shader.
- Not a single part of what I worked on felt like pixel art. This is not pixel art by any stretch of the imagination and I don't feel like any edits to the shader could push it into the realm of even being acceptable as such.
- The latest two versions feel like a huge step in the wrong direction. I disagree with the folks that more complex dither would benefit this. We are routinely telling new people to forget about dither, as it seldomly adds interest, and here's people trying to convince you that it needs more complex dither.

So again, I can see you pushing this to be acceptable alongside pixel assets, but I don't see how this emulates pixel art.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Ai on October 01, 2015, 02:40:11 pm
There was exactly one person arguing for more complex dithering, which was Cherno.

TBH I prefer anything that doesn't produce those weirdly-clustering isolated 'bits of checkerboard' we were getting before.  Increasing dither levels to 4 (2x2 bayer matrix) or more would do that IME. No dither at all would also do that.

From the look of it, the current version uses 64 levels (8x8 bayer matrix or a relative thereof), which I agree is unrelated to pixel art (as is 4x4 bayer, more or less. 2x2 is the only dithering that has obvious aesthetic relationship to pixel art.).

So in case that's not clear in relation to what I said before: I noticed that the variables in the dither matrix (14x14, 16 levels) didn't add up with my experience of how to get reliable results from dither matrices (given an N*N matrix, the number of levels in it should ideally be either N*N or N).
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Gil on October 01, 2015, 03:27:52 pm
TBH I prefer anything that doesn't produce those weirdly-clustering isolated 'bits of checkerboard' we were getting before.  Increasing dither levels to 4 (2x2 bayer matrix) or more would do that IME. No dither at all would also do that.
But I specifically enjoyed those bits of checkerboard :(

I agree that the 64 levels or whatever does not really create interesting results, which is what my point is mostly.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Seiseki on October 01, 2015, 03:45:26 pm
I think the selective bits of dithering made it look more hand drawn than rendered, which I think is good.
But at the same time, I think dithering in general makes it look like a 3D render saved in gif format.

Howard is doing some really impressive and new stuff here, yet I feel like the replies are starting to turn into discouragement.
We might know everything about hand drawn pixels, but doing it in a shader is something different, be patient.

@howard

I thought the shader worked something like HD index painting, where it lowers the color count and adds dithering.
Is it a custom shader for each model or is the skybox just a special case?

Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Cherno on October 01, 2015, 05:05:48 pm
I'd like to make it clear that I would like to see the _option_ of using more complex dithering, not _only_ more complex dithering :) The more settings that are open to the user, the better.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Gil on October 01, 2015, 09:38:33 pm
Howard is doing some really impressive and new stuff here, yet I feel like the replies are starting to turn into discouragement.
Oh, I think I plainly stated before in the thread that this whole thing is HUGELY exciting to me :). I love what he's doing and I can't wait to see more.

I'm just trying to give it the old Pixelation hard-ass approach, because I feel that the higher the quality of the artist, the more important we are very critical. It's pretty obvious that Howard can handle what we throw at him :)
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Howard Day on October 01, 2015, 09:51:55 pm
Gil: ..Uh, wow - so, I'm a little surprised you're taking this to the "this can't possibly ever work" level... To be honest, I thought we'd been working pretty well together, and I thought I'd been making great progress. Honestly, I and you both know this will never replace pixel art - that's not the point - the point is to get as close as we can, and I don't think we're there yet. I've reversed the dithering back to a simple 2x2 checkerboard for all further main builds, but when it's released people can change that to whatever. I realize that you're just trying to be a hardass - but try working with me. :)
Ai: Getting exact Bayer dithering patterns is more complex than you'd think. You have to put all the info into a single texture.
Seiseki: Thanks - and yeah, each model has a special shader on it - right now all the objects are using the SAME special shader/material - but this will mean you can mix and match these assets with other objects that aren't pixeled.
Cherno: Oh, okay - well in that case, it's as simple as providing the grayscale image you want to dither with. You can do some pretty cool stuff:
(http://www.hedfiles.net/PixelShader/pixart_75.gif)
which used this image to dither:
(http://www.hedfiles.net/PixelShader/ditherTest5.png)

More to come.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Ambivorous on October 01, 2015, 10:07:14 pm
Oh right, you're the Wings of Saint Nazaire guy!
Keep up the good work man, your stuff is awesome. And this thing is awesome.  :y:

I actually don't have anything of value to add though, so just carry on.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Ai on October 01, 2015, 11:59:11 pm
TBH I prefer anything that doesn't produce those weirdly-clustering isolated 'bits of checkerboard' we were getting before.  Increasing dither levels to 4 (2x2 bayer matrix) or more would do that IME. No dither at all would also do that.
But I specifically enjoyed those bits of checkerboard :(

The same ones? I had no issue with the chunks of checkerboard that were connected to solid pixel regions of the same color. Ones that are isolated 'pop out' visually to me, looking like a small bland separate object, which I think is not generally the desired result. That's the parts I'm talking about.
I would say that generally, pixel art that uses dithering avoids such things (usually by providing a mass for the dithering to 'attach' to).

Ai: Getting exact Bayer dithering patterns is more complex than you'd think. You have to put all the info into a single texture.
I don't understand what the difficulty is. Bayer matrices are exactly small 2d textures that you threshold; which as I understood is exactly what you are already doing.
For example, what would be the problem with using this:

(http://i.imgur.com/n5X3y85.png)
(2x2 bayer matrix tiled 128x128 times, intensities used are 25% 50% 75% 100%)
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Gil on October 02, 2015, 07:45:30 am
Gil: ..Uh, wow - so, I'm a little surprised you're taking this to the "this can't possibly ever work" level... To be honest, I thought we'd been working pretty well together, and I thought I'd been making great progress. Honestly, I and you both know this will never replace pixel art - that's not the point - the point is to get as close as we can, and I don't think we're there yet.
The reason I took that route is because I went in with graphicsgale, ready to provide to edits that could help you out and found there's very few pixels there that felt placed. I do a lot of cleanups of processed stuff and that's what it felt like, like I'd had to clean the whole image.

I'm a bit skeptic that you'll ever get rid of most of those, but hey, if you're optimistic, I'm all for it. I personally would cop-out at this point and go for a simpler mechanical style that I can get away with more easily. So, sure, let's see how far we can get first, pretty sure a lot of work can still be done.

So, that latest version is wonderful, I feel like the earlier quality is back again. What's more is that the lady looks pretty good for the first time. Not sure why she's behaving so well right now, but I'd try to hang on to that. The dither pattern shows off that feature really well and I like it.

What I'm wondering right now is if there's a way to cluster stuff more. Basically, I see clusters as a bias to join the surrounding colors. In pseudo-algorithm something like: "The closer a pixel is in color to neighboring pixels, the more likely it is to change its color to those pixels". For example, a single grey pixel in a field of white would probably change color to white. A black pixel wouldn't though, it's too far removed from the surrounding white to feel bias. The result would have to be that areas of one color clump together more. This bias would be introduced before dithering, or it will try to "fix" the dither pattern.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Howard Day on October 02, 2015, 10:11:41 pm
Ambivorous: Yo. :)

Ai: Oh, well the issue is that I'm not smart. I didn't understand what you were saying, and was going about it a totally wrong way. Sorry. I've put your dithering matrix, as is, into the build - and it looks pretty cool!
(http://www.hedfiles.net/PixelShader/pixart_90.gif)
Gil: Hmm. pixel clustering is likely something I could write a shader for, but only on DXT11 platforms... I'm looking at ideas on how to push it more - like the 2-pixel minimum clustering thread - but forcing the shader to do things the way I want is turning a bit awkward.
Still working on it, but I think my next task is to make a smoke and background shader. those two things are bugging me. :D

And the build's been updated.
http://www.hedfiles.net/PixelShader/index.html
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: ptoing on October 02, 2015, 11:20:34 pm
This is pretty cool. I was wondering if you could perhaps use a depth-buffer to determine a shadow threshold for things like the legs and such, to get a similar effect like in Capcom fighting games, where far limbs will be darker and the like.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Ai on October 02, 2015, 11:53:15 pm
Ambivorous: Yo. :)

Ai: Oh, well the issue is that I'm not smart. I didn't understand what you were saying, and was going about it a totally wrong way. Sorry. I've put your dithering matrix, as is, into the build - and it looks pretty cool!
Yeah, that seems to fix the issue with clusters of checkerboard without looking overly un-pixely. Looks nice, especially the napalm on the woman is closely approaching my idea of how it should ideally look.

On a slightly different but related topic..
I'm not necessarily saying this is a good idea, just putting it out there because the texture you posted earlier inspired it:

Have you considered the possibility of a bias value for the dither matrix? A value that you add to the texture value before thresholding it? That would allow tuning 'ditheredness' up or down (hopefully on a per-object basis, so that you can dither your landscapey stuff and VFX, and have no dither on your characters).

Quote
Gil: Hmm. pixel clustering is likely something I could write a shader for, but only on DXT11 platforms... I'm looking at ideas on how to push it more - like the 2-pixel minimum clustering thread - but forcing the shader to do things the way I want is turning a bit awkward.
2pix minimum would be nice as an option in a similar vein to the option I suggest above (that is, as a way of emphasizing the different roles of visual elements by differentiating them stylistically). It is surprisingly hard to solve well, though, IME -- a fair bit of planning goes into getting a final result that is both reasonably clean and attractive.

Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Howard Day on October 03, 2015, 12:19:03 am
ptoing: Hmm - but one that only affects the shadowed areas? Yeah, I can do that. :D

Ai: Oh, good. Glad it fixes it, sorry I was an idiot. In better news, the Dither bias already exists! It's settable on the objects shader generally, and within each object as well, as the vertex color of green drives how much dither the surface gets. In some cases, mostly the transparent alpha ones, color dithering intensity is separately settable from alpha dithering. It also provides and overall alpha bias offset - the flame on the lady is the same as the explosion - with the alpha bias dropped down a bit to make the flames less billowy. Still mulling over pixel clustering ideas - there might not be a good solution.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Cyangmou on October 03, 2015, 01:35:10 am
while it is impressive as it's own style I won't necessarily call the current result that it's anywhere close to pixel art, except being "made out of pixels".

There are some disagreements what and what is not pixel art and I am pretty open for everything stylewise, however for myself pixel art is the very controlled use of pixels to make an image.

Pixel art is also restricted heavily by the grid and one of the main things one have to keep in mind next to angles is readability.
If I check the current artwork here, it looks like a color reduced 3D model with a dither filter.

Why?
It's not clean in terms of outlines
there is a lot of half blurring and half assed AA going on where the underlying lines of the 3d model fall through the grid.
A pixel artist always would draw very clean outlines and AA them with not more than 4 colors

The model is not a great design to be realized in pixel-art
This might sound stupid, but there are limits what one can do with pixel art - and those limits don't apply to the drawing level, rather down to the design level.
Pixel art is definitely not about drawing stuff with fat pixels - it's about using a grid and the suggestion of clear cut blobs of colors to illustrate something - clean and readable.
Your model has by far more subtle planes than a pixelartist would ever use in his work and the shading of the planes happens via dither.
If I'd do something like this per hand I wouldn't use any dither. I would clearly seperate the planes with different color values on a palette.
In your model there are some planes (especially round ones) which have 5 colors plus dither, which is an effect no pixel artist would ever go for. It takes to long and works against having clear and strong clusters.

If you get the outlines cleaner and can reduce the colors per plane used to a clear value you will already be much closer to the looks of actual pixel art. If that's possible you maybe want to try to get this working.



The points where I think a shader like this won't work, is that with pixel art the artist liberally can break tons of stuff a 3d program would handle differently, in terms of angles used, shading enhancement like all 2d does. Pixel art is a lot further away from realism just because the pixel artist always considers readability as credo.

Since I haven't worked with a shader like that and the only thing which is close to it is actually Guilty Gear XRD's anime filter. I think it will be kinda similar to that if you get it working - tons of additional work and information to add to models, to maintain what one would do right away with 2D techniques.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Ai on October 03, 2015, 02:23:31 am
A pixel artist always would draw very clean outlines and AA them with not more than 4 colors

In your model there are some planes (especially round ones) which have 5 colors plus dither, which is an effect no pixel artist would ever go for. It takes to long and works against having clear and strong clusters.
That seems slightly problematic:
(http://i.imgur.com/nzW9KNK.png)
(http://i.imgur.com/PycipNe.png)
(remade gfx for Bub-n-Bros. Lots of colors (24 - 266 per icon), no dirty tools used (not even lowered opacity), coherent on a pixel level)

Either the originals are pixel art and these are not -- purely through color count increase -- or color count is not a suitable metric.

I do agree with all the other points you made though, especially about model design.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Cyangmou on October 03, 2015, 02:52:38 am
That seems slightly problematic:
(http://i.imgur.com/nzW9KNK.png)
(http://i.imgur.com/PycipNe.png)
(remade gfx for Bub-n-Bros. Lots of colors (24 - 266 per icon), no dirty tools used (not even lowered opacity), coherent on a pixel level)

On a personal level I won't consider this as pixel art, because there is no deeper meaning behind where which colors went and I can't see any mastery in those sprites just from the technical approach of drawing. It seems much more like index painting or art "made with pixels"
On the questions of outlines: not everything has an outline, sure. In Howards case we definitely have an outline though - one which isn't clean.
What I meant is that no pixel artist who places colors intentionally would use more than 4 colors per AA, because it's really hard to manage. It's possible, sure - I did it on my own - sure. It's not really economical if it comes down to the time you have to invest to get it right though.

For me pixel art is the art which uses consciously placed pixels to get the idea across. This means mainly clean and crisp.
But we can discuss what and what's not pixel art in the general discussion, please don#t let's start this discussion in Howie's thread, which should stay as clean as possible for his awesome work.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Ai on October 03, 2015, 03:39:25 am
I don't really want to get into a discussion of what is or isn't pixel art, rather, I was just pointing out a contradiction in what you said; That the originals, which were pixeled, are pixel art, and these, which were also pixeled [ie each pixel carefully hand-placed with 1px brush], directly by refining the original pixel art, are not pixel art, according to your criteria.
It seems clear to me that a criteria which is self-contradictory is not one that Howard ought to use in judging the 'level of similarity to pixel-art' of his results.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: ptoing on October 03, 2015, 10:04:42 am
ptoing: Hmm - but one that only affects the shadowed areas? Yeah, I can do that. :D

Yeah, basically something where the z-depth defines a pretty on/off kinda scenario for the shadow of a specific limb.

Here you can also see how his arms change brightness as they move to and away from the "camera midpoint" or what you would like to call it.
(http://i238.photobucket.com/albums/ff172/CAPCOMWORLD/SF3_Hugo/hugo-grab.gif)
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Gil on October 03, 2015, 02:46:12 pm
I *think* the smartest way to go about that is to change the gamma of the occlusion map. Stuff is not darker the farther it's away in those sprites, as the highlights tend to be equally bright, it's just that the highlights become less big. Also, I think you need to clamp your RGB values more. RGB should be clamped so it doesn't go darker than the darkest color, resulting in areas of dark color. For example, if the darkest color is RGB(X,Y,Z) and the lightest color is RGB(U,V,W), the light value which is clamped to (0,1) should be clamped to (X,U) on the blue axis, (V,Y) on the red axis, etc. Might be worth trying out.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Howard Day on October 05, 2015, 09:38:05 pm
Cyangmou: Well, First of all, I'm tickled you took a look at it, at all! Thanks. :D And I completely agree, this is in no way pixelart. It's a 3d model with trickery - but if I can make the tricker fool the eye into thinking, even a little, that it is pixelart, then I will consider the experiment and method to be a success! I can get the outlines cleaner, and reduce the colors per plane, as you said - I've made some major improvements on those fronts here lately. As for the model pre-processing to get the right input into the shaders, it's really not that much of an issue - most is done automatically. The rest is just building the model with the results in mind - literally no different from any other type of 3d modeling. I totally agree that the mech isn't the best shape for this sort of thing - that's entirely intentional. If I had started with a shape that was easy to translate into pixelart, then what's the point? This is all about doing things the hard way to make sure they work. :D
Ai: Yes - there's no total agreement what pixel art is. Which actually works in my favor a bit - I can definitely edge closer to an indeterminate goal than I can to a hardline set of exact specifications. :D
ptoing: Yes! I can do that exact thing, and have - it's in the screenshots below and in the build! It actually is a great addition - non-shadowed models now have a ton more depth than they used to. One minor downside is that angled view of objects can look a bit odd - see the fighter or 3/4s view mechs. Because it's shading the back half of the model from the camera's view, it's not really taking the lighting into account as shadows actually do.
Gil: and yes, that's exactly what I've done. It simply changes how the lighting UVs are calculated, same as the AO does. I'm also looking into clamping values - but I gotta say, keeping them unclamped allows for a lot more lighting opportunities.

Okay, the build is updated! 3 major additions and changes to speak on:
Pixel Locking: This adds the shader  clamping ability of locking each vert of the model to a pixel location - based on the grid of pixels in the camera render. It can be turned on and off, and has the benefit of making some shapes much, much cleaner - while losing some edge detail because of the need to lock to the nearest pixel.

Depth Cueing: The models are now darkened on local object ZDepth - this currently only works correctly on the static, unanimating models - the walking mechs, as they are made up of many smaller objects, don't have the proper information to determine their depth cues. Currently it's a general solution that's a bit too intense in shadowed mode, and not intense enough in Non-Shadowed mode. Working on adding the switch to flip between those options.

Better Palette managing, and AA quality: I've changed the way the textures/lighting ramps work on the objects - they now have a minimized number of colors with a ton of inter-ramp borrowing. For example the white paint colors actually use the dark metal colors half way down. This is very similar to how actual pixel art color palettes are managed. This has the benefit of making color clusters more unified, making AA in general use less colors, and forcing the end result to be cleaner. I took the number of unique colors from 160 to 90. I am going to get that even farther down.
I've also sharpened up the AA through what else - trickery - to help try and clean up some of the lines.

Anyhow, on to the imagery!
(http://www.hedfiles.net/PixelShader/pixart_100.png)
(http://www.hedfiles.net/PixelShader/pixart_101.png)
(http://www.hedfiles.net/PixelShader/pixart_102.png)
(http://www.hedfiles.net/PixelShader/pixart_103.png)
Please, any further feedback would be welcome!

Enjoy!



Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Cyangmou on October 05, 2015, 11:21:40 pm
looks much better  :y:

actually where / how / with which programs will this shader actually work?
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Howard Day on October 06, 2015, 08:49:34 am
Thanks - I still think it needs improvement in a few key areas - though I think it might be time to construct a demo object that follows the needs of a solid mesh that the shader can use. I think I'm going to go with something from Christian Pearce! https://instagram.com/p_e_a_r_c_e/ (https://instagram.com/p_e_a_r_c_e/)
(https://scontent-sea1-1.cdninstagram.com/hphotos-xpf1/t51.2885-15/e15/11055743_1561216884156465_433724859_n.jpg)

So this system is currently designed to work with Unity3d projects - though the specifics of how it was done aren't limited to that engine... any modern engine can pretty much do this.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Gil on October 06, 2015, 12:01:47 pm
HUGE step in the right direction. I'm even starting to see well-defined clusters appear in certain areas (for example, the lightest skin color on the girl). Biggest issue right now seems to be this:

(http://i.imgur.com/ECKlgMt.png)

The yellow part is not straight and looks kinda like when you rotate pixel art with bad software, the dark line that runs over the white just looks very grainy. Stuff like that is still too pervasive in most shots.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Howard Day on October 07, 2015, 08:39:43 pm
Gil: Uhm, that's really odd. So what you're seeing there is a combination of like 3 different things. First, the yellow part you indicated is not one piece. it's the stripe around the missile warheads! they're rounded, and tilted slightly to the side. That makes it looks like a reaaaaly janky semistraight line. The darker line over the white area is a factor of the AA being disabled. Turn that on, and it workds perfectly.  I'll work on making it work as well in the non-AA mode!
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Gil on October 08, 2015, 08:25:39 am
Heh, I'm getting confused by which shots enable what and how the model looks.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Cherno on October 08, 2015, 08:27:50 am
Just yesterday I stumbled upon this isometric realtime mech combat game "META4" which uses pixelart-textures 3d models for it's terrain but detailed multi-angled sprites for the mechs. I wonder how close to the sprites the mechs would look if done as 3d models with your shader.

http://forums.tigsource.com/index.php?PHPSESSID=e7596e359d633a5308df9df87da03180&topic=32409.100

(https://scontent-lhr3-1.xx.fbcdn.net/hphotos-xfp1/v/t1.0-9/10984475_977904705588046_4911284775102151906_n.jpg?oh=7a78b37231a93463121d120cc78ce901&oe=564B7261)
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Howard Day on October 08, 2015, 09:08:21 am
Gil: Oh. I understand that. Here're some animated Gifs of it swapping through the options.
(http://www.hedfiles.net/PixelShader/pixart_110.gif)
(http://www.hedfiles.net/PixelShader/pixart_111.gif)

Cherno: I'm very familiar with that game - and some of the issues it's had. :( Here are some of the mechs at similar angles, and various settings:
(http://www.hedfiles.net/PixelShader/pixart_115.png)
(http://www.hedfiles.net/PixelShader/pixart_116.png)
(http://www.hedfiles.net/PixelShader/pixart_117.png)

Howie
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Rydin on October 09, 2015, 08:43:08 am
Quality pixel art requires fine control of each pixel working together as a team to render exactly the intended shape and material within the restraints. Since we're dealing with computer-generated images it is easy to assume the computer can do all the generating, but what's not being considered is that there is a quality to pixel art that reflects the mood and care of the artist. The mind of an artist while pushing pixels determines the flavor of the finished product. There is a relationship between the pixels and the artist that you are underestimating Howard Day. Which clusters of pixels to keep and which to alter? Does it look better like this or like that? An artist makes these informed decisions with experience. This is why it is an art. The artist puts himself into the pixels and the computer screen gives instant output but when is a piece of pixel work complete? The artist decides. The artist guides the pixels towards a vision with time and care.

If an artist writes instructions for where pixels should go and the computer carries them out, it is not the computer's fault if the output pixels lack quality placement. The end result is still the expression of the relationship between the pixels and the artist (the programmer). It is possible to write a program to generate pixel art, but the resulting art should not get a pass on standards only because the computer is doing all the heavy lifting. With an adequate understanding of the subtleties of what makes certain pixel combinations beautiful and others unthoughtful, plus the skill to execute such understanding with code, quality pixel art could be produced by an artist's programming commands. This would be heroic and quite admirable to witness. You still have much to learn about this subtle pixel technique though. 

You can't just go around dithering everything and call it pixel art because it's "close enough". Trust me on this.

I've got a pretty solid grasp on dithering and I am still not churning out quality pixel combinations like the best. There is no shortcut to putting in hours zooming in and out fine-tuning pixel placement one pixel at a time. I've tried. And while we can tell you things that could be changed with your filter here and there to help your program "create better pixelart-like output" we are only holding you back from discovering your own relationship with pixels as you learn to understand the reasons our suggestions make the output look "better". By all means use this project to expand on your craft, but don't use it as a reason to avoid learning the 1x1 pencil tool, instead realize the more time you spend with the 1x1 pencil tool the better this project will become. I am not trying to kill your motivation for your project! I like it and I want to see it become the best it can become by trying to get you to ask yourself pixel art questions. Good luck. :y:
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Gil on October 09, 2015, 10:58:09 am
the more time you spend with the 1x1 pencil tool the better this project will become
I do notice, that as I get better as a pixel artist, I'm starting to use less and less of the 1x1 brush. It used to be that pushing individual pixels around was 90% of my time, nowadays I just seem to work in broad strokes, apply automated effects, use the selection tool a lot and then in the end slap around a few pixels (but exactly the right pixels) to tidy up the image to pixel standards.

How all that relates to what Howard is doing, I don't know, but he'll figure it out :)
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: DatMuffinMan on October 11, 2015, 01:59:45 am
the more time you spend with the 1x1 pencil tool the better this project will become
I do notice, that as I get better as a pixel artist, I'm starting to use less and less of the 1x1 brush. It used to be that pushing individual pixels around was 90% of my time, nowadays I just seem to work in broad strokes, apply automated effects, use the selection tool a lot and then in the end slap around a few pixels (but exactly the right pixels) to tidy up the image to pixel standards.

How all that relates to what Howard is doing, I don't know, but he'll figure it out :)

Really? Do you have any examples of this kind of workflow? Sounds like a really odd way to make pixel-oriented artwork (I don't mean to be disrespectful; I'm genuinely curious to see what you mean).
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Gil on October 11, 2015, 05:07:33 am
Really? Do you have any examples of this kind of workflow? Sounds like a really odd way to make pixel-oriented artwork (I don't mean to be disrespectful; I'm genuinely curious to see what you mean).
I don't tend to keep progress around (bad habbit, I should, if only for backup reasons), but I think you can see that kind of progress in this folder:
http://art.game-designer.org/pixelart/rocknroll/
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: lachrymose on October 11, 2015, 02:17:00 pm
I'd go nuts if I used the 1x1 tool often lol. Myself, I do quick line art with the 1x1 brush but quickly switch to a larger brush to blob in, and the eraser to subtract, shape and form.

Anyhoot, looking really good Howard.  :y:
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Kellawgs on October 12, 2015, 06:49:02 am
The art looks good, man. I wouldn't get too concerned over whether not it can be called pixel art, because the bottom line is that it looks good regardless of what it is.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Ai on October 12, 2015, 08:01:17 am
That locking/snapping seems to be working quite well, although I notice that where several small overlapping features are crammed in, 'sparkles' / crunchy bits may appear.. with AA off, at least. It's easy to understand why, but I'm less sure about what could be done about it. Manually turning locking off per-vertex seems pretty impractical. Ideally I guess you would detect the number of nearby vertices and don't snap the vertex if that number's too high... I suppose that's not too bad since those numbers can probably be precalculated. OTOH, it might need to be done on a per-edge basis to avoid obvious distortions of planes...


The art looks good, man. I wouldn't get too concerned over whether not it can be called pixel art, because the bottom line is that it looks good regardless of what it is.
It's more a question of whether it will *fit in* with actual pixel art, which Howard has set as a goal. That's the context of all this discussion about making it more pixel-arty.

Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: MysteryMeat on March 03, 2016, 08:16:55 am
from my friend on skype:

"If you combine this with a vertex shader that quantizes bone position to limit the framerate of character animation, you could do some pretty sick things with IK"

I don't know jack all about this "Third Dimension" stuff but here you go
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Psiweapon on March 29, 2016, 12:13:38 am
Hottest shit I've seen since the beginning of the year  :o

awesomely amazing or amazingly awesome, I can't decide.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Cherno on March 29, 2016, 11:00:35 am
from my friend on skype:

"If you combine this with a vertex shader that quantizes bone position to limit the framerate of character animation, you could do some pretty sick things with IK"

I don't know jack all about this "Third Dimension" stuff but here you go

Yes, to achieve a better effect, "limited animation" should be used, with a small sumber of animation frames. This could also just be done directly in the 2d modelling application with keyframes spaced accordingly, and then with Unity slowing the animation down suficiently.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: raphaelkox on April 03, 2016, 07:16:17 pm
from my friend on skype:

"If you combine this with a vertex shader that quantizes bone position to limit the framerate of character animation, you could do some pretty sick things with IK"

I don't know jack all about this "Third Dimension" stuff but here you go

Yes, to achieve a better effect, "limited animation" should be used, with a small sumber of animation frames. This could also just be done directly in the 2d modelling application with keyframes spaced accordingly, and then with Unity slowing the animation down suficiently.

I'm pretty sure that Guilty Gear does this to bring the 2D sprite "feel" to 3D models in XRD.
There is next to no smooth movement, the model kind of snap into the next pose.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Cherno on April 03, 2016, 07:18:55 pm
from my friend on skype:

"If you combine this with a vertex shader that quantizes bone position to limit the framerate of character animation, you could do some pretty sick things with IK"

I don't know jack all about this "Third Dimension" stuff but here you go

Yes, to achieve a better effect, "limited animation" should be used, with a small sumber of animation frames. This could also just be done directly in the 2d modelling application with keyframes spaced accordingly, and then with Unity slowing the animation down suficiently.

I'm pretty sure that Guilty Gear does this to bring the 2D sprite "feel" to 3D models in XRD.
There is next to no smooth movement, the model kind of snap into the next pose.

Indeed, here is a lengthy explaination of the whole design process of GuiltyGearXrd's art style.
https://www.youtube.com/watch?v=yhGjCzxJV3E

Very interesting stuff for thoise trying to make 3d look like 2d.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: ahrimanes on April 07, 2016, 04:17:29 pm
 :D OMG this is prety awesome.. congratulations keep debeloping!!!!  :y:
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: psyhova on April 20, 2016, 12:18:32 am
It looks awesome. Is this  shader available somewhere?
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: inspeinre on May 04, 2016, 12:59:16 pm
Hi !  ;) I would like to try this shader in the my unity3d project, its possible?
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Howard Day on May 08, 2016, 01:16:19 am
Hey, guys! I'm sorry I've been so absent - I am still fiddling with this now and thn, but it's still not ready for release. There is some good news, tho - a guy by Kode80 *has* put out a very capable package on unity to do this.
http://kode80.com/blog/2016/02/23/pixelrender-for-unity3d/index.html
I am going to keep working on my version, but for now - I'm keeping it to myself.
Sorry.
Airfloater Junker make it better? :D
(http://www.hedfiles.net/PixelShader/pixart_150.png)
(http://www.hedfiles.net/PixelShader/pixart_146.png)

I hope to have more soon!
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Howard Day on May 08, 2016, 08:33:18 am
Update - added better shading and a floating rock. Gotta have a floating rock!
(http://www.hedfiles.net/PixelShader/Rock.gif)
That one has no AA - here's a zoomed up one with a new pass attempt.
(http://www.hedfiles.net/PixelShader/NewAA.gif)
Hope it looks good. :D

Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: RAV on May 08, 2016, 06:51:13 pm
Excellent. I also think that the thematic design and size of sprite go very well with this tech.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: eishiya on May 08, 2016, 10:16:56 pm
It's really cool that this is possible, but I can't help but see these as "bad pixel art". The textures and level of detail on these models don't lend themselves very well to this process, in my opinion - too much noise.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Howard Day on May 09, 2016, 10:59:57 am
RAV: Thanks!
eishiya: A totally valid point. Try this on for size - I've dropped the color count/complexity of the source textures considerably. The Ship is using a maximum of 24 colors, and the rock is using a maximimum of 16. These are using AA. I can't decide if I like it or not.  I'd love to know your thoughts...
(http://www.hedfiles.net/PixelShader/NewTextures2.gif)
Enjoy!
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: 32 on May 09, 2016, 11:09:31 am
I had the same thought and that looks infinitely more like pixel art and just generally much cleaner. The only thing that strikes me as particularly off is the shadows, the way they're so smooth and move around so much with the rotation, everything seems a bit glossy.

Loving the progress ;D
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: API-Beast on May 10, 2016, 08:28:57 am
Some thoughts:

- Setting up the color palette per sprite by hand should make the result look a bit more like pixel art. Generalist palettes don't catch the "feel" of a sprite.
- The animations are just way too smooth for pixel art, smooth movement is fine, smooth rotation is not.
- Same thing with the smoke, it's just too smooth for pixel art, try to use a hand pixeled sprite billboard instead of a particle system.
- Pixel Art animations are generally quite static, you have certain static key poses and animate between these key poses, in games you have characters often stopping in a single pose for a whole second.
- Artists often make in-between frames use less detailed shading to manage the work load and better clarity, this could be emulated by having different textures for key frames and the in-betweens.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Howard Day on May 10, 2016, 08:26:50 pm
32: Yeah, good note on the shadows - I've done some work to crisp that up.
API- All very good notes! I am sort of going for a smoother feel that exactly pixel art - that's a pretty major benefit of doing this this way, frame count does not matter. I do like your less detailed texture when in motion idea, though - I'm definitely going to give that a shot.

Okay, First pass on some foliage for the floating rocks, and new smoke shader.
(http://www.hedfiles.net/PixelShader/NewGrass2.gif)
Enjoy!
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Conzeit on May 10, 2016, 10:19:30 pm
wow 100% disagree with 32 on the shadows. it's like you're simulating being lazy O.o if you can have shading auto generated go with it :p

can you add shades back in? one midtone and one broad highlight (not specular) might work.
also you still using the gxx shadows map thing?
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Probo on May 10, 2016, 11:18:10 pm
this looks cool, but like some other people id want to push this tech to try and fool the viewer into thinking it was bona fide pixel art. As eishiya said the models seem to have too much detail in them, I think its because of the resolution, you cant really solve that by tweaking the model or the lighting.  if the image size of a game using this was 320x240 (off the top of my head as a high-for-old-games resolution, maybe higher like 480p could work?) say, and then your boat model had to be rendered at size more suitable to that overall resolution, you'd have to simplify and redesign the boat a ton, but if you got that looking good I think it'd look a lot more like what we think of when we think of pixel art.

With your tech could each object be drawn with its own perspective too?  so no matter where they are on the screen, they're always drawn as if they're being viewed from a fixed point somewhere in their centre.

like beast said, id also reduce frames of animation or go actually pose them per frame like pixel art (and reduce the frame count also). If I was making this game for real I wouldn't have the islands rotate, and just kind of bob in place, with that fixed perspective thing applied if its possible.

I realise I'm looking at it from a different angle from you though Howard! It still has its own cool look that I dig.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Howard Day on May 13, 2016, 10:57:20 am
Conzeit: Okay, brought back the shadows a bit. :D Also bumped the half-shade down in darkness - it was in there, but subtle. Too subtle. :/

Probo: Yeah, that's a possible way to take it - and rendering each model from a unique perspective would be very expensive, unless I can figure out even more levels of shader madness. :) I do like your island bob idea, so I went ahead and added that in there.

:D Okay, I added a lot more depth to the grass, and a half-shade to the coloring. I also made my own Meta-Palette out of colors and ramps from my favorite DOS games. I improved the smoke, and have completely replaced the junker with something of my own design, not ripped off from Ian Mcque. :D  I think it's much improved!

(http://www.hedfiles.net/PixelShader/NewJunker.gif)
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Howard Day on May 15, 2016, 09:40:28 am
Sunset lighting Time Of Day - and camera angle tests:
(http://www.hedfiles.net/PixelShader/sunset_01.PNG)
(http://www.hedfiles.net/PixelShader/sunset_02.PNG)
(http://www.hedfiles.net/PixelShader/sunset_03.PNG)
(http://www.hedfiles.net/PixelShader/sunset_04.PNG)
Enjoy. :D

Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Cherno on May 15, 2016, 02:50:53 pm
Perhaps you could show some renders at vastly reduced resolution, like 320x200, and see if gets too noisy or if the models are still recognizable.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: DatMuffinMan on May 18, 2016, 08:16:37 pm
just to add my 2 cents in, if I stop the animations at any given frame, the models have much more 'lonely' pixels than any pro pixel art that I've seen. I don't mean this in the way that Helm talked about them in his cluster study thread, if you're familiar with it, but just random dark/light pixels that aren't part of a larger shadow or highlight.

take this image by andylittle on PJ - (http://pixeljoint.com/files/icons/full/sad_poetry_of_the_robot_big.gif)

very detailed, lots of precise pixel placement, but there aren't any pixels that are just totally separate from their surroundings, aside from that one in the cloak (but I think that's a hole in the fabric).

edit - after looking at the renders again, it seems that you don't have this problem in the fire/smoke renders. Those are really really really nice  ;D
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Ai on May 19, 2016, 02:16:26 am
Yes; that's why I liked the 'blobby' / relatively undefined grass in Howard's previous post. It hung together better.
(the noisier grass works okay for small chunks, but not larger chunks, IMO)

I'm pretty sure that what you're talking about -IS- covered by Helm (meta-clusters? anyway, he covers the idea that isolated pixels can work if they 'aren't really isolated' -- hang together clearly with surrounding clusters.)

I agree that cleanness-wise, the smoke and fire is near faultless.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Gil on June 05, 2016, 07:41:47 pm
The main issue right now is animation. The stills look pretty good, the animated ones look nothing like pixel art IMO. You can't animate at 60FPS and expect it to look pixel art-ish. Guilty Gear Xrd devs said that's the first thing they threw out, smooth animations, because it ruins the hand-animated aesthetic immediately.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Cherno on June 06, 2016, 08:31:25 am
I think it might have more to do with the relatively high resolution in which the animations are rendered. Smooth pixel art animations can work, as seen in the Metal Slug series, but not if the sprite is so big that the illusion of a hand-drawn image is lost.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: astraldata on June 17, 2016, 04:49:51 am
The techy guy in me is giddy inside just wondering how one could take colors from a texture applied to a 3d model and render them with specific colors from a limited palette. If ever Mr. Howard Day returns to this thread one day, I'd love it if he shares his secret to how he does this, and possibly even how he might render this without using a paid plugin. I wonder if it's possible to get this effect without the f-edge plugin. I'd really love to play with this shader extensively, but I've got no money to spare. :(

Howard Day has done an excellent job with this, but I'm not sure whether the shader is usable in any game engine yet, despite him saying it was realtime. If it is usable in a game engine, does anyone know which one?



@Cherno & @Gil
I agree with you both, but at least it doesn't suffer from the ghastly look that many 2d computer games (mostly adventure point n click and RTS games) in the early 90's had where they'd take a bunch of crappy 3d models and pre-render them with really *really* low framerates and low poorly-auto-selected color-count optimizations using limited-but-also-crappy palettes at stupid-low-resolutions. Even when this is at its worse, I can't imagine it to ever look *that* bad. ;S
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Cherno on June 17, 2016, 09:23:16 am
It's all done in Unity3D.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: astraldata on July 12, 2016, 02:23:09 am
So, I've finally delved into 3ds Max's material editor and discovered how to make some pixel-art looking stuff:

(http://i.imgur.com/CgFKfmA.png)

There are still some artifacts, and this doesn't have hueshifting, so I've got a long way to go, but the body/arm/leg portions (not his eyes or mouth/nose) is an image texture material I've rendered to resemble DKC graphics.

I would love to learn how to make hueshifting and dithering possible directly through this (rather old) method, but I've yet to spend enough time to learn how to do it. The method I used was from Howard's Irkalla mech demo, so it's pre-rendered, but I would love to be able to get a similar look in Unity in realtime at some point. It'd be really cool if Howard would share even his incomplete Unity shader project with us! I would love to play with using indexed colors with 3d models directly! :)

For now though, I'll keep plugging away at pre-rendered sprite materials... I'd prefer not to reinvent the wheel anymore though...

Somehow, we game designers seem to do that quite a lot, don't we?? :(


-----------

Update:

(http://i.imgur.com/bCI79RK.png)

This one is a LOT better render-wise, and without a ton of artifacts! Got a lot of the pixel stuff sorted now! I even did a little hueshifting manually! :)
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: MrLeePerry on July 12, 2016, 07:58:46 pm
Howard, I can't say enough how much I hope you come back and put this up for sale in the asset store for Unity.  You would inspire many devs to look into these styles.  The asset you posted a link to earlier, from Kode80 is just nowhere in the ballpark of what you've been doing here.

I hope you're not discouraged or abandoning it.

I've been looking into techniques like this as well, although I'm going more for a style like Hyper Light Drifter where it's more about solid masses of colors, not outlined, and only maybe 2 colors per basic value.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Howard Day on July 20, 2017, 06:58:45 pm
Back to it, like...2 years later. Done a ton to clean up the code and methods, and the result is a lot faster than it used to be. I'm  looking into the math to modify FXAA to enforce traditional pixel clusters. I'm also going to simplify the source textures to promote less... flickery single-pixel detail.
(http://www.hedfiles.net/pixelart205.gif)
It's now using a custom palette I Frankenstein'd together from like.. 6 Classic DOS games. That was a ton of fun. :P
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: RAV on July 20, 2017, 09:16:12 pm
Good things take a while, and this is looking great. You're addressing a lot of the concerns from the start. The image quality just keeps going up.

What kind of end product do you have in mind for this? Is it more a general pixel converter, or are you gonna make a game of this?
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Howard Day on July 20, 2017, 09:20:53 pm
Well, I'm glad you see improvement...It's a bit hard to see from my perspective. :P So...in answer to your question...Both? I was just going to release the method/shader/scripts to everyone who might want to use them (after a ton of clean up and streamlining...I know what's going on, but it'd be useless gibberish to anyone else)... and I want to make a game with it. Well, a couple, actually. The first one would be a combination of Z, 7th Legion, and Cannon Fodder. Where it's an RTS based on zone control, use of a small, randomized-from-a-player-chosen-deck or power-cards (Two defensive cards, one offensive), and squad-based control, instead of enforced single-unit controls. I dunno. It's a game design that's been rattling around my head for a while, no idea if it would be any fun or worthwhile to anyone but me... but I can see it in my skull, and I need to get it out. :P
..also some Time of Day updates/ other angles...
(http://www.hedfiles.net/pixelart207.png)
(http://www.hedfiles.net/pixelart208.png)
(http://www.hedfiles.net/pixelart209.png)
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: RAV on July 20, 2017, 11:12:14 pm
I like it on every dimension, from tech to art to even your game idea. Altogether really selling me on this.
There are so many cool things happening with pixel art in recent years, and I think you got a strong positioning in all of this.
And I wonder if even 2d games could somehow benefit from this. It could reign in the colors from a lot of the effects these days.
Well, the genre is getting interesting for sure. Trying to make some greater sense and integration of that all will be a fascinating challenge.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: surt on July 20, 2017, 11:35:12 pm
My biggest issue with these is the colour mixing and dithering look to be lacking control (patches of red and grey in the rock, dithering between non-sequential greens in the grass) so they still look very much like colour reductions.
Have you considered doing something in the style of Indigo's HD index painting?
Render greyscale plus material map then hand tweak per-material gradients/dithering.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: RAV on July 20, 2017, 11:52:36 pm
It's true that these screens look less clean than the shot he posted just before. But it got me thinking a bit. There's been talk lately about bringing back a bit more "dirty grit" to pixel art. I kinda wonder, if making it as clean as possible should even be the goal, or if reveling in a slightly uncontrolled grit of techniques is an aesthetically interesting deviation for his project, pleasing in its own way. Maybe in the end it comes down again to how readable the critical action in a game would be. For example, I find it slightly difficult to focus on the ship in these latest shots. Still, getting some level of grit just right could be a strength of its own, further setting it apart visually.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Howard Day on July 21, 2017, 12:08:55 am
And these are issues I'm *aware* of, and can already at least partially fix. the issues with the noise are in part due to the texture on these objects, and the palette issues are due mostly to the fact that I'm still tuning the color useage. An index painting method would completely fix those issues, with the caveat that it's... not production friendly. At all. That's not a guess on my part - that's exactly how I did this sort of thing to begin with.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: eishiya on July 21, 2017, 12:33:19 am
Would using pre-optimized textures help with the noise? For example, avoiding colours that are difficult to reproduce cleanly with the final palette in the original textures, and using lighting colours that correspond to the hue shifting in the final palette. I'd imagine that would help the shaders avoid clashing dithering colours and ambiguous edges.
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Howard Day on July 21, 2017, 03:22:43 am
Absolutely correct! That's, in fact, the plan. :D
Update: Cleaned up and simplified greatly the textures on the rocks. Vastly improving the readability., still working on jiving the TOD color settings with the palette - it's also improved, but the sheer fact that I'm trying to do a perfectly smooth, TOD animation with light angle and hue changes makes it vastly more complex to manage. If it were easy, everyone would do it. :P
(http://www.hedfiles.net/pixelart210.png)
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Howard Day on July 26, 2017, 07:58:53 am
TOD/Explosion/Palette Animation Tests:
(http://www.hedfiles.net/pixelart300.gif)
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Howard Day on July 26, 2017, 09:33:25 pm
Zoom / straight on angle tests...
(http://www.hedfiles.net/pixelart302.gif)
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: CyberDain on August 13, 2017, 12:36:33 pm
no source code ?
looks like you correcting mesh edges based on eye angle in shader
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Howard Day on August 14, 2017, 08:49:53 pm
No source code.. meaning I haven't released the shader or scripts yet? yeah, and I likely won't till it actually works right. :P And yeah, the vertex positions are all being snapped to the precises X/y coordinate of an onscreen pixel. can look pretty gnarly when you use bigger values! Here's it at every 10px:
(http://www.hedfiles.net/pixel04.PNG)
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: Howard Day on September 04, 2017, 09:04:12 am
Working on a more...refined object to test the pixel art shader on. I had intended to do a tank, but this concept for a Millennium Falcon in John Carter of Mars from Arne caught my eye...and I was hooked.
https://twitter.com/AndroidArts/status/901527605349949441
anyhow, still very much a WIP, and a render out of Max, not re-time yet, (http://www.hedfiles.net/barsoomtest.png)
The Centennial Malagor!
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: trucoinc on September 08, 2017, 04:42:28 am
Conceit: Thanks! So, I am controlling the ramps directly. Here's the texture.
(http://www.hedfiles.net/PixelShader/MultiRamp_Test.png)
The way the shader works is that it only lets you set the vertical (v) position of the UV map - the horizontal position is skewed side to side based on the angle of the surface to the scene light, the in/out shadow value, and vertex ambient occlusion.
You can see in that map that there's some hueshifting going on - not a huge amount, but some.
If you're looking at the build (or those images I posted) the palette being used isn't exactly accurate. I've added options to palettize to classic videogames. The examples I posted are being indexed to Rise of The Triad, Arne's 64 color palette, and Duke Nukem 3d. The only one using the actual palette I made for this project is the image on the light background with no AA.
http://www.hedfiles.net/PixelShader/pixart_01.png
The difference from the IRKALLA mechs is that those were pre-rendered in Max, using a procedural max material. This is 100% realtime, designed for use in games.

lachrymose: Thanks! This actually works on everything, including phones and tablets.

I don't know if it's just me but all I see it's a pixelated image
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: astraldata on September 23, 2017, 07:52:39 am
Glad to see you working on this again Howard! -- Looks great as always! -- I hope you don't abandon it again though!!

One question:

Are you using a full-screen shader, or are you using a per-material shader? -- or a combination of both?
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: CyberDain on February 21, 2018, 03:23:07 pm
any updates ?
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: CyberDain on June 08, 2018, 12:44:10 pm
Howard Day, may be you can upload your current progress to github?
Title: Re: Realtime Rendering of 3d Meshes as Pixelart
Post by: CyberDain on January 23, 2019, 03:41:03 pm
project is dead?

very sad :(