AuthorTopic: Realtime Rendering of 3d Meshes as Pixelart  (Read 128143 times)

Offline PixelPiledriver

  • 0011
  • **
  • Posts: 997
  • Karma: +6/-0
  • Yo!
    • View Profile
    • My Blog

Re: Realtime Rendering of 3d Meshes as Pixelart

Reply #10 on: September 20, 2015, 04:58:44 am
Gratz, I love it.  :)
And knowing that it is, we seek what it is... ~ Aristotle, Posterior Analytics, Chapter 1

Offline Howard Day

  • 0010
  • *
  • Posts: 222
  • Karma: +0/-0
  • Hey, I'm a doofus.
    • View Profile

Re: Realtime Rendering of 3d Meshes as Pixelart

Reply #11 on: September 20, 2015, 08:50:00 am
Conceit: Jesus, sorry.  I try and answer with all the information! :P

RAV: Um, wow. Thanks? I'm not sure what exactly your view on this actually is - good or bad - but I can definitely tell you I'm not trying to be the next evolution of pixel art. Pixel art is a well-defined, well-understood genre of artwork, and I really cannot see anything I'm doing ever replacing that. The idea is frankly ludicrous! Everything I've done so far has been as a result of three key things:
A): Can a 3d renderer produce convincing pseudo-pixelized artwork?
B): I love solving technical puzzles!
C): I am incredibly lazy.
(C) is why I rarely do actual, you know, Pixel Art. I can - and I have in the past. I'm a huge fan of pixel art animation. I just know how to do 3d stuff too - and if I can find a lazy shortcut, I will. :)
Finally, "Surely a pixel artist will find plenty flaws in the pixel level rendering" is exactly why I posted this here. I want to know anything that stands out as needing improvement. Anything that looks off. Indigo has been assisting me with that sort of feedback from the beginning (I'm lucky enough to work with the gentleman), but I'd love any insight this wider net catches.

Seiseki: Thanks! On the "too realistic" front, I'm curious precisely what you mean. Texture? Details? Clean lines? All of this can be... tweaked. For instance, here's a shot with some noise introduced and no shadows.

...and even more noise, to ridiculous levels:


Joe: Thank you, sir! Though, as I said above, probably not the future, just another lazy shortcut. :D What sort of pixel level manipulations would you be interested in seeing?

PixelPileDriver: Thanks!

Okay, took the opportunity to put together a little kitbashed space fighter from bits. It looks pretty neat! First is in the native, intentional palette, the second is indexed to the Raptor Call of the Shadows palette. :D


..and a angled verticle SHMUP version!


Enjoy!


« Last Edit: September 20, 2015, 09:40:24 am by Howard Day »

Offline RAV

  • 0010
  • *
  • Posts: 293
  • Karma: +0/-0
    • View Profile
    • Blackbox Voxel Tool

Re: Realtime Rendering of 3d Meshes as Pixelart

Reply #12 on: September 20, 2015, 10:06:00 am
Don't worry about it, Howard. I think it's pretty cool and a good thing. It was just a general musing on it "philosophically."
I wasn't talking about it in terms of replacing pixel art either, but the two different approaches of how 3d can function and integrate for pixel art.

Quote
I want to know anything that stands out as needing improvement. Anything that looks off.
Baring some minor adjustment, I think you got it pretty much as good as it can get -- which is good enough for your purpose. It looks pretty cool.

If you don't mind me musing some more, about the fundamental problems of rendering to pixel art, which may focus more on what improvement is reasonable to expect and search for:

What you do is realistic rendering of pixel art. This can be pretty convenient to short cut some of the more tedious aspects of pixel art in a certain style.
There are simple basic rules to realistic rendering, which is applied wholesome to the image by the code.
In addition this can be somewhat adapted partially on a more or less random basis.

What gives a piece an impresssion of pixel art most however is, as I called it, artistic rendering.
And that means a conscious aesthetic decision on each pixel, that may derive from different rules, or even spite rules.
This is the heart of the problem. The human sense, reasonable or emotional, of the pixel.

Whatever further correction you attempt to do programmatically, will look okay in some place, and senseless in another, because it doesn't actually understand what it tries to render, and what it tries to convey in that to the audience. The pixel pattern rendering problem is somewhat akin to the "what's the best palette" discussion, even much worse a problem actually.

In addition, this pixel art is produced in a very typical fashion, to be this smooth. Which will inspire pixel artists to create counter-trends of increasingly ideosyncratic and unrealistic rendering techniques that have their own appeal. This means a further divergence in pixel design that increases the differentiation in impression from 3d application, making that look ever so more different from pixel art, uncanny pixel art, and in its own way limited as pixel art, even though paradoxically it was made in an attempt to push certain limits of pixel art in its strife to look like pixel art. That too is part of the art, finding new ways of expression and differentiating itself from other artforms, searching for a recognition, coherence and self-reliance as its own artform. Interesting enough, your success in automating palette application, rather makes me question further if colour limit palette orientation really is at the relevant heart of pixel art, and so as a pixel artist I'd be tempted to depart from that in response, to find my own way, maybe accelerated by how popular your approach turns out to be.

Still, despite these musings, what you do is very much interesting and worthwhile, and pixel artists will find ways to take assisted advantage of it for their own practical purpose in some form or another as suits them. I very much look forward seeing your game looking like this.

« Last Edit: September 20, 2015, 10:27:33 am by RAV »

Offline Gil

  • 0100
  • ***
  • Posts: 1543
  • Karma: +1/-0
  • Too square to be hip
    • http://pixeljoint.com/p/475.htm
    • View Profile
    • My Portfolio

Re: Realtime Rendering of 3d Meshes as Pixelart

Reply #13 on: September 20, 2015, 10:14:19 am
I think you need to create some scenes where pixel art is combined with this. I don't see a huge value in making an entire 3D game like this, the value is in making a pixel art game, and having certain parts be 3D I think. Then the question is if you can make your shader fit that particular game, to a point that it doesn't break the immersion.

Offline Ai

  • 0100
  • ***
  • Posts: 1057
  • Karma: +2/-0
  • finti
    • http://pixeljoint.com/pixels/profile.asp?id=1996
    • finticemo
    • View Profile

Re: Realtime Rendering of 3d Meshes as Pixelart

Reply #14 on: September 20, 2015, 11:48:11 am
I'm sort of on the same page with Gil, I guess.. I think that your algorithm performs well for these mecha / space fighters, especially considering it's realtime, but we won't really have an idea whether it performs well in general unless we can see it in action on a range of stuff -- landscape/tiles, less realistic people; interiors; much more toony exaggerated color ramps;even bigger (in #pixels of area terms) objects and significantly smaller objects. Things that live in very different design spaces. Some of that might seem perverse given what you are trying to do here, but IMO it's needed as a basis for comparison, to get a realistic picture of its overall performance.

I guess the overall message there is, hurry up and release it, and we can get to trying it out on that different stuff;)

...

The only crit I would make of your most recent renders is that the guns seem a bit soft. Since I'm familiar with 3d I'm aware this can arise when you increase AA levels, but maybe there is some way to counteract that selectively so you can make the important parts as sharp as they ought to be?

If I think more idealistically I would say that there are some things that you just don't want to be AAed at all. If you look at Metal Slug, for example, their strategic use of hard edges contributes a lot to the readability of the art, which IMO should be a primary concern of any game developer.

If you insist on being pessimistic about your own abilities, consider also being pessimistic about the accuracy of that pessimistic judgement.

Offline lachrymose

  • 0010
  • *
  • Posts: 345
  • Karma: +0/-0
    • View Profile

Re: Realtime Rendering of 3d Meshes as Pixelart

Reply #15 on: September 20, 2015, 12:05:35 pm
I think you need to create some scenes where pixel art is combined with this. I don't see a huge value in making an entire 3D game like this, the value is in making a pixel art game, and having certain parts be 3D I think. Then the question is if you can make your shader fit that particular game, to a point that it doesn't break the immersion.

I see the value of it in decreasing work; for large sprites or for things that require a lot of sprite animation. You could render out/animate a 3D image, use the pixel shader on it.
Then take from that the frames you need for pixel animation, afterwards its just a matter of going though and cleaning up...Certainly sounds faster then starting each frame from scratch like normal. Especially for something like 8-directional movement sprites, or sprites for fighting games.
« Last Edit: September 20, 2015, 01:25:39 pm by lachrymose »

Offline RAV

  • 0010
  • *
  • Posts: 293
  • Karma: +0/-0
    • View Profile
    • Blackbox Voxel Tool

Re: Realtime Rendering of 3d Meshes as Pixelart

Reply #16 on: September 20, 2015, 12:59:48 pm
Quote from: AI
If I think more idealistically I would say that there are some things that you just don't want to be AAed at all. If you look at Metal Slug, for example, their strategic use of hard edges contributes a lot to the readability of the art, which IMO should be a primary concern of any game developer.
If we keep looking at it as a realtime application, that's another conundrum we have: pixel art can afford to be very sharp, because it doesn't have to hide mistakes. And thus sharpness is a common trait of pixel art. The more we try to just sweepingly blur the errors out here, it will tend to look less convincing as pixel art and more like just another form of cellshading, as well as make a carefully managed colour palette less and less meaningful in the end. A very narrow line to walk in optimizing it. So that's something to keep in mind.

It also highlights something I've pointed to earlier: what does the selectivity of "there is some things you just don't want to" mean programmatically in the appiication of a technique? This is the key problem we're facing. not just from asset to asset, but even within an asset. Learning the good application of a technique is worth years of struggle for any pixel artist. Reading up the principle of AA is a long shot away from using it well to best effect, refined and selectively. I think this is pixel art at its heart, the strategy of techniques, use with care, from case to case. And I think this is very much what makes most the impression of looking pixel art.


« Last Edit: September 20, 2015, 01:24:48 pm by RAV »

Offline RAV

  • 0010
  • *
  • Posts: 293
  • Karma: +0/-0
    • View Profile
    • Blackbox Voxel Tool

Re: Realtime Rendering of 3d Meshes as Pixelart

Reply #17 on: September 20, 2015, 03:53:36 pm
Quote from: Howard Day
Thanks! On the "too realistic" front, I'm curious precisely what you mean. Texture? Details? Clean lines?
I think most of all it means the calculated correctness of whatever it applies universally. Be it perspectives, shadows, shading, whatever not done manually. it is all physically correct or solves numeric otherwise, even when it is "toonish". Since this kind of correctness in everything across all frames this smooth is not tradition to pixel art, it can look uncanny.

When you look at a lot of pixel art, they often are in some way or another decidedly "incorrect". Even a complex mash of whatever seems right.
But the point is, it is not unreasonably incorrect. it's not a matter of noise. The artist tries to achieve something specific that principle correctness doesn't help with.

They do this for various reasons, for example readability. A correct perspective is not always the most advantageous and satisfying in every situation.
And an "incorrect" decision in one part then affects decisions in other parts to suit that, like shading, etc. A whole chain of decisions that make no physical sense, yet are not random, but right.

You can for example try adjust and combine various projection settings across the scene, but there will be difficulties, since it doesn't really understand what it's doing, and it's just the tip of the iceberg of what you need doing, depending on how far you can and want take it.

It's really the exceptionality of pixel art in every other detail that is so hard to capture on whatever aspect you automate for dynamism.


I hope though, you don't read all this as if I were just trying to throw sticks on your legs. Keep up the good work and find ways to make it work, okay. Try take advantage of your methods and make the best of it. I think it's plenty cool as is, I see sense in it. I have my own stake on the topic, had a lot of thought about it, so I'm kinda passionate and talky about it. Not sure that helps though.

« Last Edit: September 20, 2015, 05:22:43 pm by RAV »

Offline Cherno

  • 0010
  • *
  • Posts: 122
  • Karma: +0/-0
    • View Profile

Re: Realtime Rendering of 3d Meshes as Pixelart

Reply #18 on: September 20, 2015, 06:44:59 pm
That's a great shader, I love it. I can see it be a moderate success on the Unity marketplace (moderate because it does caters to a niche, after all). On the other hand, I wouldn't mind getting my hands on this shader for free *whistles* ;)

Offline Howard Day

  • 0010
  • *
  • Posts: 222
  • Karma: +0/-0
  • Hey, I'm a doofus.
    • View Profile

Re: Realtime Rendering of 3d Meshes as Pixelart

Reply #19 on: September 21, 2015, 08:36:38 am
 RAV: Hmmm. Lots to think on there.

Gil: Yes. I indeed want to try that next. Something along these lines: http://41.media.tumblr.com/963cf1361b3d0290e03e876da0efcd1f/tumblr_nj7rg591yt1u54l7wo1_1280.jpg
(Concept by Nurzhan Bekkaliev!)

Ai: Yeah the blurry AA was really, really bothering me. I've since fixed it!

lachrymose: That's also a thought. :D

Cherno: Well, it's going to be released for free. :D


Okay. In new news... I've been an absolute idiot. The Indexing shader that I wrote wasn't quite doing what I thought it was. As a result...the indexed colors were being smoothly interpolated. This is, safe to say, not the intended behavior. I've since fixed it, and now, the indexed results are absolutely perfect. Kills me that I missed something this stupid. Ugh. UGH.
Here's a look at the new output:



So that's settled, and a weight off my mind. The next thing on my ToFIX list was the AA. It was very blurry - and just..looked artificial, and not pixeled at all. So, I figured out a method around that. The Scene is rendered with AA at 3x the final screen resolution.. then Nearest Neighbor scaled to the final desired resolution. Here's the image with no AA:

And with the new AA!

I'm stoked. It looks much improved to me!
Here are some more shots - pay attention to the LUT cube in the upper left. :D





I'd love to hear any and all feed back and thoughts you guys have.

Enjoy!