[Tool] 3D to 2D Blender Pixel Shader

[Alex Sinigaglia]

http://www.cgchannel.com/2015/02/this-free-blender-shader-turns-3d-models-into-pixel-art/

Free Blender shader turns 3D models into pixel art

Japanese artist Toshihiro Kushizaki has released the Pixel Art Shader: a free Blender shader that should convert any 3D model into old-school pixel art.

To render out a pixel art image, you just need a 3D model in .x format and a palette texture, which defines the set of colours to be used; but you can augment the result with standard texture maps, even including AO.

The image is generated in BMP format, and the results look pretty good, at least in the demo scene.

There’s a detailed online manual, but like the rest of Kushizaki’s site, it’s in Japanese, so if you’re a non-Japanese-speaker, you’ll need to navigate Google Translate.

Download the Pixel Art Shader on Toshihiro Kushizaki’s website (Automatic English translation)

Read an English-language user-contributed tutorial on using the shader.

Other examples:
Spoiler

I didn't see this posted here, so here it is. As always, please move, merge or delete this topic if it already exists or is in the wrong place.

I guess there already are tools that can make this happen.
Still, give it a try if you're interested and skilled with Blender.

[Kazuya Mochu]

I saw this pop up on facebook. It looks really cool and its by far the best pixelated 3d I've seen.

the only issue with 3d and pixel art is that 3d is very rigid when it comes to animation.
I had a try and replicating this using 3dsmax and did something slightly cool, but nothing as good as this. kudos for the developer!

[wzl]

I found interesting what they did with guilty gear xrd It probably doesn't qualify as pixel art, but it shows some nice handling of animations. They for one thing don't interpolate between keyframes to keep the oldschool frame-based animation feel to it. To compensate they added the effect/blur trails(?) as an extension to the mesh.
When i'm looking at it i'm still pondering whether thats actually 3d until the camera starts to orbit around the characters. It's impressingly well made.

[Kazuya Mochu]

This is what my experiments got me. no antialiasing on the render and used a gradient ramp on the diffuse, set to follow the light direction

[lachrymose]

So like, could I use this to avoid learning to animate? 

[Kazuya Mochu]

animation is animation. 2d or 3d. they just look slightly different because of the technical aspects, but they are base on the same fundamentals

[Ai]

So what is the advantage over my standard pixel-art method with Blender (which is just: Use diffuse ramps with hard edges, use either no speculars or specular ramps with hard edges; turn off antialiasing)?
All I can see so far is that perhaps it works better with multicolor layered textures.

EDIT: Examples of the results of my methods: 1 2

EDIT2: This
gives some vague clues as to what is going on beyond what base Blender does.

[PixelPiledriver]

Looks pretty cool.
Lethal League used a similar process and it looks nice.

[Conzeit]

this doesnt really look that interesting to me, it looks like a celshade script that is focused on not creating too many jaggies....(I guess there's some pallete going ons from Kazuya's post?) what I really want to see is someone tackle open source-ifying what they did with Guilty Gear Xrd. it has so much potential for doing pixelarty stuff.

there was a part about using normal maps (or something to that effect, I dont really 3D yet so I dont have perfect langauage :p) to tell the program to shade things differently from what the geometry would suggest. They used it to make very pixelarty things, what they did with the hilights of May's anvil thing struck me as specially full of potential.

I REPOST THIS SHIZNIT AGAIN BECAUSE I THINK EVERYONE SHOULD HAVE FLIPPED OUT OF THEIR MINDS AND NOBODY DID!! FSDADSDSADA

the english transcription of the GGXRD behind the scenes interview thingy.
http://www.polycount.com/forum/showpost.php?p=2099538&postcount=229
http://www.polycount.com/forum/showpost.php?p=2107579&postcount=241

original article
http://www.4gamer.net/games/216/G021678/20140703095/
http://www.4gamer.net/games/216/G021678/20140703095/index_2.html

*cast spell on 3d experts of the forum so they flip out about this shit and make a big fucking community activity out of creating some mad script maggiks that make it really easy for a 3d moron like me to do this kind of shit*

[Ai]

this doesnt really look that interesting to me, it looks like a celshade script that is focused on not creating too many jaggies....(I guess there's some pallete going ons from Kazuya's post?) what I really want to see is someone tackle open source-ifying what they did with Guilty Gear Xrd. it has so much potential for doing pixelarty stuff.

there was a part about using normal maps (or something to that effect, I dont really 3D yet so I dont have perfect langauage :p) to tell the program to shade things differently from what the geometry would suggest.

That would be a normal map if it had anything to do with normals . I guess the right thing to call it might be a shader input texture -- a texture that the shader uses, that isn't directly displayed in any normal sense.

They used it to make very pixelarty things, what they did with the hilights of May's anvil thing struck me as specially full of potential.

It's an anchor, no? Kind of different from an anvil

the english transcription of the GGXRD behind the scenes interview thingy.
http://www.polycount.com/forum/showpost.php?p=2099538&postcount=229
http://www.polycount.com/forum/showpost.php?p=2107579&postcount=241

Hey, that's really interesting.
It has some things in common with Paperman IMO.
I think you could do most of that stuff using Freestyle in Blender.

Looking at the things in detail, you have:
* line thickness info in Alpha channel of vertex color
* ambient occlusion in Red channel of vertex (since it's using a cel shader, this is functionally just an offset for the light/dark threshold )
* separate ambient occlusion layer also in Green channel of control texture (used to make areas always shadowed or always lit)
* Specular size and intensity in B and R channels of control texture
* SSS color is a separate texture, nothing complex there.
* Vertex color Green and Blue channels are ignored.

The axis-aligned textures are really weird, honestly. I understand how it works but it's gonna be a pain to get the angles working how you want.

I'd like to hear more about the bone system for the face, personally -- I suspect that they automatically manipulate the bones relative to the camera normal so that the geometry looks sculpted for whatever perspective you are viewing it at. (in addition to the normal animation keying they refer to in the second translated article)

Bone scaling in 3d -- not sure I know of any other system that does this out of the box. It's obviously very useful for exaggerated movements. Not sure how to do in Blender.

The mentioned postprocessing effects are definitely doable in Blender, not sure how to make them make sense in relation to pixel art though.

[Conzeit]

I'm glad you think it can be done (freestyle? you mean no extra coding?) in blender.

LOL it'd be pretty funny to have someone fight with an anvil XD dunno what I was thinking.

The artist made ambient occlusion on the R channel of vertex (that's what I was confusing with normal maps, probably because it's displayed as a grayscale texture :p) is what interests me most, this one trick is what takes this game from regular crappy fake celshading to something that can really stand next to hand drawn celshading.
Not sure about the ambient occlusion they store on G channel, self-shadowing tends to help bring volume, for example look at later KOF sprites. XI does it a lot

Vertex normals are adjusted to simplify the light response and eliminate small scale polygonal shadows. Apparently the export from Softimage to UE3 automatically regenerated normals so they had to rewrite it. Faces were adjusted by hands, other parts by transferring normals from proxy objects, with the examples of Sol's pants or character hair in general.

I dont really understand this part but I *think* "simplify the light response and eliminate small scale polygonal shadows" means they're eliminating more micro-self shadowing? this seems smart if it helps create the celshading they did.

The line thickness variation is great  (at alpha channel of vertex?), I would've probably exagerated it more. Specially interesting is the part that they map the how much it scales with the camera (green channel of vertex) All sorts of scaling with camera seem very pixel relevant, just like smaller res sprites tend to have bigger heads to fit expressions (that they swap to more super deformed head models on zoom out sems relevant too).

Basically on a lit surface the response will be something like (light color + ambient color) * texture color, while in shadow the response is ambient color * texture color * SSS color, so you can tint the shadows.

Defining shadow color defintively seems useful what with all our hueshifting and all.

I'm just now realizing that the gradient map of this 2dBlender Pixel shader plus this trick for defining shadow color would be really cool, you get  two ramps for the shading to go down.

Very much like the Street Fighter 3 faking of ambient lights.

look at how on the lit parts of ibuki and Leo here the darkest tones are orange, while on the darkened parts the darkest tones are a different hue (blue for ibuki purple for Leo)

I'm having a hard time finding the specular stuff that I liked about May's anchor...hm =/ it's probably just what they illustrate with  I-no's image, in wich they use the ILM texture and R is specular intensity, B is specular size. but the way it looks  here

 really sells me on animeish hilights.

BTW I know what R,G,B and alpha channels are, I have a somewhat vague idea of what vertex shading is, but I have no idea what a vertex texture is, or a control texture or SSS texture is a ILM texture for that matter.

definitively agree about the bones in the face. I think they either had even more swappable heads, or they just deformed the mesh to benefit each predefined shot for the cutscenes, and for regular ingame view just assumed regular sidescroller point of view. kind of like Meindbender does with their stuff.


There's a rigging demostration of it on vimeo =)

and people have replicated it on blender =)

[PixelPiledriver]

Just wanted to jump in and blather a bit.

I have a somewhat vague idea of what vertex shading is

Vertex Shaders are passed data from vertices.
Then each vertex is processed, one at a time.
Some of the data is then passed to the Pixel Shader.
Each pixel is then processed using the data passed from the vertex shader, and then a single color is returned as the result in (r,g,b,a) format.

The entire process could just be called a Shader.
But you could also call it Vertex Shading.
Fact is you cant draw a single pixel without a vertices in the shader pipeline. --> in general
It's just built that way.

but I have no idea what a vertex texture is, or a control texture or SSS texture is a ILM texture for that matter.

SSS texture = Sub Surface Scatter texture
ILM = probly Illumination Texture for glowy stuff

Not too sure about control texture or vertex texture.
I'd have to see the context.
But in general you can pass in textures as any sort of data.
The colors they represent are used to process a calculation of sorts, so they could be really anything at all.
You could have a fade texture, a blur texture, a y axis texture, an x axis texture, a daylight texture, etc.
I once wrote a shader that passed in positions of items in the level as a texture.

[wzl]

Fact is you cant draw a single pixel without a vertices in the shader pipeline. --> in general
It's just built that way.

you can use Points or Line/LineStrips though. Efficiency is a different matter though
And those too are routed through the shader pipeline.

[Ai]

Conceit, you should definitely check out Blender's Freestyle functionality since you are interested in what they are doing here. eg. Google images shows a good variety of different effects achieved by freestyle. You can see cel shading is fairly popular. Every so often somebody posts an anime-style cel-shaded character using Freestyle on the Blender Google+ page.

What PPD has said is accurate, but if you really want to get a sense of how it works I suggest you do at least a little experimenting with weight-painting to see how the values blend and how you are limited by the color being per-vertex.

If you don't understand the vertex normal discussion very well,, you definitely could benefit from more 3d experience. I guess that the adjustment they perform might be considered as a type of selective 'sharpen' filter in terms of 2d imaging .. making the normals of adjacent faces differ more so that light will tend to light up one or the other but not both. IIRC there are existing 'filters' to apply this kind of adjustment to a mesh in blender, but usually they adjust the actual mesh, not only the normals.

[PixelPiledriver]

you can use Points or Line/LineStrips though. Efficiency is a different matter though
And those too are routed through the shader pipeline.

You called my "in general" bluff. 
I wanted to imply there are more layers to the topic without actually talking about them.
However, just to return volley:
How are points and lines defined?
Verts.
 


Also here's some great examples for understanding the potential benefits of modifying normals.
http://wiki.polycount.com/wiki/VertexNormal

[lachrymose]

Tried to use the shader, won't render the pretty geometry. All I get are black frames.
Whats everyone running on? Trying to narrow things down. I'm running Win 8 64bit, AMD GPU.

Edit: After more reading around...AMD cards are indeed the problem.