Thanks for your comments. You bring up some good points.
I don't get this look. It's pixel art, but you distort the pixels and filter them? Seems to me that's a lot of work for not much gain...
Actually, I have spent hours trying to minimize the distortion, and trust me, it looks a lot better than it would otherwise. You see, each character is being drawn on its own invisible plane in the 3D world. I turn all of those planes to face the camera, so you don't ever see the characters edge-on. That much looks pretty good in first-person or over-the-shoulder camera modes. But at an isometric camera angle, such as you see in most tactical or strategy games, things start to go wrong: When the characters face a camera above them, then toward the sides of the screen they're almost horizontal. And if they're next to a wall, when the sprite tilts to face the camera, the head clips into the wall and you just see a torso sticking out.
So instead of making the sprites face the camera, I have the sprites turn toward the camera's plane, so that all of the sprites are lined up perpendicular to the camera's line of sight. And I have them ignore the camera's Z position, so that they stay upright. And I tilt them slightly inwards as they get to sides of the screen. That looks pretty good, but then the sprites seem to get really short as they pass under the camera. So I make the sprites taller as they go under the camera. That looks OK actually, but you can see the shadows going bonkers as the sprites stretch and tilt. So I make the characters themselves not cast shadows, but a similar sprite, invisible to the camera, that doesn't stretch, casts the shadow.
So why don't I just use an isometric camera with no vanishing points? Wouldn't that solve everything? It would solve a lot, certainly, but UDK doesn't do true isometric. The best approximation is moving the camera way out and zooming the field of view way in. But at a distance, the depth buffer starts making approximations, and the rugs blend into the floor, and curtains blend into the wall. So I have to pick a camera distance that strikes a balance between minimizing the distortion and not confusing the depth buffer.
So why can't I just draw the sprites directly, flat against the screen, without all the effort described above? I could, but then they wouldn't be affected by lighting. They wouldn't disappear behind 3D scenery. UDK would let me draw them as a particle effect, which would draw them flat against the screen with no distortion, and properly placed and occluded behind other objects in the scene, but that would still cause the problem with clipping into walls and other obstacles--characters with their torsos sticking out of walls, like I mentioned above.
And that's why I'm rendering those pixel characters the way I'm rendering them. It's not ideal, but it's probably the best that I can do without having to write a whole custom game engine.
If you were to simply paint some body parts and do bone animation, you could get smoother animations, do more variation, in less time.
That is very, very true. And if I were to do it again, that's what I would do. But early on in development I decided I wanted the characters to look very old-school. I wanted to recreate the look of Final Fantasy Tactics and Ogre Battle. And those ancient games used sprite sheets--not this newfangled 2D bone animation. Well, after going through all of the effort and expense of actually drawing (or commissioning) the sprite sheets, I'm thinking bone animation might not have been such a bad idea after all. Oh well. At least I can tell myself that this way, I'm being true to my original artistic intent, or something.
Why the outline filter btw?
The stylistic difference between the pixel characters and a rather detailed 3D environment is kind of jarring. I felt that they needed some sort of visual theme to tie them together. I thought that a Sobel edge detection post process filter was the best way to unify the look. I've heard some people say they like it and I've heard some people say they don't like it. In the end, it's just a matter of style and taste, and I thought I should go with it.
it may look cool if you were doing something like this ... although i think the reason this look can be pulled off is because of the fixed camera angle and the fact that the sprites aren't skewed.
Yes, those pixel characters look good in that 3D setting for a variety of reasons. But I think at its most basic level, it works because it's simple. I'm trying to achieve something a little more complicated from a technical standpoint, and it's going to involve some trade-offs. But if I can earn comparisons to games like Fire Emblem, then I'll be glad just to be mentioned in the same sentence.