AuthorTopic: Pixel Art with supervised learning algorithms  (Read 5190 times)

Offline Grasshoper

  • 0001
  • *
  • Posts: 4
  • Karma: +0/-0
    • View Profile

Pixel Art with supervised learning algorithms

on: June 02, 2016, 07:54:11 pm
Dear all,

I am new to this forum and don't know if it's the right place to ask this (right?) question.
I am part of a small and autoproduced electro/rock musical duo and we plan to have a pixel art music video to promote our upcoming album.

We are currently implementing a supervised machine learning algorithm that would 'learn' how to automatically convert movies images to pixel art. We don't know if it is going to run smoothly and we expect a lot of noise on the resulting output. But who knows?
In order to get the algorithm to work we need to have a data sample, ie. real images that have already been converted to pixel art graphics by artists. We need both the real images and the rendered one. I've been heavily browsing the internet lately and could find some samples for our data but I am afraid we would need more than just a few samples to succeed in this direction.

May I ask if someone knows a place where such data exist (original picture + corresponding rendered one?)
I would also be happy to answer any interrogation.
Please be free to ask.

All the bests,

Grass

Offline Kasumi

  • 0010
  • *
  • Posts: 275
  • Karma: +1/-0
    • View Profile

Re: Pixel Art with supervised learning algorithms

Reply #1 on: June 02, 2016, 11:27:23 pm
Jinn has a series of nude images he has pixeled, and he tends to link the actual image in the description. That said, they tend to lack the background which will add extra work for you some way.

Prowler has some stuff.

(originally here, but inactive):


And here is its reference (from here)



I'm not sure it's terribly common to 1:1 pixel an image. Even demoscene stuff which usually strives to be reasonably close in a lot of ways tend to deviate from the source image to fit the theme of the demo.  I can think of a few others in my head that I remember a reference image being associated with, but I'm not sure I'll actually be able to find them. If I can, I'll update the post.
« Last Edit: June 02, 2016, 11:30:31 pm by Kasumi »
I make actual NES games. Thus, I'm the unofficial forum dealer of too much information about the NES

Offline yrizoud

  • 0010
  • *
  • Posts: 330
  • Karma: +0/-0
    • View Profile

Re: Pixel Art with supervised learning algorithms

Reply #2 on: June 03, 2016, 08:48:18 am
There is this site, though it concerns only demoscene art, so I'm not sure if it fits what you want.
http://www.kameli.net/nocopy/

Offline Grasshoper

  • 0001
  • *
  • Posts: 4
  • Karma: +0/-0
    • View Profile

Re: Pixel Art with supervised learning algorithms

Reply #3 on: June 03, 2016, 03:55:32 pm
Many thanks to both of you for the links.
I started to browsed the associated repositories but could not find usefull samples for the time being, altough having not browsed the whole repository tree (lot of stuff in it!).
I feel that I was way too much consice in my original post. I am looking for original scenes or pictures turned into pixel art (preferably by artists themselves rather than automatic pixeliation  :).
An example lies in this URL:
http://www.shutterstock.com/blog/how-to-turn-any-photograph-into-pixel-art-with-photoshop

And one other requirement for the algorithm to work is that *drumroll* both images have the same resolution. If not it will add some undesired noise to the output.

But again, thank you very much for you links, I will continue to browse the suggested links.

Cheers!

Offline 32

  • 0011
  • **
  • Posts: 535
  • Karma: +1/-0
    • @AngusDoolan
    • http://pixeljoint.com/p/19827.htm
    • View Profile

Re: Pixel Art with supervised learning algorithms

Reply #4 on: June 03, 2016, 11:13:28 pm
I don't think you're going to have much luck finding a body of work that does that. It's extremely specific and doesn't seem to have much artistic merit. I can think of maybe some times artists have done studies of fruits and nudes etc but I don't think anyone is just reducing a photo to pixel art. The linked gallery to Jinn's stuff is probably the closest you're likely to get. You could manipulate the original photos to match his output.

If that won't work I think you would probably need to hire some artists to specifically perform the task.

Offline MysteryMeat

  • 0100
  • ***
  • Posts: 1997
  • Karma: +1/-0
  • "The new alternative to q-tipping your cat!"
    • mysterymeat
    • spoiledmysterymeat
    • View Profile
    • My rad art blog!

Re: Pixel Art with supervised learning algorithms

Reply #5 on: June 03, 2016, 11:33:55 pm
Try looking up old movie games for the commodore, they used a bunch of screens traced over from movies
Like this one, from the Terminator game:

PSA: use imgur
http://pixelation.org/index.php?topic=19838.0 also go suggest on my quest, cmon
MAJOR BORK TALLY: |

Offline Grasshoper

  • 0001
  • *
  • Posts: 4
  • Karma: +0/-0
    • View Profile

Re: Pixel Art with supervised learning algorithms

Reply #6 on: June 04, 2016, 08:25:38 am
Again, thank you very much for those fast and clear replies.
Great idea about the Commodore and Atari games!
I will also continue to check for the pictures in Jinn's repository.
Indeed, the proposed algorithm will not perform as well as an artist (I'm 100% sure of that) and -as you mentioned- hiring would reaaaally be the best option. However, I'm afraid that it's way to expensive for our  short budget (it's a music video about 3/4 mins long).
But when we will be signed to a big major company we will hire a bunch of artists for sure! I would loved to have a pixel-art music video clip, it's really beautifull :)

Offline 32

  • 0011
  • **
  • Posts: 535
  • Karma: +1/-0
    • @AngusDoolan
    • http://pixeljoint.com/p/19827.htm
    • View Profile

Re: Pixel Art with supervised learning algorithms

Reply #7 on: June 04, 2016, 08:30:48 am
I mean hire some artists to recreate some photos for you to use. Maybe even scenes from the music video which would probably give you the best results if that's specifically what you're using this for

Offline Grasshoper

  • 0001
  • *
  • Posts: 4
  • Karma: +0/-0
    • View Profile

Re: Pixel Art with supervised learning algorithms

Reply #8 on: June 05, 2016, 07:11:37 pm
I mean hire some artists to recreate some photos for you to use. Maybe even scenes from the music video which would probably give you the best results if that's specifically what you're using this for

Oh! I get it. Well I think it would be one of the best option.
Meanwhile, I just figured that I can possibly train a network directly on local area of pixel-art pictures. I gathered in-game screenshots from Bitmap Brothers (Xenon2, Gods, Magic Pockets) and from other games. I will try to apply this to an image and see if it will succeed. I have little hope, but why not?
Anyway, I will let you know about the results  :)

Offline hapiel

  • 0010
  • *
  • Posts: 144
  • Karma: +0/-0
  • PJ mod
    • hapiel
    • http://pixeljoint.com/p/9092.htm
    • View Profile

Re: Pixel Art with supervised learning algorithms

Reply #9 on: June 18, 2016, 08:10:40 pm
These two works by Cure were traced from his own paintings, links to the originals are in the description:
http://pixeljoint.com/pixelart/78167.htm
http://pixeljoint.com/pixelart/78883.htm

Offline nessx007

  • 0001
  • *
  • Posts: 40
  • Karma: +0/-0
    • medleystudio
    • medleystudio
    • View Profile

Re: Pixel Art with supervised learning algorithms

Reply #10 on: June 19, 2016, 08:23:56 am
Fascinated by the concept! Machine-learning-anything always grabs my interest. Not sure I can offer much help (and it might be too late anyway) but I remembered seeing this guy's shader for Blender that converts 3D models to pixel art:

http://www.cgchannel.com/2015/02/this-free-blender-shader-turns-3d-models-into-pixel-art/

Sure it's not exactly "photographs" but it would provide some kind of streamlined method of feeding in full-res textured models and rendering images with the pixelation shader toggled on and off to create the before-and-after photos you'd need.

There's another guy who seems to have mastered the hard lines of pixel art being rendered live in his game engine here:

https://www.youtube.com/watch?v=h5SjwrJen3o

That sort of defined line work around objects is one of the tougher things to simulate with automation (I'd imagine), but again using the same method above (assuming you had access to whatever engine this developer is using, and how he's pulling it off) you could render images from 3D modeled scenes in full res, and then again with the pixelation shader. Might be just enough to guide the machine learning along, but hey what do I know!

Good luck and be sure to post again if you do manage to come up with something!
Hope this little blurb finds you well! We've got a fun little series documenting our progress on our SNES puzzle throwback game here:
https://www.youtube.com/watch?v=NomLd6Wcbro
Or check out our game art on tumblr:
http://medleystudio.tumblr.com