Critique => Devlogs & Projects => Topic started by: Indigo on June 11, 2018, 06:01:15 am

Title: Index Painter
Post by: Indigo on June 11, 2018, 06:01:15 am
So it's been 4 years since I wrote about my HD Index Painting technique (http://danfessler.com/blog) and there was a huge amount of interest - more than I could have imagined (over 73,000 notes on tumblr alone!).  Generally speaking, people were excited at the possibilities it presented, but the common feedback I kept receiving was it's simply too complicated and cumbersome to use in practice for most people, not to mention it required expensive professional software to even try.

The idea of making a dedicated app for this process started pretty much immediately after publishing my article, but at the time, I determined it wouldn't be a viable endeavor.  I was deep in the midst of working on Chasm on top of my day job, leaving me with very little spare time.  And even if I had more time, the vast majority of work would be in recreating basic photoshop-like features such as their excellent dynamic brushes, as well as the table-steaks of any good image editing software such as undo-redo, layers, selection, transformations, etc - all before even starting to work on the pixel art aspects at all.  I wasn't sure how to approach making it with my limited programming knowledge, and it wasn't even clear to me how large the market would be for a specialized tool such as this.

Quite a lot has changed in that 4 years that has made me reconsider the idea.  Chasm is near complete and will soon ship, meaning my free time is about to become more open.  More and more indie games have adopted the technique and it's been wonderful and fascinating to see how they've applied it and hearing their success stories of how it benefited their speed and workflow.  It's become clear that if such a tool were to be available, there would be a decent market for it.  I've also become a more competent programmer since then, getting more experienced with next-gen web technologies such as node and react, as well as graphics pipelines/shaders which would be necessary for this project.

So here we go!

I'm building a dedicated HD Index Painting application called Index Painter (working title) built in Javascript using Node, React, and WebGL, and it'll be bundled as a stand-alone desktop application using Electron.

Recently there's been a resurgence of new pixel art tools in development, and some extremely excellent newcomers such as aseprite and pyxel edit.  It's important to me that this project doesn't become just more noise in the market.  As such, it has the following design goals:

1) Index-painting focused
It doesn't need to compete with other pixel art applications, nor should it.  Instead it should focus on what it has to offer which is completely unique and unavailable in other programs - namely the ability to HD index paint.

2) Simple workflow
The complicated layer stack that Photoshop required was obviously too much hassle for most, but *any* workflow that deviates too far from standard pixel art workflow will likely alienate and push out potential users.  Ideally the workflow should feel completely comfortable to traditional pixel artists as well as digital painters and shouldn't involve convoluted steps or modes to switch between.  If you want to pixel traditionally, grab the 1 pixel pencil tool.  If you want to index paint, grab the soft brush. simple as that. There should be zero friction transitioning between the two, instead, it should feel like one seamless process.

3) Familiar/flexible user interface
Too often, pixel art applications either re-invent the wheel or are stuck in archaic paradigms when it comes to UI/UX. It's important to me that this tool adopts modern standards and feels familiar and easy to approach for new users.  The interface needs to be streamlined and unopinionated; adapting to your preferences and workflow.

In this stage of development, I will be pushing preview builds up to the web to get early feedback, but at some point I will likely transition to a pay-what you want beta phase with a downloadable build.

The preview will be pushed here continually:
Title: Re: Index Painter
Post by: Indigo on June 11, 2018, 08:11:06 am
Current progress:

Some time ago I began doing some perf tests of the HTML5 2D Canvas to see if it was a viable option for making a performant graphics application. I wouldn't be worried if I were making a traditional pixel art application where you only need to directly draw pixels to the screen, but my requirements were far greater.  In addition to direct pixel blitting, I needed to handle not just soft brushes and blend modes, but also redrawing the entire canvas twice for any canvas mutation; once for the change itself (brush stroke or whatever), and again to re-index it into pixel art which can be quite expensive.

I made a test program that generates a soft brush with a gradient and simulates a low-opacity brush stroke moving around the canvas which builds up over time.  I optimized the indexed canvas by only recalculating any "dirty" part of the canvas that has changed.

Indexed blit test in Chrome (left) and firefox (right) with HD data on top, and their respective indexed version below
(https://i.imgur.com/QdbkTdf.gif) (https://i.imgur.com/t3IyFWi.gif)
live demo: http://danfessler.com/projects/blit/

I was pretty satisfied with the results.  With some optimization, the performance was acceptable with limited brush sizes, however, it was apparent it would be very annoying to keep the two canvases synchronized.  Unfortunately, there were some inconsistent artifacts in different browsers.  For some reason, chrome was dithering my canvas!  Although that definitely is an intended feature down the road, I want it to be user controlled.  The issue turned out to be with the generated gradient the brush was using.  Chrome dithers gradients to give a smoother appearance and there is no way of turning it off.  I couldn't use static images for brushes because they need to be dynamic (size, softness, angle, etc).  The only way around this is to generate the brush pixel-by-pixel, which would be yet another convoluted step to maintain and performance hit.  This was getting close to a deal breaker.

in parallel, I was also developing a UI-framework, written in React, that I could leverage for various projects and tools that was largely inspired by Adobe.  I even open sourced a small piece of it called React-PanelGroup (https://github.com/DanFessler/react-panelgroup) which has been fairly successful, seeing over 1000 downloads per week and has even been picked up by Facebook.

Screen Shot of React-based UI framework

I decided to combine my learnings from these two experiments and begin writing the app - this time using WebGL for canvas drawing.  I'd never used WebGL before, but I've gotten fairly familiar with 3d graphics pipelines at work with Unity and Unreal so the transition wasn't too bad.  WebGL is definitely FAR more verbose. Something that used to take 10 lines can easily take over 100.  But in lieu of being terse, it makes up for it in raw power and control.  No longer am I subject to the whims of each browser implementation, I'm talking directly to the graphics card and can control every pixel exactly how I desire.

I reimplemented photoshop's basic brush settings (size, flow, hardness, spacing, roundness, and angle) with the exception of opacity as that would require a two-buffer setup.  Immediately it was clear that WebGL was the answer.  The performance was insanely good.  It felt desktop-class.

First WebGL painting test with photoshop-style brush settings including pen pressure:

Despite the performance boost, If the mouse moved too quick there wouldn't be enough data to draw a smooth brush stroke and would end up with a hard-angled line, so I used a Hermite spline to interpolate between mouse samples.  This required that the brush stroke would unavoidably lag behind the brush by 1-frame, but it was a worthy tradeoff.  It was particularly tricky to maintain the "spacing" setting which controls the pixel distance between each stamp of the brush as there's no way to walk a Hermite spline a specified unit length.  With the advice from some fine folks at a local Hype O'Clock game-dev meetup, the solution was to break the spline into a series of linear segments, and then walk them traditionally.

Brush stroke before and after Hermite interpolation:
(https://i.imgur.com/j7HU0on.gif) (https://i.imgur.com/vWjLRLB.gif)

This past couple weeks has been spent polishing the UI, including tab switching, styling, text-inputs etc.  I also finished the widget context menu which supports actions, radio-select, and multi-select options

Title: Re: Index Painter
Post by: Kellawgs on June 11, 2018, 08:45:52 am
Hello Dan, I am eager to follow you on this journey as you innovate the next logical step in the evolution of modern pixel art. I personally believe that in the future as human beings become increasingly integrated with technology it will be most optimal for artificial intelligence to contrive the images and code directly from our minds. Until then, I am confident this tool will allow users to conceptualize their imaginations more now than ever.

I am privileged to have first seen the workings of this on your twitch streams, and intend to support you as this goes on.
Title: Re: Index Painter
Post by: Pawige on June 11, 2018, 05:24:22 pm
Coming out of a very long hibernation to say this looks amazing!  :y:
Title: Re: Index Painter
Post by: Indigo on June 11, 2018, 06:57:25 pm
Thanks guys!  Holy crapper Pawige, it sure has been a long time.  Missed yo' face