Hi,
Interesting (and handy!) tool, it reminds me of one I was going to make but haven't gotten around to.
My number 1 recommendation for improvement:
Make the source code available.
Really nothing else could help quality quite as much as that.
Other comments:
* In my opinion, with the default settings, the last three rows should be merged, considering how long the top row is. This may relate to the way you measure hue and saturation.
* You may need to switch to a more accurate colorspace like L*a*b to get really good quality sorting. For instance, try your program on
http://pixelation.wayofthepixel.net/Themes/dark/images/smflogo.gif -- IMO it produces quite confusing results.