I'm no pro, I might just have a little bit more experience with pixelling dirt. If you keep working at it, you'll have me beat in no time!
Try to work from dark to light when you're drawing in the tiles, and maybe use a larger brush. Instead of adding shadows, instead start with a solid shadow (your darkest brown), and add in the lighter areas, colour by colour, and avoid the tempration to add in 1px details unless you feel you really need them. It'll be easier to get nice clusters that way (especially if you use a larger brush instead of 1px), and it approximates how light interacts with clumps of dirt - it hits the flat/rounded surfaces, and the cracks between them remain dark.
To make the colour transitions feel more natural, try to think of the dirt clumps as a giant, irregular dither pattern. Don't cut your use of the brightest colour all at the same horizontal level - mix in some highlights lower, some darker clumps higher. Same goes for clump sizes, don't be afraid to have larger clumps lower too, and some more small ones higher.