I think the most important things you should focus on are increasing contrast, reducing colour count, and focusing on shape over texture. Right now, you're correct in that the beard has too much noise - rather than trying to create a hair texture, think about how the light from your light source would fall on something with the shapes and curves a beard has. I've done a quick edit of your sailor's face, where I increased contrast, focused on lighting, and made all the shapes a bit larger and blockier to increase readability.