To me, graphics are only necessary to the level they are enough to convey the basic abstract meaning to the game mechanics, but for that they must be used properly. It's all about matching multiple contexts together.
For example, a mere triangle in Asteroids was enough for a spaceship, but only because the game responds to the user commands by moving it like a spaceship. If it moved like a car, the triangle would suddenly be considered a car. A square can be a bullet if it behaves like a bullet, but it could also be the free roaming hero, like in Atari's Adventure. Audio comes in handy here giving us a familiar contexts (explosions, bangs, crashes, splashes), but even the audio doesn't need to be very accurate.
But this is just game mechanics, in my opinion, which is rather bare in any meaning or purpose. Gameplay, at least to me, is the reason behind those chosen mechanics. If you just want to create a metaphor for life (like in Passage), a exploration-friendly fantasy world (Sword and Sworcery) or an action-adventure platformer, the mechanics will just be picked to convey that message. All the extra graphics, sounds, music and storylines are just there to deepen that purpose, to give it some extra meaning and maybe some other message. Gameplay is just the motivation for interaction, and "art games" try explore just that.
So what defines a good game to someone depends more of what level of concrete-ness they expect and understand from that certain gameplay, and if any of the features are too abstract or lacking, they'll feel like they're missing something. If you just can't imagine the square being your hero, then you won't feel motivated to be the square.
