A banana let's say then, will be yellow when light hits it. If that light is also yellow, awesome. If it's not yellow, the banana's colour will be the colour of the light mixed with yellow. I think this is pretty much correct.

Sorry, but that's pretty much wrong, actually.  For example, if a pure yellow object (meaning no "blue wavelengths") gets hit by pure blue light (only blue wavelengths) and nothing else, the object will appear black.  This is because all the blue light (or more correctly, all the wavelengths of the blue light) was absorbed by the yellow object, so there's nothing left to reflect. If a pure yellow object gets hit by a pure purple light in stead, the object will appear red, because the blue wavelengths got absorbed, but the red ones got reflected.

The effect is pretty much like what you get if you create two layers in an image editor, draw the object in the bottom layer and the color of the light in the top layer, and then set the top layer combine mode to multiply.