That is just untrue. References are INTEGRAL to learning! Muscle groups? bone structure? character design details? How on earth will you use that without reference!
At later stages it IS good to move off them so you can train yourself in certain poses, but for beginning stuff it's an INTEGRAL part of learning.
as for the frustration, don't let that fuck you up too bad. I admit I kind of skimmed your rant, but the main points of "I'm grinding my gears and nothing happens" is a very relatable problem.
Try new things! Look into everything from character design to animation to 3d modeling, EVERYTHING you can think of! Use references as liberally as you need!
And ESPECIALLY do the basics. It seems like pointless busywork, but draw cones and circles and cubes until you get them down and you'll have a MUCH easier time with 3d visualization later on.