I totally agree and I always make sure I try without a metronome as much as possible, but often struggle to progress for the reasons given below. (Lifting text from a thread I created a week or so ago). Definitely a mental/physical disconnect. Best I can manage is flip-flopping between metronome and not. Hopefully I can rely on this less and less.
"After warm-up I will get my tremolo going as fast as I can. It feels good, smooth and sounds really fast. I then check against the metronome to find that I was only playing between 160 and 170 bpm. I would then bump the metronome up to 190 or 200 and play it fine and in time.
Its like my mind and body are calibrated to play up to the 160-170 mark in the absence of external stimulus ie. a beat/metronome. For the remainder of a practice session, I could play near enough at 200bpm tremolo without the metronome as a guide, like I was re-calibrated, temporarily at least.
I distinctly remember when working on my tremolo, that I relied on my metronome, but ensured that it was bumped way outside the stringhopping zone and my capabilities at the time."