I know it’s not a very popular or inspiring answer, but in my experience, either genetics or something that’s likely congenital can absolutely be a limiting factor in playing fast. For me, it’s fretting hand ability. I’ve already posted about this before so I won’t belabor the point but I’ve spent years trying to get my fretting hand to behave at speeds higher than 90bpm. At a certain point it just “does the wrong thing” and seemingly no amount of practice has really helped. Even at slower speeds, my fretting hand has always been clunky, imprecise, and slow. Meanwhile, it sounds like most people don’t really struggle with the fretting hand at all until you start getting into crazy territory (> 150bpm at the very least).
Certainly genetics (or whatever other congenital limitations one may have) shouldn’t cause somebody to entirely give up on something they really love, but the trope of “there is no talent, only practice, and if you haven’t figured it out it means you haven’t been practicing hard enough” is very tiring and discouraging to hear. I know that generally people mean well, especially on this forum, but I suspect that those who say things like this haven’t had the experience of actually being really bad at something they were passionate about. It’s fairly crushing.
Coming from the other side, another hand-related hobby I engage in quite a bit is bouldering. I’ve been doing it on and off for over a decade. And for whatever reason… it comes extremely naturally to me. I’m the best climber in my social group and it’s not because I spend an extraordinary amount of time training (I don’t). When I’m on the wall, I often just have an intuition for which moves will work well and how to execute them. And often times once I’m done with a problem, I don’t even remember quite how I did it. My point is, I’m able to do difficult bouldering problems due to a combination of hard work and dedication but also simply because I’m unexplainably good at it through no doing of my own and it would absolutely be a lie to claim otherwise.