It's not transphobic. You're overusing that word, and you're hurting OUR cause.
It is not transphobic to recognize an actual difference. It is not transphobic to want to find solutions that work well for everybody. And it is not transphobic to recognize that people born a male, who go through puberty a male, develop a body different than that of a female. And those differences absolutely can make a difference in sports. This is 100% a fact. Hormone suppression aside, men are taller, on average. Men weigh more, on average. Men are stronger, on average. These are facts, indisputable. Stating as much is not transphobic.
The reason you've probably only had 1 transgender person when an olympic medal is because there aren't a lot of transgender people, and winning an olympic medal is fucking hard.
The vast majority of debate from Republicans who want to ban transgender people from anything, yes. I agree.
But there are plenty of liberal minded, LGBTQ supporting individuals who recognize that at the highest level of sports, being more a male and going through puberty as a male, is likely to give you advantages over female competitors. And at the highest level of sports, it seems fair to control for that.
If we are talking about children playing sports with no real consequence. Everyone should play. I'd say only at the collegiate, olympic and professional levels should we be concerned about fairness.