I only found out within the last couple of years that it’s considered a slur
No. It was always a slur. It was still a slur when we were younger, we were just too immature to understand that. It just means that we grew up and learned that it's denigrating and (most of us) stopped using it once we grasped that.
They're not properly referred to by that word. They USED to be referred to by that word back when they (falsely) believed that such disorders made them inferior.
Classifying those who were mentally less-developed as "retarded" became the excuse for a world that would try all types of monstrous experiments, ostensibly in an attempt to "heal them" because this was they only way "those people" could be a use to society; as lab rats for things like lobotomies, electro-shock therapy and any other wacky insanity that the early 20th century had dreamed up.
In effect, while "retarded" was once the so-called medical terminology it very quickly became shorthand for an entire group of people that could be dismissed and used because they'd serve no other purpose otherwise.