I think the simplest way to sum up the whole issue is that AI really is, exactly as the name indicates, "Artificial Intelligence."
But the important bit thing isn't that it's artificial in the sense of "man-made." It's that it's artificial in the sense of "fake."
It's fake intelligence. Pretend intelligence. The illusion of intelligence. A superficial resemblance to intelligence.
And yes, to the degree that it's treated as real intelligence, it can't help but make the world dumber, in the exact same way that all fakes dilute the real thing.