I think a lot of people fail to reconcile this in their minds because of how massive and destructive European colonization and imperialism of the early modern era was - and how relevant it still is. Abuses tend to spring from power imbalances, and the power imbalance between the increasingly-industrial European colonial powers and those they brutalized was massive. It was a horrific crime and and evil that reverberates in both the colonizer and the colonized to this day.
But like... the Kingdom of Largeland, 1300 AD, conquering the Kingdom of Smallland, 1300 AD? That kind of imperialism can go either way. There's no massive technological imbalance that can't be remedied by adaptation or allegiance.
Fundamentally, the question is "Are the PEOPLE in the polity better off or worse off, morally, than they were under previous elites? Are they more free? More equal? More democratic? More socially mobile? Etc" And in the case of early modern imperialism, which is what most people are familiar with, in almost all cases the answer is "No", and often with a side serving of literal genocide.
It would be extremely difficult to argue that the Scramble For Africa, for example, was justified because a small proportion of people in a small proportion of the polities brutalized were better off. A handful of slaves in a handful of regional systems that could be argued to be worse than the repugnant system of colonial 'corvee' and extraction installed by the European powers does not justify the subjugation of the other 98% of people to a status of near-slavery. And nothing could justify the Belgian Congo.
But when Cyrus of Persia conquered the Neo-Babylonian Empire and was welcomed as a liberator, was it because he had mind altering powers? No. You could argue that it was a case of "Victors write the history" and that we're just not getting the full story, but in that case, it would have to be "And the losers choose to believe it" in an era when oral transmission was more common than the written word. Even hundreds of years later Cyrus was celebrated by the conquered as a positive figure. Being ruled by a foreign elite is not always worse than being ruled by a local elite - and oftentimes, in an era before strong senses of nationalism, anyone who's more than a day's walk away is a foreign elite anyway. When given the choice between two foreign rulers, the tendency is to prefer the better of the two.
In the past, it's shitty polities against largely equally shitty polities, and the question then becomes "Which rule is less shitty for the people whose status is potentially changing?" Since the imperial core of pre-modern polities do not enjoy massive military advantages, man-for-man, over their periphery, the tendency is for stable and successful polities to make some form of reasonable accommodation with the conquered, which can mean things get somewhat worse, somewhat better, or stay effectively the same after imperialist conquest.
... of course, sometimes the result is the same as early modern imperialism - degradation and genocide. But it's less certain that that is always the outcome of one polity conquering another without the context of the advantages of Europe over distant locales in the early modern period.
