A complicated topic.
"Paganism" in the sense of "Ancient Greek, Roman, Egyptian or Mesopotamian" is fairly straightforward; in "On the Leading Article in No. 179 of Kölnische Zeitung" Marx wrote:
That with the downfall of the ancient states their religions also disappeared requires no further explanation, for the "true religion" of the ancients was the cult of "their nationality", of their "state".
So in this case Marx would likely argue that their religion is simply "our state; our nation; 'us' as opposed to 'them'". Interestingly, Marx argues that constitutionalists are similarly religious.
"Paganism" in the sense of "animism" and/or indigenous religions generally is more contentious, because Marx wasn't able to access sources on them other than second/thirdhand accounts by bigots. So while Marx in Capital makes claims about 'primitive nature worship of the savage', I'm unsure what he is actually talking about.
Explanations from actual indigenous people about their spirituality/'religion' (e.g. God is Red and Bridging Cultures) have little in common with Marx's depiction. Some (e.g. Frank Black-Elk in "Observations on Marxism and Lakota Tradition") assert that their religion/spirituality is dialectical-er and materialist-er than Marx.
It doesn't come up much yeah. He sorta beats metaphysics to death in the 1840s and then drops the topic as far as I can tell from what Marx-works and biographies I've read.
Ancient societies (actually, anything pre-1500ish) is sorta an unknown for Marx and Engels compared to us.
For the middle ages, they have the Renaissance and Enlightenment "Middle ages suck" circlejerk to rely on, as medieval archival research wasnt nearly as thorough.
For Rome and Greece, they had Roman and Greek histories and mythologies that survived; none of the massive professional archaeological excavations or examinations of inscriptions and surviving archival documents was availible.
Before that, their only source is—The Bible. Unironically; until combined archaeology, linguistics and imperialism enabled access to the ancient tablets and such, the Bible was all there was for stuff that old.