I think a lot of this is definitional. And I think it's important to be clear on definitions and that imperialism and colonialism under capitalism are different. However, I also think things that could be categorized as colonial or imperial existed prior to capitalism.
As Lenin wrote:
Colonial policy and imperialism existed before the latest stage of capitalism, and even before capitalism. Rome, founded on slavery, pursued a colonial policy and practised imperialism.
This doesn't mean the crusades necessarily fall into those categories, but it also doesn't mean we can simply dismiss it.