Labor friends of X and Labor against Y groups really ought to see the writing on the wall and join parties that actually reflect their values.
We have preferential voting, doing so has no downside, unless you are foolishly loyal to a particular party.
The Labor party is a misnomer at this point. It's the "More-friendly-to-labour-than-the-Liberals Party".
P.S. In my opinion, it's not a sensible point of view to be loyal to a party. Support a party, sure, become a member, work for it, as long as it mostly still aligns with your views. If it doesn't, you should leave and join others, or start your own. The Labor party is beyond reform at this point.