Actually - a lot of closed source programs are still vulnerable to the supply chain attacks you mention where a bad actor has got access to their codebase. This has happened and been reported on, plus I'm sure, plenty of occasions where it was hushed up for reputational reasons. And - much commercial software still uses FOSS dependencies, so is also vulnerable to the same situation you describe for that. Worst of both worlds.
I don't think either system is inherantly better than the other in terms of computer security. Each has different and overlapping vulnerabilities.
Much of the UK's regular government stuff is online and very simple too - their website is actually very good. It doesn't integrate everything though. The health service is particularly fragmented and communication is often by post and not that good.