Yeah, it's really not that hard, Skyrim just has you crossing a completely imaginary border that probably doesn't even matter to 99% of people at the wrong time when the empire is watching, you could have done it for literally any reason, thats good enough.
Like they even do the amnesia thing in Starfield after you get zapped by the macguffin, an amnesia inducing vehicle accident is like one of the most classic cliches they could have used instead.
It's really frustrating because its such a basic narrative tool to factor in what routes of communication are available, as well as the time and space required.
I remember an early plot point in the Shadowrun SNES game is to try and get ahold of a credit card and a working phone so you can actually call people, because theres a price on your head so every time you step outside theres basically at least one assassin trying to gun you down, which makes it pretty undesirable to try and take the subway everywhere rather than making a phone call.
And then you can instead there take into account that remote communication is unsecure, so perhaps you can't call people sometimes, or you have to use more secure methods that take more time and are limited in scope, there's just a lot you can do when the default form of communication isn't just teleporting to peoples houses to personally chat with them for 1 single minute then teleporting away again.