You might have better luck with Jellyfin, than Plex. Plex uses online authentication tools, which is used for not just user, but server management. In contrast, Jellyfin can be ran completely locally.
Now one thing to note is that neither solution will properly detect your media files properly. You’d need to manually input file details. Usually these servers would do a quick online search, to detect that your movie is what it is. You could import this data, but you’d need an internet connection to acquire it. If you do not mind all that busy work, then you should be fine.
Now the remote… honestly, no idea. I’m pretty sure Android TV has a button remapper app, which might help… Do modern Chromecasts use Android TV? I haven’t used them since their second generation. Best do some research yourself, or wait for another reply.