this post was submitted on 03 Mar 2024
6 points (100.0% liked)

Selfhosted

50253 readers
504 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 2 years ago
MODERATORS
ruud@lemmy.world
Loki@lemmy.world
CannaVet@lemmy.world
devve@lemmy.world
HybridSarcasm@lemmy.world
HybridSarcasm@lemmy.hybridsarcasm.xyz
6
Self Hosting Fail (feddit.nl)
submitted 1 year ago by padook@feddit.nl to c/selfhosted@lemmy.world
17 comments fedilink hide all child comments
 

I woke up this morning to a text from my ISP, "There is an outage in your area, we are working to resolve the issue"

I laugh, this is what I live for! Almost all of my services are self hosted, I'm barely going to notice the difference!

Wrong.

When the internet went out, the power also went out for a few seconds. Four small computers host all of my services. Of those, one shutdown, and three rebooted. Of the three that ugly rebooted some services came back online, some didn't.

30 minutes later, ISP sends out the text that service is back online.

2 hours later I'm still finding down services on my network.

Moral of the story: A UPS has moved to the top of the shopping list! Any suggestions??

you are viewing a single comment's thread
view the rest of the comments
[–] Kuinox@lemmy.world 4 points 1 year ago* (last edited 1 year ago) (3 children)

When you are bored, backup a VM then hard kill it and see if it manage to restart properly.
Software should be able to recover from that.
If it doesn't, troubleshoot.

[–] Deebster@programming.dev 3 points 1 year ago

That reminds me of Netflix's Chaos Monkey (basically in office hours this tool will randomly kill stuff).

[–] BlackAura@lemmy.world 1 points 1 year ago

When I built my home server this is what I did with all VMs. Learned how to change the start up delay time in esxi and ensured everything came back online with no issues from a cold built.

Rip VMware.

[–] pezhore@lemmy.ml 0 points 1 year ago* (last edited 1 year ago) (1 children)

While I appreciate the sentiment, most traditional VMs do not like to have their power killed (especially non-journaling file systems).

Even crash consistent applications can be impacted if the underlying host fs is affected by power loss.

I do think that backup are a valid suggestion here, provided that the backup is an interrupted by a power surge or loss.

Edit: even journaling file systems aren't a magic bullet. I've had an ext4 fs get corrupted when IO was interrupted by power loss. I get the down votes for mentioning non-journaling FS, but seriously folks, use the swiss cheese method of protecting your stuff... backups, redundant power/UPS, documented/automated installation/configuration.

[–] taladar@sh.itjust.works 1 points 1 year ago

most traditional VMs do not like to have their power killed (especially non-journaling file systems).

Why are you using a non-journaling file system in 2024 when those were common 10+ years ago?