irmadlad

So, right out the gate, I realize Watchtower has been abandoned. I get that. Still, it's a great app. 95% of the time, it works every time.

The issue I have is as follows:

INFO[93549] Found new searxng/searxng:latest image (5613a99caf5d) 
INFO[93565] Found new prom/prometheus:latest image (b5239da13ab0) 
INFO[93584] Found new ghcr.io/karakeep-app/karakeep:latest image (c5fcb1a653c6) 
INFO[93592] Stopping /CADVISOR-PROMETHEUS (342680d2a684) with SIGTERM 
INFO[93593] Creating /CADVISOR-PROMETHEUS                
INFO[93594] Stopping /SEARXNG (0842827f17a9) with SIGTERM 
INFO[93597] Creating /SEARXNG                            
ERRO[93597] Error response from daemon: the container-wide MAC address must match the endpoint-specific MAC address for the main network, or be left empty 
INFO[93597] Removing image 9e24e9a9806c                  
INFO[93597] Session done                                  Failed=1 Scanned=38 Updated=2 notify=no

It doesn't do this everytime, and it's not one particular container. It seems random. For the example of SearxNG, I have not specified a MAC address.

docker run -d \
  --name searxng \
  -p 8989:8080 \
  -v /srv/docker/searxng/data:/etc/searxng \
  -e SEARXNG_BASE_URL="https://" \
  -e SEARXNG_INSTANCE_NAME="SearxNG" \
  -e SEARXNG_CONTACT_INFO="mail.com" \
  -e SEARXNG_LANGUAGE="en-US" \
  -e SEARXNG_AUTOCOMPLETE="duckduckgo" \
  -e SEARXNG_THEME="simple" \
  -e SEARXNG_OUTGOING_METHOD="default" \
  -e SEARXNG_ENABLE_METRICS="true" \
  -e SEARXNG_ENABLE_CAPTCHA="false" \
  -e SEARXNG_ENABLE_INFINITE_SCROLL="true" \
  -e SEARXNG_ENABLE_PIWIK_ANALYTICS="false" \
  -e SEARXNG_ENABLE_ADVANCED_SEARCH="true" \
  -e SEARXNG_ENABLE_PRIVATE_RESULTS="true" \
  -e SEARXNG_ENABLE_TORIFICATION="false" \
  -e SEARXNG_ENABLE_HTTPS_EVERYWHERE="true" \
  -e SEARXNG_ENABLE_PROXY="false" \
  -e SEARXNG_ENABLE_PLUGINS="true" \
  searxng/searxng

In fact, I've never specified a specific MAC for any of my containers, and just let Docker do what it do. I'm just wondering if anyone has run up on this issue, and what did you do to remediate it, besides use another updater app? I like Watchtower, but only 95% of the time. LOL

It's not a huge issue spinning the container back up. Takes less than 30 sec. However, that's not how it's supposed to work, so I seek input.

ETA: Just adding this for anyone who is testing the Watchtower fork or are thinking about it. Day 2, and everything is running just fine. I have not had any of the issues that I had with OG Watchtower:

time="2025-07-01T17:28:21Z" level=info msg="Found new netdata/netdata:edge image (7cbd870f51b0)"
time="2025-07-01T17:28:42Z" level=info msg="Found new influxdb:2 image (b4355848b856)"
time="2025-07-01T17:28:54Z" level=info msg="Found new ghcr.io/karakeep-app/karakeep:latest image (c5fcb1a653c6)"
time="2025-07-01T17:29:03Z" level=info msg="Found new redis:latest image (ed3a2af6d0d4)"
time="2025-07-01T17:29:04Z" level=info msg="Stopping /CADVISOR-REDIS (84a28cf8904d) with SIGTERM"
time="2025-07-01T17:29:06Z" level=info msg="Stopping /ROOT-INFLUXDB2-1 (a14e51a1b7db) with SIGTERM"
time="2025-07-01T17:29:07Z" level=info msg="Stopping /NETDATA (bde1241c0073) with SIGTERM"
time="2025-07-01T17:29:09Z" level=info msg="Creating /NETDATA"
time="2025-07-01T17:29:09Z" level=info msg="Creating /ROOT-INFLUXDB2-1"
time="2025-07-01T17:29:10Z" level=info msg="Creating /CADVISOR-REDIS"
time="2025-07-01T17:29:10Z" level=info msg="Session done" Failed=0 Scanned=38 Updated=4 notify=no
time="2025-07-02T17:28:07Z" level=info msg="Found new openlistteam/openlist:latest image (0926eda3de9a)"
time="2025-07-02T17:28:23Z" level=info msg="Found new portainer/portainer-ce:latest image (71de3839351a)"
time="2025-07-02T17:28:32Z" level=info msg="Found new ghcr.io/karakeep-app/karakeep:latest image (c5fcb1a653c6)"
time="2025-07-02T17:28:57Z" level=info msg="Found new bigbeartechworld/big-bear-btop:latest image (737e9af25da3)"
time="2025-07-02T17:29:01Z" level=info msg="Stopping /BTOP-PLUS-PLUS (751299b8b022) with SIGTERM"
time="2025-07-02T17:29:04Z" level=info msg="Stopping /PORTAINER (aa754cb14780) with SIGTERM"
time="2025-07-02T17:29:05Z" level=info msg="Stopping /OPENLIST (2ca56a872944) with SIGTERM"
time="2025-07-02T17:29:06Z" level=info msg="Creating /OPENLIST"
time="2025-07-02T17:29:06Z" level=info msg="Creating /PORTAINER"
time="2025-07-02T17:29:07Z" level=info msg="Creating /BTOP-PLUS-PLUS"
time="2025-07-02T17:29:08Z" level=info msg="Session done" Failed=0 Scanned=38 Updated=4 notify=no

So, recently I spun up cAdvisor to provide some metrics for the Grafana dashboard. I created both the docker-compose.yml and prometheus.yml thusly:

prometheus.yml:

scrape_configs:
- job_name: cadvisor
  scrape_interval: 5s
  static_configs:
  - targets:
    - cadvisor:8080

docker-compose.yml

services:
  prometheus:
    image: prom/prometheus:latest
    container_name: prometheus
    ports:
    - 9090:9090
    command:
    - --config.file=/etc/prometheus/prometheus.yml
    volumes:
    - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
    depends_on:
    - cadvisor
  cadvisor:
    image: gcr.io/cadvisor/cadvisor:latest
    container_name: cadvisor
    ports:
    - 8080:8080
    volumes:
    - /:/rootfs:ro
    - /var/run:/var/run:rw
    - /sys:/sys:ro
    - /var/lib/docker/:/var/lib/docker:ro
    depends_on:
    - redis
  redis:
    image: redis:latest
    container_name: redis
    ports:

- 6379:6379

Placed them both in /tmp/cadvisor/ and ran docker compose up. All well and good, got some metrics to feed Grafana and all would seem jippity jippity.

Next day I notice Prometheus is off line. Hmm, check everything out. Logs complaining of a missing prometheus.yml. On a hunch I recreated the above prometheus.yml and placed it back in /tmp/cadvisor/, restart Prometheus, and it fires right up no runs, no drips, no errors. Before I uploaded the new prometheus.yml, I notice that there is a directory now named prometheus.yml in /tmp/cadvisor/, which is empty. Deleted it.

Next day, same scenario. Missing prometheus.yml, directory called prometheus.yml in /tmp/cadvisor/. I thought well, if it's getting deleted, change the permissions, and continued my daily affairs.

Today, same exact scenario. So, wtf, over? Run some commands:

stat /tmp/cadvisor/prometheus.yml
sudo lsof /tmp/cadvisor/prometheus.yml
grep "delete" /var/log/syslog

I can see that the file IS being deleted, but I cannot seem to trace down what is deleting it. It's like there is a cron job that fires off every day at a certain time and deletes prometheus.yml, and in it's place, creates a directory called prometheus.yml effectively taking Prometheus offline. I have no such cron job tho.

Any ideas? Suggestions? Ancient wizardry? Any mystical incantations or tomes to consult?

I FINALLY figured out how to drive the Grafana bus. I know, I know...pretty basic stuff. In the past I always drooled over dialed out dashboards I've seen. Charts, graphs, readouts, dials...but I never could quite wrap my head around it all.

Well, now I have charts, graphs, readouts, logs, dials, and it feels pretty good to have finally learned something new that I've been whacking away at for a while. The plus side is that the whole Graphana+Promtail+Loki stack sips lightly on my resources, whereas in the past, things like the ELK stack, Opensearch, Graylog, etc, really devoured a lot of resources. I really tried with those but just didn't like how ravenous of a RAM appetite they had.

I'm still using lnav. It's quick and dirty and gets the job done for logs. But if I want to look at data, with nice a prettyfied interface, I go look at Grafana. So now the process will be to, dial out and graph every last little thing until I suffer from info overload, and then ease it back to just what is necessary. It's a fucked up process, but it's how I do.

As my lady friend is apt to say: It's the little things.....

ETA: I have cAdvisor rockin' plus influxDB, and the the thing is only sipping like max 2 GB. I've got logs, dials, charts n' graphs. I got bitches in the living room gettin' it on....This is amazing for such little processing power.

I'm looking for a knowledge management system, or at least I think I am. Scrolling around in a notepad ++ of more than 300k lines gets to be a chore. Yeah, I document just about everything I do. They say that we never really forget anything, and that it's our faulty recall system. Well, my recall system is shit. While Notepad++ does allow searching, I guess I'm looking for something a bit more elegant.

I'm looking for something I can dump my notes into a database and be able to search them for a particular command or phrase. I do use ByteStash for all my compose files, but ByteStash doesn't let me search for commands, or command strings like I keep in my notes, or at least I haven't been able to get ByteStash to do that. It's pretty jammy for compose files tho.

Am I asking for too much? Perhaps someone uses something like this for their notes and such or even something entirely different for notes and documentation.

Kind Regards

ETA: Thank you all for your recommendations. I gave each a serious look. Some of the ones like emacs and logseq I downloaded the windows binary to give them a go. So, the winner is Obsidian. It just seems to mesh with my flow. I found a community plugin that encrypts my notes, and I really like that. I also like the fact that you can specify how long you want Obsidian to remember the encryption password, and then revert back to encrypted. Very handy option with the plugin.

Thanks again.

So, this has always bugged me. How do you validate a Docker container? No one wants to pull a laced up container, so there has to be a way one can check. Of course, sticking to original docker containers from Docker Hub would be one method I suppose. Is there some kind of scan one can do? I do this on my Windows computer; scan before installing. Besides looking at code that I would have no idea what is going on, what protocols do you guys use?

Lately, I've been thinking of implementing a secrets management system such as Infiscal, etc. Does anyone use this or something similar like Hashicorp?

How hard would it be to deploy on a pre-existing set up? How does that work? Do you call the required secret in your Docker compose? What makes a secret manager more secure than pulling secrets from an .env file?

Which secret manager is the most popular/better among selfhosters?

In looking for an app to view logs that doesn't require a lot of overhead, I stumbled upon Logwatch. After running it through it's paces, it seems to be pretty capable from docker, fail2ban, to sys logs.

I got to wondering if there are other such log viewers I could try that are in the same genre. Logwatch doesn't greate pretty graphics and dialed out dashboards, but it's fairly quick, I can view from a range of dates and times, and a variety of logs.

I checked out GoAcces, but it seemed geared towards web related logs like webpage hits, etc. With other options requiring elastisearch, databases, etc, they just seemed heavy for my application.

Anyone have any suggestions. So far, Logwatch does what it says on the tin, but I'm curious what others have tried or still use.

ETA: Thanks all for the recommends. I'm still going over a couple of them, but lnav seems like what I'm looking for.

Has anyone here ever used Slimtoolkit or any similar app? I think there are a few of them out there that claim lighter containers and thus faster load times.

I came across it while doing some reading. It intrigues me. Do these apps actually work or is it just SoftRam v2? Is the end result safe for production?

I have never built a Docker container before, however, why make them 'bloated' in the first place?

So, I finally installed Watchtower to auto update my containers. I was a little hesitant because just letting apps auto update kind of makes me a little nervous. Even Windows updates give me bouts of trepidation. Everything went well, there was a little hiccup with Netdata but resolved in less than 5 minutes.

My question is that there are four remaining containers that haven't been updated: Speedtest Tracker, Portainer, Doppler Task, and Dockge.

2025-04-19T06:00:46.510622594Z INFO[38092] Session done                                  Failed=0 Scanned=48 Updated=0 notify=no
2025-04-19T08:00:46.040690535Z INFO[45292] Session done                                  Failed=0 Scanned=48 Updated=0 notify=no
2025-04-19T10:00:45.952863778Z INFO[52492] Session done                                  Failed=0 Scanned=48 Updated=0 notify=no
2025-04-19T12:00:47.755915129Z INFO[59694] Session done                                  Failed=0 Scanned=48 Updated=0 notify=no
2025-04-19T14:00:50.046498408Z INFO[66896] Session done                                  Failed=0 Scanned=48 Updated=0 notify=no

Is this indicative of an issue? Do I just need to update these four manually or will Watchtower eventually update them?

Additionally, has anyone ever had any problems with auto updating? It does make me a bit nervous, however I think I will get used to relying on Watchtower.

I have a couple Docker containers that use email as an alert system or just for info like completed jobs. The server I would like to host them on has, for whatever reason, blocked email ports and you have to pay extra to have them turned on.

It seems to me tho, that I should be able to port all email through Tailscale to a local or even remote email client. For instance, in the case of setting the parameters in the Docker compose, it would look something like this:

SITE_NAME: mycoolwebsite
DEFAULT_FROM_EMAIL: email address
EMAIL_HOST: smtp
EMAIL_HOST_USER: email address
EMAIL_HOST_PASSWORD: email password
EMAIL_PORT: 100.x.x.x:587

Then, configure the local email client to listen on 100.x.x.x:587.

Would this be doable, or is there a better way?

Today I gained a little more knowledge about Caddy, and I thought I'd share in case someone is having the same issue.

I've been biting my nails worrying about Caddy updating certificates. Everything I had read told me not to sweat it. That Caddy had my back and wouldn't let any certs expire. Well, two did, today. So I set about today, after I got all my chores done, to see if I could figure out wtf.

Long story short, I had a inconsistency in the format of my Caddy file. It didn't affect the function of the file to the extent that it would not provide the certificate in daily use, but apparently I confused Caddy enough so that it couldn't determine when certs were expiring, and reissue the cert.

If you run the following:

caddy reload --config /etc/caddy/Caddyfile 

And you get something like this:

2025/04/09 21:49:03.376 WARN    Caddyfile input is not formatted; run 'caddy fmt --overwrite' to fix inconsistencies{"adapter": "caddyfile", "file": "/etc/caddy/Caddyfile", "line": 1}

It's a warning that something is askew. Not to worry tho, you can fix it thusly:

Make a backup assuming etc/caddy/Caddyfile is where your Caddyfile is:

cp /etc/caddy/Caddyfile /etc/caddy/Caddyfile.bak

Next we'll ask Caddy nicely to please reformat in an acceptible form:

sudo caddy fmt --overwrite /etc/caddy/Caddyfile

Trust but verify:

caddy validate --config /etc/caddy/Caddyfile

Now run:

caddy reload --config /etc/caddy/Caddyfile

You should be golden at this point.

Cheers

I am toying with the idea of using one of my Tailscale instances as traditional VPN, using the exit node features. I think I have that part down to a note as far as what has to be done in order for this to happen.

My question is if there are any security risks or security provisions that need to be made to keep the envelope secure. I am the only user of my Tailscale network, so I don't have to worry about another user jacking things up. However, I am concerned about the implications of the visibility of the exit node I would be connecting to.