Sal

joined 3 years ago
MODERATOR OF
[–] Sal@mander.xyz 3 points 3 months ago (2 children)

It is done now :) Sorry for the delay!

[–] Sal@mander.xyz 1 points 3 months ago (3 children)

Thanks a lot for sharing! It has been nice thinking about this topic again.

When I wrote my response I was hyper-focused on the concept of "antenna-like" resonances due to the wavelength of the radiation, so it was interesting to read about the 500 Hz resonances that I assume are due to natural frequencies of tiny hairs or other oscillators in bees. I did not even consider these.

I have in the past heard about some micro-wave and also IR-assisted chemistry. It does make some intuitive sense that the excitation of particular modes that displace molecules or groups of molecules along a reaction coordinate might help speed up a specific reaction pathway, but I remember a few years ago that the data was not there to support that. I believe a professor at my university was doing some experiments in which the reactants are placed within a cavity that is resonant with such a mode in an attempt to increase reaction rates via some cavity enhancement. From what I can see they have not published on this topic yet, but it seems to be related to this concept: https://pubs.aip.org/aip/jcp/article/154/19/191103/565904

I am very curious about your LoRa bee sensors. What kind of sensors are you using?

[–] Sal@mander.xyz 3 points 3 months ago (1 children)
[–] Sal@mander.xyz 3 points 3 months ago (3 children)

I will upgrade in a few minutes. From what I have read it might take several minutes.

[–] Sal@mander.xyz 3 points 3 months ago

It looks like that account was deleted by someone with access to the account. Most likely by the account owner. When a user deletes their account there is no form to fill in a reason, the data is just wiped.

[–] Sal@mander.xyz 3 points 3 months ago (1 children)

Great, added. Thank you!

[–] Sal@mander.xyz 5 points 3 months ago (2 children)

Awesome, thank you! I have added you as a moderator.

[–] Sal@mander.xyz 2 points 4 months ago (1 children)
[–] Sal@mander.xyz 25 points 4 months ago (6 children)

AAAHHHH 😭 Such a relief. This was stressful.

The websites makes use of Contabo's object storage service for storing and serving images. Earlier during the month, I got an e-mail saying that there would be a 12-hour downtime window due to a migration. Alright, no problem.

A bit before this users reported problems with uploads. I figured this was related to the migration.

After the "migration", images still caused problems due to some "administrative rule" preventing uploads. I sent messages daily to Contabo. At first they were responding that "their engineers were looking into it, that this was related to the migration, and that they had no time estimate". OK, I can be patient....

After a few days I lost some of my patience and pressed them more often, until they started sending me generic messages and eventually saying that "the problems were already fixed" and suggesting it may be some configuration error on my side.

This is where it got stressful, because my knowledge of the implementation of the images back-end is superficial. So... That meant I would need to study the details and either fix it myself or show proof that it was their fault.

These past few days I have tried to do some reading and tested configuration changes, but I am the kind of person that puts too much on their plate and I have been quite busy, so this has been an additional stress point.

Today, images suddenly started working (until it came down again due to some logs piling up). This tells me that it was not a configuration issue and it was with high probability something on Contabo's side that they fixed today. Uploads are working as well now. Contabo support never shared any information about the specifics of what went wrong, so I don't know.

This doesn't mean that I don't need to study the details, but at least now the pressure is lower. My plan now is to do some studying on how images are stored and served and once I have a good understanding I can migrate the instance to a dedicated server. I will also look into techniques for making a server more resilient when things do go wrong.

[–] Sal@mander.xyz 5 points 4 months ago (1 children)

Yes, it is pretty frustrating.

I keep sending them messages every day. Today's reply:

This is connected to the migration and there is no ETA currently.

Not very happy with them. I am looking into how to move the images somewhere else.

[–] Sal@mander.xyz 4 points 4 months ago

Thanks! <3 Cost is not the issue. The reason why I selected this option was because it seemed a good choice at the time. Moving the image database around is a bit of a hassle so I chose to just keep it as it was even after learning of other options. Last time it took several days of downtime to move the images over because the transfer speed into/out of the object storage is very low.

If I would have known that this would happen now, I might have taken this opportunity to move all the images over. But I was not expecting this amount of downtime. Now I worry that trying to move images from one place to another when the object storage is acting up is risky, and also I am busy.

I sent another message to support this morning and received:

We are sorry for the inconvenience caused. Please note that our technicians are working diligently on this matter, there is no ETA.

Sorry 😓

[–] Sal@mander.xyz 3 points 4 months ago
20
submitted 1 year ago* (last edited 1 year ago) by Sal@mander.xyz to c/mander@mander.xyz
 

The instance has been updated to version 0.19.4!

You can see the official release post and notes here: https://lemmy.ml/post/16585359?scrollToComments=true

Since this update required some manual intervention anyway to update the database version, I took the opportunity to perform some overdue server maintenance and upgrades.

I have migrated the server to a more powerful configuration, and changed the way image files are being stored and served (object storage as opposed to filesystem storage).

I am also aware of some federation issues that have affected this and other instances. I am not sure whether this upgrade will deal with these problems automatically. I will be looking into it and hopefully will implement some solution soon.

Sorry for the downtime experienced in the past few days, but hopefully with the more powerful server the experience will be smoother and will remain so for a while.

For other admins and users interested, this is a bit of information about the server


The previous configuration:

Hosting provider: serverspace.io

8 GB RAM

4 Core CPU

50 Mbps network

400 GB SSD

Cost: $55 / month

The 400 GB SSD was being used to store both the server's database and the image files, and this component in isolation had a monthly cost of $36. I have migrated the images to an object storage provider ($6/month for 500 GB), and used the resources previously used for disk space to increase the other parameters. The new configuration is a lot better and less expensive.


New configuration:

Hosting provider: serverspace.io

Object storage provider: contabo.com

16 GB RAM

6 core CPU

50 Mbps network

100 GB SSD

500 GB Object storage

Cost: $47/month

Thanks a lot to the lemmy and pict-rs devs for the hard work they put into developing the infrastructure!

 

I am performing a migration that moves the images files from disk storage to object storage. During this migration the image back-end needs to be turned off, and so images will not be accessible.

It should still be possible to comment.

I am not sure how long this will take.

This is part of a few optimizations that I plan to do before I upgrade the instance to version 0.19.4.

EDIT: Update, June 17 06:08 UTC

The transfer is working, but, after 12 hours, only ~50 GB have been transferred to Contabo's object storage (rate of about 1.16 MB/s). This is a lot slower than I imagined, and, if this rate continues, it will take about 3 more days to transfer the remaining 300 GB.

I am not sure if this slow rate is to be expected for moving files to object storage, or if something is wrong. If any of you has experience with this I would appreciate some tips! I have also asked in the Matrix chat in case someone there can provide me some feedback as well.

EDIT: Update, June 18, 07:44 UTC

Migration is done!

272.16 GB of image data has been transferred to object storage. This took about 37 hours. This means that transfer occurred at a rate of 2 MB/s. From other admin's experience, this transfer was quite slow. I am making use of Contabo as my object storage provider, and I suspect that the speed of the transfer was limited on their end. Another admin reported that they used backblaze b2 as the object storage provider and they experienced much faster transfers. All of this should have no influence on how the site performs from the end-user's point of view, I am documenting this for other admins who learn from other's experiences by searching through lemmy.

Object storage is a much cheaper way to save files in the cloud. Before this, I have stored the image files into the same SSD disk where the instance lives (rented from serverspace.io). The cost of this disk space goes up as the disk space needs increase. At the current disk size of 400 GB renting that space was costing $36/month and increased $0.9/mo for every 10 GB.

Object storage is a specialized way of saving and serving files from the cloud. Object storage is much cheaper, at €6 for 500 GB (from Contabo), and so it is a much more scalable way of storing the instance's files. The only reason I did not do this earlier was because I was not sure about how to do it and kept pushing it back. But now, it's done! I would recommend other admins to do this as early as possible, as migrating many files is more of a hassle than migrating a few.

Istill need to do a few more tasks (moving the instance to a new server with a smaller disk, upgrading the postgres database to be compatible with 0.19.4, changing the pict-rs database format from sled to postgres, and then upgrading to 0.19.4). But first I will let the instance run for a few days with object storage to make sure nothing is breaking.

Thank you for your patience :D

 

This is an ebook about quantum computing made for a general audience. The ebook was made freely available via arXiv by the author Chris Ferrie, who is known for writing science books for children such as the popular Quantum Physics for Babies.

view more: ‹ prev next ›