RoundSparrow

joined 2 years ago
MODERATOR OF
[–] RoundSparrow@lemmy.ml 1 points 2 years ago (3 children)

Why does the concept of a multi-reddit need to extend outside of the user’s instance?

it doesn't need to. But why would you not want it when communities are multi-instance?

perhaps I made a mistake introducing the privacy concern first. As now the whole topic seems to negate the very reason so many people have requested MultiReddit on Lemmy. The privacy issue isn't even essential, I just wanted to have a discussion about it as a general topic. I'm already building the code so that it can be done entirely without anyone sharing their personal subscribed list.

[–] RoundSparrow@lemmy.ml 4 points 2 years ago* (last edited 2 years ago) (1 children)

report a bug to lemmy about the broken quoting.

I have, weeks ago.

Consider posting to the official rust playground and creating a shared link.

I did share a link to GitHub, is that not good enough or something? Here is a screen shot for you.

[–] RoundSparrow@lemmy.ml 34 points 2 years ago* (last edited 2 years ago) (3 children)

Most common cause is people changing their language settings in their profile. It's a daily occurrence. The app really needs to tell people "25 messages not displayed because you are only viewing in Spanish".

[–] RoundSparrow@lemmy.ml 6 points 2 years ago

Recently I’ve noticed my feed has become almost entirely the main meme instance. The algorithm gives me 4 meme posts then a technology post then load more memes

Yes, same issue, and I'm using lemmy-ui...

Lemmy's backend Top/Active/Hot are pretty primitive. I'm experimenting with some ways to weigh smaller less-popular communities... because +20 vote on meme topics is noise, but +20 on some focused community can be a big deal. hot_rank doesn't take that into account and just looks at published date and score. It's pretty tricky to get new things into the backend, so it may be a while.

[–] RoundSparrow@lemmy.ml 2 points 2 years ago* (last edited 2 years ago) (1 children)

I personally would need to dig into testing and code again to give answers with confidence. What I'm trying to say more than anything is... don't assume. The level of mistakes in Lemmy's more technical back-end code are pretty high from my experience, especially when it comes to multiple servers involved (comment deletes not being sent to all servers was a situation I tracked down). What I do know is that there is very little written out there about people actually tearing it apart and showing what works... a lot of stuff gets logged in server logs as errors that almost nobody can explain. Either it's mistakes in apub JSON or other non-lemmy servers, or older versions of Lemmy, etc.

When you say packages get forwarded to whatever instance wanted (if I understand correctly) you don’t “unpack” (e.g check if it’s a valid request)

the pack metaphor isn't that great. But it is signed, and the receiving server checks a signature. But I really have not seen anyone discuss how those signatures are exchanged in the first place, and I've seen people say they re-installed their entire instance - which I assume generates a new set of signing keys for the same domain name.... and I know Lemmy starts with 1 in index for post, comment, person - and would end up generating the same numbers for different content.

I haven't seen much eye towards auditing any of this works, and if it even is a good design. Even 2 months ago there were some aggressive timeouts that were causing delivery to fail. And when something fails, the person who comments or posts doesn't get notified....

There is some deep stuff in lemmy., every community has a private key/public key pair, as does each person - but I'm not even sure that is used at all and was an ambition. I rarely see the topic actually come up and I've been listening for this kind of deeper technical topics... and created !lemmyfederation@lemmy.ml to try and better organize it.

Thanks again, and sorry for the ramblings.

I'm pretty much rambling myself... my repeat point is: don't assume. I would not describe Lemmy as battle-hardened against attacks or spoofing that someone can find to bypass the current logic.

[–] RoundSparrow@lemmy.ml 2 points 2 years ago

I wouldn't trust that assumption, goarmy.com

[–] RoundSparrow@lemmy.ml 1 points 2 years ago* (last edited 2 years ago) (1 children)

It’s still early days

Lemmy has been on GitHub since February 2019, over four years. It isn't new at all. Several instances go way back.

The answer is: ORM.

[–] RoundSparrow@lemmy.ml 3 points 2 years ago* (last edited 2 years ago) (11 children)

Like normal people! :D Go to another instance, if you find something interesting

So "normal people" would go read another instance just to bring a single comment or post over. They may as just well join the other instance. Which is what I see actually happening.... Many of these lemmy accounts are the same person duplicated to route around crashes.

But it’s a little bit sad, that you’ve never done this and only look at all.

If you had any clue what I have been personally doing with lemmy for the past 90 days, you would laugh at yourself. I've been logging in to several different Lemmy servers every single day for months and posting about my observations... such as when a big instance put new user interfaces online, upgrade their backend, crash on their home page, etc.

[–] RoundSparrow@lemmy.ml 1 points 2 years ago (3 children)

i’d hope the query planner can still do something reasonable.

PostgreSQL specifically guards against queries with more than 8 joins... and Lemmy plows right past that.

[–] RoundSparrow@lemmy.ml 3 points 2 years ago* (last edited 2 years ago) (3 children)

Having worked with lemmy_server code for months, I wouldn't trust it in practice. Unless someone produces a pretty through test to validate that it actually works as intended when out-of-band data delivery attempts are made. And for discussion sake, I am assuming comment and posting "messages", and not private messages... which is a whole different set of behaviors.

when a user homed on lemm.ee comments on a post where the community is hosted on lemmy.world, it would get delivered to lemmy.ml by lemmy.world, not the origin lemm.ee instance.

Messages are sent with a digital signature that only the original instance could craft.

I don't think the actual message carries a signature from the origin instance, in this case lemm.ee - I could be wrong, because it may not unpack and repackage it before forwarding to subscribed instance servers. But there are some very short timeouts on these signatures and HTTP connections, and I think it does unpackage it and repackage a public comment message.

I think lemmy.ml would need to go fetch the profile for the then unknown user to be able to get the display name and other details of the lemm.ee comment creator to display on the comment. The signature for the comment I think would actually only validate lemmy.world to lemmy.ml - and if lemmy.ml already has the profile of the lemm.ee user stored locally it may not verify it (and could be an entirely different install with the same username). But I don't think many have ventured into study the code in these areas. The whole process hasn't been examined that much and I wouldn't be surprised at all if there are issues with post and comment getting dropped because servers aren't all reachable at the right times for these steps.

[–] RoundSparrow@lemmy.ml 5 points 2 years ago (13 children)

How exactly do I deactivate the “All Feed” to not pollute my server and avoid distractions to the users, but let them comment on external posts

How would they ever find the external posts to comment on in the first place?

[–] RoundSparrow@lemmy.ml 5 points 2 years ago* (last edited 2 years ago) (1 children)

Personally I think the issue is more that there is blind loyalty to team sports in USA culture, and no matter how many bad things are documented about a specific person (Donald Trump, Richard Nixon)... people are loyal to the image of that person, the brand and logo. People are raised in the USA to be inundated with breakfast cereal and toy company logo/brand recognition. It's a faith system. Breakfast cereals and fast-food "Happy plates" that fund a lot of children's TV are incredibly unhealthy and profit machines - and parents think this is psychologically healthy.

Politics will eventually be replaced by imagery. The politician will be only too happy to abdicate in favor of his image, because the image will be much more powerful than he could ever be. -Marshall McLuhan

 

lemmy.world announcing to public that this is installed on their production server: https://lemmy.world/post/1061471

 

I spent several hours tracing in production (updating the code a dozen times with extra logging) to identify the actual path the lemmy_server code uses for outbound federation of votes to subscribed servers.

Major popular servers, Beehaw, Leemy.world, Lemmy.ml - have a large number of instance servers subscribing to their communities to get copies of every post/comment. Comment votes/likes are the most common activity, and it is proposed that during the PERFORMANCE CRISIS that outbound vote/like sharing be turned off by these overwhelmed servers.

pull request for draft:

https://github.com/LemmyNet/lemmy/compare/main...RocketDerp:lemmy_comment_votes_nofed1:no_federation_of_votes_outbound0

EDIT: LEMMY_SKIP_FEDERATE_VOTES environment variable

 

Grep the lemmy server code for "= LocalSite::read" - and I find that even for a single vote by an end-user, it is doing an SQL query to the local site settings to see if downvotes are disabled.

Can some Rust programmers chime in here? Can we cache this in RAM and not fetch from SQL every time?

PostgreSQL is telling me that the 2nd most run query on my system, which is receiving incoming federation post/comment/votes, is this:

SELECT "local_site"."id", "local_site"."site_id", "local_site"."site_setup", "local_site"."enable_downvotes", "local_site"."enable_nsfw", "local_site"."community_creation_admin_only", "local_site"."require_email_verification", "local_site"."application_question", "local_site"."private_instance", "local_site"."default_theme", "local_site"."default_post_listing_type", "local_site"."legal_information", "local_site"."hide_modlog_mod_names", "local_site"."application_email_admins", "local_site"."slur_filter_regex", "local_site"."actor_name_max_length", "local_site"."federation_enabled", "local_site"."captcha_enabled", "local_site"."captcha_difficulty", "local_site"."published", "local_site"."updated", "local_site"."registration_mode", "local_site"."reports_email_admins" FROM "local_site" LIMIT $1

 

Lemmy Rust code: https://github.com/LemmyNet/lemmy/blob/cb28af508d030683561bdc9dde2299cf818f2c6b/crates/api_common/src/community.rs#L76

Allow directory of communities specific to a particular Lemmy instance, ones already know to the local server.

 

This is a Link to a Lemmy posting, it is suggested you comment on the original posting in the !asklemmy@lemmy.ml community.

 

There are users who are going to upload the same reaction images and content to comments all over the place. And those users will be on their same home server. Increases performance for all the people reading the comments. Reduces storage for the server operator at the expense of some extra calculations.

 

Comment number 1 million: https://lemmy.ml/comment/1000000

Beehaw.org has 506483 comment IDs, Lemmy.world has 631495 comment IDs, 604565 on feddit.de instance, 636812 on lemmygrad.ml instance, 504751 on lemm.ee instance, 544522 on the lemmy.one instance. This is significant to performance of Lemmy, how much data is in the PostgreSQL backend.

 

Here is from 5 days ago:

https://lemmy.ml/post/1494755

And I did a post just now in test to the same news link:

https://lemmy.ml/post/1607677

Spot checking recent ones, they all seem to no longer fetch the summary preview? Although lemmy.ml has been running 0.18.0 more than 5 days, so some other change?

 

Specifically the database backend. Is the polling for notifications causing more database load? From my personal testing, lemmy.ml has the same performance problems with 0.17.4 that it does with 0.18.1 and I haven't seen anything in the code changes that are that significant with database.

 

Ok, so I have some code to crawl a posting of a community and compare two servers for comments missing. It looks bad today. Both of these servers are version 0.18.0 and have been upgraded for several days.

missing 0 unequal 0 11 on https://lemmy.ml/ vs. 11 on https://sh.itjust.works/
missing 35 unequal 1 48 on https://lemmy.ml/ vs. 14 on https://sh.itjust.works/
missing 4 unequal 0 9 on https://lemmy.ml/ vs. 5 on https://sh.itjust.works/
missing 6 unequal 0 9 on https://lemmy.ml/ vs. 3 on https://sh.itjust.works/
missing 1 unequal 0 1 on https://lemmy.ml/ vs. 0 on https://sh.itjust.works/
missing 6 unequal 0 12 on https://lemmy.ml/ vs. 6 on https://sh.itjust.works/
missing 3 unequal 0 8 on https://lemmy.ml/ vs. 5 on https://sh.itjust.works/
missing 3 unequal 0 6 on https://lemmy.ml/ vs. 4 on https://sh.itjust.works/
missing 22 unequal 0 42 on https://lemmy.ml/ vs. 20 on https://sh.itjust.works/
missing 5 unequal 0 15 on https://lemmy.ml/ vs. 10 on https://sh.itjust.works/
missing 8 unequal 2 17 on https://lemmy.ml/ vs. 9 on https://sh.itjust.works/
missing 3 unequal 0 3 on https://lemmy.ml/ vs. 0 on https://sh.itjust.works/
missing 0 unequal 0 10 on https://lemmy.ml/ vs. 10 on https://sh.itjust.works/
missing 11 unequal 0 24 on https://lemmy.ml/ vs. 13 on https://sh.itjust.works/
missing 1 unequal 0 2 on https://lemmy.ml/ vs. 1 on https://sh.itjust.works/
missing 13 unequal 0 37 on https://lemmy.ml/ vs. 24 on https://sh.itjust.works/
missing 3 unequal 0 7 on https://lemmy.ml/ vs. 4 on https://sh.itjust.works/
missing 0 unequal 0 10 on https://lemmy.ml/ vs. 10 on https://sh.itjust.works/
missing 60 unequal 2 186 on https://lemmy.ml/ vs. 126 on https://sh.itjust.works/
missing 10 unequal 2 51 on https://lemmy.ml/ vs. 41 on https://sh.itjust.works/
missing 16 unequal 0 51 on https://lemmy.ml/ vs. 36 on https://sh.itjust.works/
missing 31 unequal 3 128 on https://lemmy.ml/ vs. 97 on https://sh.itjust.works/
missing 0 unequal 0 4 on https://lemmy.ml/ vs. 4 on https://sh.itjust.works/
missing 2 unequal 0 5 on https://lemmy.ml/ vs. 3 on https://sh.itjust.works/
missing 15 unequal 1 67 on https://lemmy.ml/ vs. 52 on https://sh.itjust.works/
missing 4 unequal 0 53 on https://lemmy.ml/ vs. 49 on https://sh.itjust.works/
missing 0 unequal 0 5 on https://lemmy.ml/ vs. 5 on https://sh.itjust.works/
missing 0 unequal 0 0 on https://lemmy.ml/ vs. 0 on https://sh.itjust.works/
missing 1 unequal 0 19 on https://lemmy.ml/ vs. 18 on https://sh.itjust.works/
missing 0 unequal 0 2 on https://lemmy.ml/ vs. 2 on https://sh.itjust.works/
missing 0 unequal 0 22 on https://lemmy.ml/ vs. 22 on https://sh.itjust.works/
missing 0 unequal 0 16 on https://lemmy.ml/ vs. 18 on https://sh.itjust.works/
missing 0 unequal 0 7 on https://lemmy.ml/ vs. 7 on https://sh.itjust.works/
missing 3 unequal 0 27 on https://lemmy.ml/ vs. 24 on https://sh.itjust.works/
missing 2 unequal 0 32 on https://lemmy.ml/ vs. 30 on https://sh.itjust.works/
missing 3 unequal 0 21 on https://lemmy.ml/ vs. 18 on https://sh.itjust.works/
missing 3 unequal 1 16 on https://lemmy.ml/ vs. 13 on https://sh.itjust.works/
missing 3 unequal 1 47 on https://lemmy.ml/ vs. 44 on https://sh.itjust.works/
missing 1 unequal 0 24 on https://lemmy.ml/ vs. 23 on https://sh.itjust.works/

The number of comments is based on loading comments, not the counts at the top of the posting.

 

Heavy loads have been brought up several times, major social events where people flock to social media. Such as a terrorist bombing, submarine sinking, earthquake, nuclear meltdown, celebrity airplane crash, etc.

Low-budget hosting to compete with the tech giants is also a general concern for the project. Trying not to have a server that uses tons of expensive resources during some peak usage.

Load shedding and self-tuning within the code and for SysOps to have some idea if their Lemmy server is nearing overload thresholds.

See comments:

view more: ‹ prev next ›