Lemmy Federate

95 readers

1 users here now

Updates and questions about lemmy-federate.com

founded 5 months ago

MODERATORS

iso@lemy.lol

mapping threadiverse relationships (slrpnk.net)

submitted 2 weeks ago by activistPnk@slrpnk.net to c/lemmyfederate@lemy.lol

6 comments fedilink hide all child comments

cross-posted from: https://slrpnk.net/post/24729225

Are there any open datasets that track who is federated / defederated with who? I have the Lemmyverse datasets but it has nothing on node relationships.

lemmy-federate.com appears to have the info I am after, but it only appears to be reachable in a GUI webpage. Any way to get the dataset?

you are viewing a single comment's thread
view the rest of the comments

[–] iso@lemy.lol 2 points 2 weeks ago (1 children)

You can easily fetch through the API. You can either check API routes from the repo or I can give you some examples if you want to?

[–] activistPnk@slrpnk.net 2 points 2 weeks ago* (last edited 2 weeks ago) (1 children)

I don’t have full time Internet access. I pop into a library periodically. So I have found it useful to fetch the lemmyverse dataset and import it. Takes just a few seconds to fetch the whole dataset. Then when I am offline I search the DB to decide where to post, and write my posts offline.

I’m trying to grasp what you are suggesting. Do you mean I write a program to reach out to each instance and harvest the data? I think there are around 150 lemmy instances. I’m not sure how quickly that can be done, or if it would be welcome. When I visit slrpnk.net using my browser, there is an anti-bot check which takes a few seconds. I don’t imagine cURL or the like would get past that, which I suspect is some javascript code that must be executed.

In your repo I see routes/community.ts and nodes.ts. But therein I do not see a list of instances or communities to visit. Is ts for typescript? That lang is unknown to me.

[–] iso@lemy.lol 2 points 2 weeks ago (1 children)

Here you go:

https://lemmy-federate.com/api/community.find?input=%7B%22skip%22%3A0%2C%22take%22%3A10%7D
https://lemmy-federate.com/api/instance.find?input=%7B%22search%22%3A%22%22%2C%22skip%22%3A0%2C%22take%22%3A10%2C%22enabledOnly%22%3Afalse%7D

You can change the skip and take values to paginate through list.

[–] activistPnk@slrpnk.net 1 points 2 weeks ago* (last edited 2 weeks ago)

thanks!

The instance list is tiny. But perhaps that doesn’t matter since instances can be derived from the community list.