Selfhosted
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.
Rules:
-
Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
-
No spam posting.
-
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.
-
Don't duplicate the full text of your blog or github here. Just post the link for folks to click.
-
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
-
No trolling.
Resources:
- selfh.st Newsletter and index of selfhosted software and apps
- awesome-selfhosted software
- awesome-sysadmin resources
- Self-Hosted Podcast from Jupiter Broadcasting
Any issues on the community? Report it using the report flag.
Questions? DM the mods!
view the rest of the comments
Thats why i was considering training my own model if possible. Ive been toying around with kobold.CPP and gpt4all which both have RAG implementations.
My idea is to essentially chat with documentation and as a separate use case, have it potentially be a AI search engine but locally hosted. I do still prefer to search myself, but fuck man, searches have gotten so bad, and the kobold.CPP web lookup feature was pretty neat IMO.
So yea you're not wrong, I'm just hoping that if in train it and or give it documentation it can reference when answering, it will be suitable. Mostly AI has been good for me as kind of a rubber ducky when troubleshooting and helping me search for things when I have some specific question and in don't want "top 5 things vaguely related to your question" results.
Interesting, I mainly have used text generation webui which has a search support plug in, kinda nifty to use my searxng instance for it. It's a bit finicky though.
Another thing to keep in mind then (apologies if this is just repeating info you already know), you'd also want to keep in mind your total potential context size in relation to the model size, since both take up VRAM. Reading search results/pages can eat up a lot
Yea I'm aware but I appreciate the insight :) so far my local ai experience has been lack luster so I'm hoping that training and RAG will make up for the context size at least a little. Ifnit can answer accurately in the first place, it may not need as big of a context window.
If you haven't tried using RAG in some form, I would recommend giving it a go. Its pretty cool stuff, helps make models answer more accurately based on the documentation you give them though in my case, ive had limited success. Tbh, chatgpt has become my last resort when I just wanna get something done but I don't like using it due to the privacy concerns, not to mention the ethical issues I have with ai training in general from big tech.
How is searxng BTW? Would you say its good to host or do you use a normal search engine more often? Or do you just use it for the AI search plugin?
Ive actually been thinking about using it rather than duckduckgo but was also hopeful the search index they are working on would be enough to satisfy my needs, or that a self hosted AI enabled search engine would work well enough when I need it.
I've completely replaced my searching with searxng, it is a little slower and ofc if I have an outage or something at home I have to go back to a different search temporarily but overall I like it a lot.
It was one of the first things I set up last year with my homelab because I am attempting to degoogle a fair amount, the Ai search stuff was just a fun test
Thats rad, thanks for the info. I may follow suit, been trying to degoogle myself lately.
For sure, good luck and have fun :D