Technology
This is the official technology community of Lemmy.ml for all news related to creation and use of technology, and to facilitate civil, meaningful discussion around it.
Ask in DM before posting product reviews or ads. All such posts otherwise are subject to removal.
Rules:
1: All Lemmy rules apply
2: Do not post low effort posts
3: NEVER post naziped*gore stuff
4: Always post article URLs or their archived version URLs as sources, NOT screenshots. Help the blind users.
5: personal rants of Big Tech CEOs like Elon Musk are unwelcome (does not include posts about their companies affecting wide range of people)
6: no advertisement posts unless verified as legitimate and non-exploitative/non-consumerist
7: crypto related posts, unless essential, are disallowed
view the rest of the comments
That's like says smartphones are fundamentally a surveillance technology. There's truth to it, but it's not inherent to the technology. It's a deliberate act by people using the tech that we allow for whatever reason.
Right, you can still do traditional advertising without the targeted metrics provided by smartphones, but....
AI LLMs literally require a corpus of language to learn from. Thus the "Large Language" part of "LLM." The amount of data these models need to function is so staggeringly huge there is no way they can compile all that data without scraping the entire internet and pirating a bunch of copyrighted books.
It's fundamentally a surveillance technology, because the technology fundamentally cannot function without that large dataset of language to begin with. It needs massive amounts of data that have to be surveilled to be achieved, because unless you're Reddit or Facebook, your own site probably doesn't contain enough data to fill out the needs of the LLM. Thus you need to scrape the internet for more data in hopes of filling it out.
Books3 is used widely as part of "The Pile" and is clearly all of the content of private torrent tracker Bibliotik. People theorize Books2 is all of the books from Library Genesis. To be able to make their models work, they have to scrape the internet and pirate thousands of books to make it functional at all.
This is also fundamentally why AI starts to fail so quickly, because these tools have been used to flood the internet with AI generated pages, which in turn become training data for AI, which means the training data is tainted with AI generated garbage, which will further degrade the LLM. On the plus side, I guess, is that if they keep using this kind of business model, they will unintentionally make their AI pretty useless within a few years by flooding the internet with useless, incorrect data.
I very much disagree with the characterization that training an LLM on a book is pirating said book. We might see copyright owners release their materials in the future under licenses that disallow this, which is their right (though it's not clear to me that any copy is being made). In my opinion there's not a lot difference between me training an LLM on said book and me using the story as inspiration for my own book. I suspect we'll never agree on that one.
Pretty amusing that you think scraping published data somehow constitutes surveillance, though.
If the source is literally a piracy website that serves up applications on how to remove DRM from ebooks, it's absolutely piracy. You can't just deny the source and be like "it's not piracy!" The way the data came into your hands was illicitly, not legally. Especially if DRM has been circumvented and removed before it came into your hands.
They didn't go out and buy copies of thousands of books.
I don't, I was making a point about how absurdly large the language models have to be, which is to say, if they have to have that much data on top of thousands of pirated books, it means they fundamentally cannot make the models work without also scraping the internet for data, which is surveillance.
And if they went to a library and scanned all the books?
I mean, it's just not surveillance, by definition. There's no observation, just data ingestion. You're deliberately trying to conflate the words to associate a negative behavior with LLM training to make your argument.
I really don't get why LLMs get everybody all riled up. People have been running Web crawlers since the dawn of the Web.
The AI literally observes the training data
Insofar as my computer observes the data on my hard disk. But I suspect you know what I meant.