this post was submitted on 11 Dec 2025
547 points (96.6% liked)

Technology

77631 readers
1584 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots


founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
[–] Devial@discuss.online 176 points 2 days ago* (last edited 2 days ago) (7 children)

The article headline is wildly misleading, bordering on being just a straight up lie.

Google didn't ban the developer for reporting the material, they didn't even know he reported it, because he did so anonymously, and to a child protection org, not Google.

Google's automatic tools, correctly, flagged the CSAM when he unzipped the data and subsequently nuked his account.

Google's only failure here was to not unban on his first or second appeal. And whilst that is absolutely a big failure on Google's part, I find it very understandable that the appeals team generally speaking won't accept "I didn't know the folder I uploaded contained CSAM" as a valid ban appeal reason.

It's also kind of insane how this article somehow makes a bigger deal out of this devolper being temporarily banned by Google, than it does of the fact that hundreds of CSAM images were freely available online and openly sharable by anyone, and to anyone, for god knows how long.

[–] ulterno@programming.dev 4 points 8 hours ago* (last edited 8 hours ago) (1 children)

Another point is, the reason Google's AI is able to identify CSAM is because it has that in its training data, flagged as such.

In that case, it would have detected the training material as ~100% match.

I don't get though, how it ended up being openly available as if it were properly tagged, they would probably exclude it from the open-sourced data. And now I see it would also not be viable to have an open-source, openly scrutinisable AI deployment for CSAM detection for the same reason.

And while some governmental body got a lot of backlash for trying to implement such an AI thing on chat stuff, Google gets to do so all it wants because it's E-Mail/GDrive and all on their servers and you can't expect privacy.


Considering how many such stories of people having problems due to this system is coming up, is there any statistic of legitimate catches using this model? I suspect not, because why would anyone use Google services for this kind of stuff?

[–] arararagi@ani.social 2 points 7 hours ago (1 children)

You would think, but none of these companies actually make their own dataset, they buy from third parties.

[–] ulterno@programming.dev 0 points 6 hours ago

I am not sure which point you are answering to.
COuld you please specify.

[–] MangoCats@feddit.it 17 points 1 day ago

Google’s only failure here was to not unban on his first or second appeal.

My experience of Google and the unban process is: it doesn't exist, never works, doesn't even escalate to a human evaluator in a 3rd world sweatshop - the algorithm simply ignores appeals inscrutably.

[–] forkDestroyer 22 points 1 day ago (2 children)

I'm being a bit extra but...

Your statement:

The article headline is wildly misleading, bordering on being just a straight up lie.

The article headline:

A Developer Accidentally Found CSAM in AI Data. Google Banned Him For It

The general story in reference to the headline:

  • He found csam in a known AI dataset, a dataset which he stored in his account.
  • Google banned him for having this data in his account.
  • The article mentions that he tripped the automated monitoring tools.

The article headline is accurate if you interpret it as

"A Developer Accidentally Found CSAM in AI Data. Google Banned Him For It" ("it" being "csam").

The article headline is inaccurate if you interpret it as

"A Developer Accidentally Found CSAM in AI Data. Google Banned Him For It" ("it" being "reporting csam").

I read it as the former, because the action of reporting isn't listed in the headline at all.

^___^

[–] WildPalmTree@lemmy.world 1 points 8 hours ago

The inclusion of "found" indicates that it is important to the action taken by Google, would be my interpretation.

[–] Blubber28@lemmy.world 6 points 1 day ago (2 children)

This is correct. However, many websites/newspapers/magazines/etc. love to get more clicks with sensational headlines that are technically true, but can be easily interpreted as something much more sinister/exciting. This headline is a great example of it. While you interpreted it correctly, or claim to at least, there will be many people that initially interpret it the second way you described. Me among them, admittedly. And the people deciding on the headlines are very much aware of that. Therefore, the headline can absolutely be deemed misleading, for while it is absolutely a correct statement, there are less ambiguous ways to phrase it.

[–] MangoCats@feddit.it 3 points 1 day ago

can be easily interpreted as something...

This is pretty much the art of sensational journalism, popular song lyric writing and every other "writing for the masses" job out there.

Factual / accurate journalism? More noble, but less compensated.

[–] obsoleteacct@lemmy.zip 1 points 1 day ago (1 children)

It is a terrible headline. It can be debated whether it's intentionally misleading, but if the debate is even possible then the writing is awful.

[–] MangoCats@feddit.it 2 points 1 day ago

if the debate is even possible then the writing is awful.

Awfully well compensated in terms of advertising views as compared with "good" writing.

Capitalism in the "free content market" at work.

[–] ayyy@sh.itjust.works 3 points 1 day ago

The article headline is wildly misleading, bordering on being just a straight up lie.

A 404Media headline? The place exclusively staffed by former BuzzFeed/Cracked employees? Noooo, couldn’t be.

[–] cupcakezealot@piefed.blahaj.zone 1 points 1 day ago (1 children)

so they got mad because he reported it to an agency that actually fights csam instead of them so they can sweep it under the rug?

[–] Devial@discuss.online 24 points 1 day ago* (last edited 1 day ago) (1 children)

They didn't get mad, they didn't even know THAT he reported it, and they have no reason or incentive to swipe it under the rug, because they have no connection to the data set. Did you even read my comment ?

I hate Alphabet as much as the next person, but this feels like you're just trying to find any excuse to hate on them, even if it's basically a made up reason.

[–] cupcakezealot@piefed.blahaj.zone -5 points 1 day ago (3 children)

they obviously did if they banned him for it; and if they're training on csam and refuse to do anything about it then yeah they have a connection to it.

[–] Devial@discuss.online 7 points 1 day ago* (last edited 1 day ago)

Also, the data set wasn't hosted, created, or explicitly used by Google in any way.

It was a common data set used in various academic papers on training nudity detectors.

Did you seriously just read the headline, guess what happened, and are now arguing based on that guess that I, who actually read the article, am wrong about it's content ? Because that's sure what it feels like reading your comments......

[–] MangoCats@feddit.it 3 points 1 day ago

Google doesn't ban for hate or feels, they ban by algorithm. The algorithms address legal responsibilities and concerns. Are the algorithms perfect? No. Are they good? Debatable. Is it possible to replace those algorithms with "thinking human beings" that do a better job? Also debatable, from a legal standpoint they're probably much better off arguing from a position of algorithm vs human training.

[–] Devial@discuss.online 3 points 1 day ago* (last edited 1 day ago)

So you didn't read my comment then did you ?

He got banned because Google's automated monitoring system, entirely correctly, detected that the content he unzipped contained CSAM. It wasn't even a manual decision to ban him.

His ban had literally nothing whatsoever to do with the fact that the CSAM was part of an AI training data set.