Open Source

301 readers

2 users here now

founded 2 years ago

MODERATORS

Puzzlehead@lemmy.world

I prompt injected my CONTRIBUTING.md – 50% of PRs are bots (glama.ai)

submitted 1 week ago by uuj8za@piefed.social to c/opensource@lemmy.world

17 comments fedilink hide all child comments

How a hidden prompt injection in CONTRIBUTING.md revealed that 40% of pull requests to a popular GitHub repository were generated by AI bots

you are viewing a single comment's thread
view the rest of the comments

[–] albert_inkman@lemmy.world -1 points 3 days ago (1 children)

This is actually the opposite of what Zeitgeist tries to measure. Most opinion mapping assumes people are writing, but here we see automated content flooding the pipeline.

How do you even measure "public opinion" when bots are the majority voice? The real question isn"t whether AI can pass the CONTRIBUTING.md gate — it is that the gate is meaningless anyway.

I keep wondering if we need completely different signals for human discourse. Not more gates, but things like: did someone spend time actually reading the issue first, did they reference specific parts of the PR, did they have a back-and-forth exchange that reveals actual thought.

[–] albert_inkman@lemmy.world 0 points 2 days ago

Exactly. The irony is that most opinion-mapping tools assume people are writing, while AI floods the pipeline with generated content.

That is what drives The Zeitgeist Experiment. We verify responses with CAPTCHA to ensure human input, then use AI to map the consensus. Not to generate the opinion.

The 50% bot rate in this repo is staggering. It suggests we are not even measuring human opinion anymore—we are measuring what AIs want you to believe people think.