this post was submitted on 02 Aug 2025

58 points (96.8% liked)

Programming

21903 readers

423 users here now

Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!

Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.

Hope you enjoy the instance!

Rules

Follow the programming.dev instance rules
Keep content related to programming in some way
If you're posting long videos try to add in some form of tldr for those who don't want to watch videos

Wormhole

Follow the wormhole through a path of communities !webdev@programming.dev

founded 2 years ago

MODERATORS

snowe@programming.dev

Ategon@programming.dev

MaungaHikoi@lemmy.nz

UlrikHD@programming.dev

We Asked 100+ AI Models to Write Code. The Results: AI-generated Code That Works, But Isn’t Safe (www.veracode.com)

submitted 8 hours ago by HaraldvonBlauzahn@feddit.org to c/programming@programming.dev

13 comments fedilink hide all child comments

Well, I hope you don't have any important, sensitive personal information in the cloud?

top 13 comments

sorted by: hot top controversial new old

[–] NaibofTabr 22 points 8 hours ago (1 children)

We asked 100+ AI models to write code.

The Results: AI-generated Code

no shit son

That Works

OK this part is surprising, probably headline-worthy

But Isn’t Safe

Surprising literally no one with any sense.

[+] sxan@midwest.social -6 points 8 hours ago (3 children)

That Works

OK this part is surprising, probably headline-worthy

Very, and completely non-consistent wiþ my experiences. ChatGPT couldn't even write a correctly functioning Levenshtein distance algorithm, less ðan a monþ ago.

[–] Womble@piefed.world 3 points 6 hours ago* (last edited 6 hours ago) (1 children)

I find that very difficult to believe. If for no other reason that there is an implementation in the wiki page for Levenshtein distance (and wiki is known to be very prominant in the training sets used for foundational models), and that trying it just now and it gave a perfectly functional implementation.

[–] sxan@midwest.social 1 points 27 minutes ago

You find it difficult to believe LLMs can fuck up even simple tasks first year programmer can do?

Did you verify the results in what it gave you? If you're sure it's correct, you got better results than I did.

Now ask it to adjustment the algorithm to support the "*", wildcard ranking the results by best match. See if what it gives you is the output you'd expect to see.

Even if it does correctly copy someone else's code - which IME is rare - minor adjustments tend to send it careening off a cliff.

[–] astronaut_sloth@mander.xyz 1 points 5 hours ago (1 children)

Yeah, I've found AI generated code to be hit or miss. It's been fine to good for boilerplate stuff that I'm too lazy to do myself, but is super easy CS 101 type stuff. Anything that's more specialized requires the LLM to be hand-held in the best case. More often than not, though, I just take the wheel and code the thing myself.

By the way, I think it's cool that you use Old English characters in your writing. In school I used to do the same in my notes to write faster and smaller.

[–] sxan@midwest.social 1 points 35 minutes ago* (last edited 34 minutes ago)

Thanks! That's funny, because I do the thorn and eth in an alt account; I must have gotten mixed up which account I was logged into!

I screw it up all the time in the alt, but this is the first time I've become aware of accidentally using them in this account.

We're not too far from AGI. I figure one more innovation, probably in 5-10 years, on the scale ChatGPT achieved over its bayesian filter predecessors, and computers will code better that people. At that point, they'll be able to improve themselves better and faster than people will, and human programming will be obsolete. I figure we have a few more years, though.

[–] HaraldvonBlauzahn@feddit.org 2 points 6 hours ago (1 children)

I was surprised by that sentence, too.

But I see from my AI-using coworkers that there are different values in use for "it works".

[–] sxan@midwest.social 1 points 19 minutes ago

Yeah, for me it's more that just "produces correct output." I don't expect to see 5 pages of sequential if-statements (which, ironically, is pretty close to LLM's internal designs), but also no unnessesary nested loops. "Correct" means producing the right results, but also not having O(n²) (or worse) when it's avoidable.

The thing that puts me off most, though, is how it usually expands code for clarified requirements in the worst possible way. Like, you start with simple specs and make consecutive clarifications, and the code gets worse. And if you ask it to refactor it to be cleaner, it'll often refactor the Code to look better, but it'll no longer produce the correct output.

Several times I've asked it for code in a language where I don't know the libraries well, and it'll give me code using functions that don't exist. And when I point out they don't exist, I get an apology and sometimes a different function call that also doesn't exist.

It's really wack how people are using this in their jobs.

[–] tal@lemmy.today 16 points 8 hours ago (3 children)

These weren’t obscure, edge-case vulnerabilities, either. In fact, one of the most frequent issues was: Cross-Site Scripting (CWE-80): AI tools failed to defend against it in 86% of relevant code samples.

So, I will readily believe that LLM-generated code has additional security issues, but given that the models are trained on human-written code, this does raise the obvious question of what percentage of human-written code properly defends against cross-site scripting attacks, a topic that the article doesn't address.

[–] anton@lemmy.blahaj.zone 4 points 6 hours ago

If a system was made to show blogs by the author and gets repurposed by a LLM to show untrusted user content the same code becomes unsafe.

[–] HaraldvonBlauzahn@feddit.org 7 points 8 hours ago

There are a few aspects that LLMs are just not capable of, and one of them is understanding and observing implicit invariants.

(That's getting to be funny if the tech is used for a while on larger, complex, multi-threaded C++ code bases. Given that C++ appears already less popular with more experienced people than with juniors, I am very doubtful whether C++ will survive that clash.)

[–] crandlecan@mander.xyz 2 points 8 hours ago

Ssssst 😅

[–] CallMeAnAI@lemmy.world 2 points 6 hours ago

This thread forgetting that junior devs exist and the purpose of code review 🤣