NotAwfulTech

I've finished grading all of the entries so far. I don't think that we'll get any more, so here's a preview of the upcoming blog post.

The tier listings are as follows:

• B tier: Corbin S. (Task 1), Corbin S. (Task 2), Corbin S. (Task 3)
• C tier: Piper M. (Task 1)

Admittedly, we didn't get a whole lot of players, but that's it. That's the entire tier listing. I had three things I wanted to do in my spare time. I did them and got an average ranking based on my average predictions of the future; I met expectations. Piper also placed and I greatly appreciate her sportsmanship here.

My solutions are available as notes and source code. For Task 1, I have three main commits: one, two, three, and a bugfix. For Task 2, the commits are internal to my homelab, but I do have notes and source code. Finally, for Task 3, I put the entire repository into a flat gist including notes, source code, and Nix flake.

It occurs to me that this audience might not immediately understand how hard the chosen tasks are. I was fairly adversarial with my task selection.

Two of them are in RPython, an old dialect of Python 2.7 that chatbots will have trouble emitting because they're trained on the incompatible Python 3.x lineage. The odd task out asks for the bot to read Raku, which is as tough as its legendary predecessor Perl 5, and to write low-level code that is very prone to crashing. All three tasks must be done relative to a Nix flake, which is easy for folks who are used to it but not typical for bots. The third task is an open-ended optimization problem where a top score will require full-stack knowledge and a strong sense of performance heuristics; I gave two examples of how to do it, but by construction neither example can result in an S-tier score if literally copied.

This test is meant to shame and embarrass those who attempt it. It also happens to be a slice of the stuff that I do in my spare time.

I especially love the third task because that's exactly the kind of shit you get thrown on your plate in the field as a SWE.

There's been some work an old member of our team did a year ago. No one remembers what it was, but it is important. Please do something.

That's almost exactly one of the first tasks I got as an intern when I was starting out. THIS is what they are saying LLMs are going to replace.