Fuck AI

The thing with comparing sci-fi AI to modern LLMs is that virtually no science fiction AI that I know of actually acts the way LLMs do. They tend to be good at things modern AI is bad at, like logical reasoning or advanced math, but bad at things that AI can already do, like generate images that at least look like they could be human art, or write text that appears emotionally charged. They also tend to be directly programmed in ways that a singular (usually genius, but still) individual can pick through and understand, rather than being trained in a black box sort of manner that is very difficult for a human to reverse-engineer.

That isnt surprising, sci-fi writers arent oracles after all and just having AI of some kind probably makes a story more realistic than just assuming the technology never gets invented even far into the future, but in my view these kinds of sci-fi AI are basically a different, hypothetical technology going for the same end result. As such, I dont really expect even a very advanced iteration on what we have will look like star trek AI, any more than modern cars tend to fly or run off miniature nuclear reactors the way sci-fi of decades ago saw cars of the future. I dont think it will look like skynet either. I do think we might get some interesting science fiction in the coming decades exploring what a very advanced version of the technology we do have might end up like though. It probably wont be terribly accurate either, but Id bet it will be closer than works where the basis for extrapolating AI tech is "what if the calculator could talk and think".

Star Trek, at least before Roddenberry's vision was corrupted, was a fictional post scarcity socialist utopia. The AI served the crew and society. It wasn't there to exploit and replace them on behalf of a handful of Ferengi billionaires.

Star Trek's ship AI is nothing like the huge financial scams, exploitation and concentration of power, wealth and political influence we are currently seeing. AI hate mostly isn't about machine learning and its applications. It's about the people behind it.

When we have warp drives, fusion, replicators, universal basic income and free universal health care we may have a different view.

Star fleet has rigorous education and training

AI does not replace their critical thinking

I thought I was in !startrek@startrek.website for a moment...

My take is that even if you consider LLMs to fall under the umbrella of "AI" (I don't), they appear to be a completely different technology than the Enterprise-D computer, which is more like highly advanced natural language processing.

would you want a perfect AI that was incapable of lying or harbouring anything untrue?

It's not really possible for an AI to know what's true with 100% accuracy, but I do think it's technically possible to invent an AI that is honest. It's important to remember that LLMs are actually "hallucinating" 100% of the time. The only reason they are ever correct is because the training data was correct.

Calling the ship voice command interface an AI is quite a stretch... even with the much more lenient definitions getting thrown around these days.

I don't think there is much evidence of the Enterprise's computer being used to do much more than provide or process basic information. You basically never see the characters in Star Trek rely on the computer for creativity or solutions at all, from what I remember. On top of that, we don't know how the computer on the Enterprise was created or what the ethical implications of it may be.

Star Trek is a show that delves into ethical dilemmas often, and the problem with today's generative AI is an ethical and legal one, not necessarily a technological one.

Today's generative "AI" is much more like the Borg than Commander Data or the Enterprise-D: it is powered by the forceful assimilation of human culture for the benefit of those that own and control it. We are also quite literally being told (by the stakeholders) that resistance against this new technology is futile and that we must adapt to a new reality in which our work will be assimilated whether we like it or not. There is no consent, let alone compensation. They are simply on a neverending mission to take everything within reach for their own benefit...

Mate I'd live in Banks' Culture without a heartbeat's thought if I could.

The problem isn't AI as a concept, it's the underlying societal disregard for ethics in the face of profits.

In Star Trek AI is used in many applications that a humanoid cannot reasonably do themselves entirely manually (not without serious augmentation, which is heavily frowned upon).

The amount of heavy lifting the ship's computer does for navigating the galaxy at warp speeds, or to process some incredibly advanced calculations almost instantly. This would go significantly slower if a humanoid had to do that all manually.

And yet...

The crew of the Enterprise D aren't merely prompting the ship's AI. Yes, there is that too, but they're typically doing that while also being very hands-on with the ship's systems too. The crews in Starfleet have the expertise of deep level computer programming, to the point of physically arranging computer systems if need be.

There is no "vibe coding" in Starfleet.

But then there is the Holodeck.

I'm OK with certain applications of the Holodeck, like spontaneously creating virtual activity or recreation areas. These things aren't considered works of art, and aren't considered worthy substitutes over the real thing either. You find them on ships and stations because they're the best available substitute, the alternative being crewman slowly going mad from seeing nothing but sterile corridors all day. I don't think I've heard of recreational Holodecks for regular individuals (unlikely due to some regulation, more likely that fulfilment is achieved in other ways if you're surrounded by a real planet).

However, Voyager sort of fucks with this idea with the crew "creating holonovels". This is essentially a vibe coder's dream; being able to create fully interactive narrative driven videogames with nothing but prompts. That said... Even in Voyager, using someone's likeness is heavily frowned upon. So there is still an expectation of originality, rather than merely rearranging a reference dataset. I can maybe forgive Voyager on the basis of its premise; they're stuck on the other side of the galaxy far from home. What else would they even do? But that's about as lenient as I can be.

On a societal level, however, real skill in a craft is still greatly appreciated, perhaps even preferred, over something computer generated. Despite there being replicators and no money, there are still bars, pubs, restaurants, wine makers, beer brewers, jewelers, you name it, there are still people going to these places to get a real cooked meal and have a real experience in a real location. Craft, skill, humanoid created instead of generated, these things are still valued enough for there to exist whole streets full of shopkeepers and hospitality providers on Federation worlds.

"AI" as it exists now is created to deliberately replace humans skill, to take from human skill without offering credit or appreciation, to make humans obsolete in the creation of immediate end results that can be sold as products or give instant gratification. The Federation as a society is shifted massively away from such a mentality.

There's nothing realistic about Star Trek.

This needs to be hammered more. It's an awesome setting for exploring contemporary social issues; that's the point. But no matter how technologically advanced we get, it's just not based on even plausible physics/engineering. Neither is Terminator.

I think the most plausible 'extrapolation' I've seen is Orion's Arm:

https://www.orionsarm.com/eg-article/486e75a54a1ae

https://www.orionsarm.com/xcms.php?r=oa-timeline

And, as an aside, I adore this Mass Effect story: https://archiveofourown.org/works/42006774/chapters/105462066

They extrapolate alteration of human biology, nanotechnology, plantery engineering, STL space travel, and AI, and that future looks nothing like more-or-less unaltered baseline humans walking around on a space boat with glorified voice assistants. Consciousness is uploaded and downloaded. Artifical realities are vast. Whole celestial bodies are manipulated for all sorts of purposes. People can inhabit an array 'bodies' and realities that make stuff like starship bridges/hallways and colonies on planets seem silly. There are 'stratas' of consciousness literally orders of magnitude apart, states of being incomprehensible to each other, all coexisting in an expanding bubble of civilization thats still younger and (in some ways) more primitive than the TNG Federation.

And we aren't that far from that. Including the "techpocalypse" it predicts.

Terminator and Star Trek, and classic sci fi, don't depict this because they're stories aimed at humans living right now, and their interpersonal relationships and contemporary social/political issues. More realistic extrapolations are tougher settings for that.

So the question is… would you want a perfect AI that was incapable of lying or harbouring anything untrue? Basically you could ask it anything and it would give you the correct answer.

Where I'm going with this is that this is that, to me, this not a realistic question. Practically, it'd be silly to relegate a "true" AI as a dumb voice assistant on a space boat; they're conscious beings, even if they're shackled or so highly specialized.

They're not so different from human beings at that point. In a more realistic situation, you could be uploaded and repurposed as the Enterprise voice assistant, and have your consiousness duplicated and mashed with complex systems; pondering if you'd want to be Siri seems kind of silly.

AI that was incapable of lying

What does that actually entail? The AI that is programmed with only right wing talking points is going to think that those are the objective truths even if they aren't so if you put guardrails on it to only say the "truth" you're going to get lies.

I think the best we can hope for is minimal bias in the AI we develop.

I have a theory here:

There is definitely problematic gen AI in the Star Trek universe, but it's only ever adressed through the holodeck. It's made pretty clear that everyone can create programmes there with simple voice prompts. It has also been shown that there are no formal rules for using the likeness of living people in those programmes. This is an oversight in my opinion because that problem would be a common concern. The existence of this kind of technology suggest that any kind of entertainment media can be easily created on a prompt, even through the ship's computers.

On the other hand, we rarely hear about contemporary human-made literature. When literature is mentioned it's usually alien or 20th century. Wouldn't this suggest that it plays no role anymore? Maybe there are still human writers, but the general public isn't interested in such things, since they can get what ever they want from a computer.

So my bottom line is that concerning generative AI Star Trek actually shows how problematic it is, probably by accident. I wouldn't want Star Trek-level AI, but at least it doesn't kill everyone.