I know, I'm just saying it's not theoretically impossible to have a phone number as a token. It's just probably not what happened here.
the choice of the next token is really random
It's not random in the sense of a uniform distribution which is what is implied by "generate a random [phone] number".
A full phone number could be in the tokenizer vocabulary, but any given one probably isn't in there
I mean the latter statement is not true at all. I'm not sure why you think this. A basic GPT model reads a sequence of tokens and predicts the next one. Any sequence of tokens is possible, and each digit 0-9 is likely its own token, as is the case in the GPT2 tokenizer.
An LLM can't generate random numbers in the sense of a proper PRNG simulating draws from a uniform distribution, the output will probably have some kind of statistical bias. But it doesn't have to produce sequences contained in the training data.
It's a number and complexity refers to functions. The natural inclusion of numbers into functions maps pi to the constant function x -> pi which is O(1).
If you want the time complexity of an algorithm that produces the nth digit of pi, the best known ones are something like O(n log n) with O(1) being impossible.
The direct connection is cool, I just wonder if a P2P connection is actually any better than going through a data center. There's gonna be intermediate servers right?
Do you need to have Tailscale set up on any network you want to use this on? Because I'm a fan of being able to just throw my domain or IP into any TV and log in
I just use nginx on a tiny Hetzner vps acting as a reverse proxy for my home server. I dunno what the point of Tailscale is here, maybe better latency and fewer network hops in some cases if a p2p connection is possible? But I've never had any bandwidth or latency issues doing this
It gets around port forwarding/firewall issues that most people don't know how to deal with. But putting it behind a paywall kinda kills any chance of it being a benevolent feature.
Possible reasons include:
fun
inflicting needless suffering on fish [applies if you hate fish]
It's got a very high barrier to entry. You kinda have to suffer through it for a while before you get it. And then you unlock a totally different kind of suffering.
The last time I had fun with LLMs was back when GPT2 was cutting-edge, I fine-tuned GPT2-Medium on Twitch chat logs and it alternates between emote spam, complete incoherence, blatantly unhinged comments, and suspiciously normal ones. The bot is still in use as a toy, specifically because it's deranged and unpredictable. It's like a kaleidoscope for the slice of internet subculture it was trained on, much more fun than a plain flawless mirror.
1000 or so moves
Ok, this nerd-snipes me a bit. The real point of the Towers of Hanoi problem, at least if you're doing it in a discrete math class, is to count exactly the number of moves for n discs. Call this number f(n).
To make this post easier to read, we will label the discs from 1 to n, with 1 being the bottom disc and n being the top. We will also label the pegs A, B, and C, with A being the starting peg and C being the target peg. This lets us describe moves in a compact notation: 3: A -> C means disc 3 moves from A to C.
For n=0, we vacuously need 0 moves to win.
For n=1, we need just 1 move: 1:A->C.
For n=2, we can do it in 3 moves, so f(2) = 3. The sequence is:
Now suppose we know f(n) for n between 1 and N, with N >= 2. For n=N+1, we can move N+1: A -> C and attempt to use our strategy for moving the remaining N discs from A to C, shuffling N+1 around as needed. Since it's the smallest disc, we can always move it to any pillar, and any time we want to move something to the pillar it's on, it must be moved first. Let's see how that plays out for N=2:
We hazard a guess that every step of the N-disc strategy is preceded by a move of the N+1 disc, plus one final move. In other words, f(N+1) = 2f(N) + 1. Some careful analysis will justify this guess.
Now we have a recurrence relation for f(n), but how to solve it? A classical technique is "guess the formula magically, then prove it by induction." It's certainly doable here if you compute a few values by hand:
f(2) = 3
f(3) = 2(3) + 1 = 7
f(4) = 2(7) + 1 = 15
f(5) = 2(15) + 1 = 31
f(6) = 2(31) + 1 = 63
You'll probably see it right away: f(n) = 2^n - 1. Indeed, we can prove this by induction: the base case n=2 holds as f(2) = 3 = 2^(2) - 1, and if f(n) = 2^n - 1 then f(n+1) = 2f(n) + 1 = 2(2^n - 1) + 1 = (2^(n+1) - 2) + 1 = 2^(n+1) - 1 as desired.
We may conclude f(12) = 2^(12) - 1 = 4095. In other words, with 12 discs, exactly 4095 moves are required.
--
Bonus: an alternative to the "guess and prove" technique that is generalizable to a broad class of recurrence relations. The technique is called "generating functions." Given the sequence f(n), we consider the formal power series
F(x) = f(0) + f(1)x + f(2)x^(2) + ... + f(n)x^(n) + ...
This F(x) is the "ordinary generating function" for the sequence f(n). Strictly speaking, it may not be a well-defined function of x since we haven't made any assumptions about convergence, but we will assume (for now, and prove later) that the set of such formal power series behaves algebraically much like we expect. Namely, given the recurrence relation above, we can write:
F(x) - f(0) - f(1)x = f(2)x^(2) + f(3)x^(3) + f(4)x^(4) + ... + f(n)x^n + ...
= (2f(1) + 1)x^(2) + (2f(2) + 1)x^(3) + ... + (2f(n-1) + 1)x^(n) + ...
= 2x(f(1)x + f(2)x^(2) + ... + f(n)x^(n)) + (x^(2) + x^(3) + ... + x^(n) + ...)
= 2x(F(x) - f(0)) + x^(2)/(1-x)
In our case, we have f(0) = 0, f(1) = 1, f(2) = 3 so we can write more succinctly:
F(x) - x = 2xF(x) + x^(2)/(1-x)
Solving for F,
F(x)(1 - 2x) = x + x^(2)/(1-x)
= x(1 + x/(1-x))
F(x) = x(1 + x/(1-x))/(1 - 2x)
= x/(2x^(2) - 3x + 1)
Ok, great. We've found that our generating function, convergence notwithstanding, is that rational function. We can use partial fraction decomposition to write it as
F(x) = 1/(1 - 2x) - 1/(1-x)
which has the advantage of telling us exactly how to compute the coefficients of the Taylor series for F(x). Namely,
1/(1-x) = 1 + x + x^(2) + ... + x^(n) + ...
1/(1 - 2x) = 1 + 2x + 4x^(2) + ... + 2^(n) x^(n) + ...
So F(x) = (1-1) + (2-1)x + (4-1)x^(2) + ... + (2^(n)-1)x^(n) + ...
The nth coefficient of the Taylor series for F about 0 is 2^(n)-1, and by the definition of F as the ordinary generating function for f(n), we have f(n) = 2^(n) - 1. (The rigorous justification for ignoring convergence here still needs to be done; for now, this can be seen as a useful magic trick.)
Yippee I missed these