sisyphean

joined 2 years ago
MODERATOR OF
[–] sisyphean@programming.dev 4 points 2 years ago* (last edited 2 years ago)

Here is an example of tokenization being biased toward English (using the author's Observable notebook):

This is the same sentence in English and my native Hungarian. I understand that this is due to the difference in the amount of text available in the two languages in the training corpus. But it's still a bit annoying that using the API for Hungarian text is more expensive :)

[–] sisyphean@programming.dev 3 points 2 years ago* (last edited 2 years ago) (3 children)

The best hacker is of course the one who can guess the password the fastest (all-lowercase, dictionary word).

[–] sisyphean@programming.dev 7 points 2 years ago

Oh yes, terrible indeed. Saved.

[–] sisyphean@programming.dev 2 points 2 years ago

It will work with any bigger instance because of federation. All communities with subscribers from an instance are available on that instance. Si site:lemmy.world, site:programming.dev, etc. will work.

[–] sisyphean@programming.dev 1 points 2 years ago (1 children)

Made the switch 4 years ago. No regrets.

[–] sisyphean@programming.dev 3 points 2 years ago (1 children)

Nice! This could be used to visualize history in tutorials or presentations.

[–] sisyphean@programming.dev 12 points 2 years ago (1 children)

Here is your Lemmy Gold:

Lemmy Gold

[–] sisyphean@programming.dev 1 points 2 years ago

As everything else by Lilian Weng, this is a very good no-nonsense overview of the state of LLM-based agents. Highly recommended.

[–] sisyphean@programming.dev 6 points 2 years ago* (last edited 2 years ago)

YAML is extremely complex for a configuration format and it has many really weird edge cases:

https://noyaml.com/

The problem is IMHO made worse because it looks so friendly at first glance.

[–] sisyphean@programming.dev 2 points 2 years ago (4 children)
  1. Only on programming.dev, at least in the beginning, but it will be open source so anyone will be able to host it for themselves.
  2. I set up a hard limit of 100 summaries per day to limit costs. This way it won’t go over $20/month. I hope I will be able to increase it later.
[–] sisyphean@programming.dev 1 points 2 years ago

Yes, for a moment you think “oh, there’s such a convenient API for this” and then you realize…

But we programmers can at least compile/run the code and find out if it’s wrong (most of the time). It is much harder in other fields.

[–] sisyphean@programming.dev 3 points 2 years ago* (last edited 2 years ago)

Hungarian here. It is safe to drink without boiling. People only boil water for baby formula to be extra safe.

 

Original tweet: https://twitter.com/emollick/status/1671528847035056128

Screenshots (from the tweet):

 

cross-posted from: https://programming.dev/post/133153

Quote:

In this work, we introduce TinyStories, a synthetic dataset of short stories that only contain words that a typical 3 to 4-year-olds usually understand, generated by GPT-3.5 and GPT-4. We show that TinyStories can be used to train and evaluate LMs that are much smaller than the state-of-the-art models (below 10 million total parameters), or have much simpler architectures (with only one transformer block), yet still produce fluent and consistent stories with several paragraphs that are diverse and have almost perfect grammar, and demonstrate reasoning capabilities.

Related:

 
 

Old but gold.

 

The original thread is on the devil’s website and I don’t want to direct traffic to it, so here’s a link to the tweet instead:

https://twitter.com/davidfowl/status/1671351948640129024?s=46&t=OEG0fcSTxko2ppiL47BW1Q

 

Quote:

In this work, we introduce TinyStories, a synthetic dataset of short stories that only contain words that a typical 3 to 4-year-olds usually understand, generated by GPT-3.5 and GPT-4. We show that TinyStories can be used to train and evaluate LMs that are much smaller than the state-of-the-art models (below 10 million total parameters), or have much simpler architectures (with only one transformer block), yet still produce fluent and consistent stories with several paragraphs that are diverse and have almost perfect grammar, and demonstrate reasoning capabilities.

Related:

view more: ‹ prev next ›