this post was submitted on 11 Jul 2023
18 points (95.0% liked)

ChatGPT

9818 readers
56 users here now

Unofficial ChatGPT community to discuss anything ChatGPT

founded 2 years ago
MODERATORS
marcar@lemmy.world
18
GPT-4's details are leaked. (threadreaderapp.com)
submitted 2 years ago by manitcor@lemmy.intai.tech to c/chatgpt@lemmy.world
16 comments fedilink hide all child comments
 

cross-posted from: https://lemmy.intai.tech/post/72919

Parameters count:

GPT-4 is more than 10x the size of GPT-3. We believe it has a total of ~1.8 trillion parameters across 120 layers. Mixture Of Experts - Confirmed.

OpenAI was able to keep costs reasonable by utilizing a mixture of experts (MoE) model. They utilizes 16 experts within their model, each is about ~111B parameters for MLP. 2 of these experts are routed to per forward pass.

Related Article: https://lemmy.intai.tech/post/72922

you are viewing a single comment's thread
view the rest of the comments
[–] Call_Me_Maple@lemmy.world 1 points 2 years ago

Yeah, I agree. Just because it's bad now, doesn't mean it won't be good later. The only way to fix the problems something has is by recognizing them, and working on them. Imagine if people didn't point out the AI hallucinations? We'd never get anywhere with LLMs. Not shitting on the first guy, I get where he's coming from it's a damn cool time to be alive, LLMs are incredible and can only get better from here(that is of course if they don't keep slapping F*****g censors on it!). but it's important for us to recognise the flaws in the system so it and we can grow.