Loras which are experts on certain topics?
Loras have been used in image generation as sub models that are highly trained on a specific subject. Causing the non focused image generation models to generate much better images about the lora subject.
For some reason it took till now for someone to try it with a large language model and a concentrated expert module? Everyone had this idea from the first month chatgpt got released.
I swear its similar to the idea of letting a vision model see a computer screen and giving its large language model colleague acces to a mouse and keyboard.
All of this stuff had already been tested, deemed to much of a culture shock and then shelved to slowly prepare the word for it all. Its the only explanation on why these ideas are only now getting tried.