As I wait for the Courts to rule on my two petitions (Habeas & FOIA) my thoughts have turned to this enormous AI bubble that is about to burst. First, it’s important to understand that LLM’s are not AI. As the link points out, AI would mimic the intelligence of a specific animal, for example – a DOG. Thus, if a computer could mimic the intelligence of a dog, it would be DOG Artificial General Intelligence (AGI).
As I love dogs (except for their poop) I would love an AGI android dog in the form of a Corgi. And I would bet that there would be a HUGE demand for such an android Dog. So why don’t we have one? Because an LLM can never achieve DOG intelligence.
A very simplified explanation of what an LLM does is best explained with a numbers example. I’ll call it a large number model or LNM. I train my model on the counting number set, say up to 1 billion. So my LNM knows that the counting set starts with 1 and ends with 1 billion. So when you ask it “What number comes after 300,207?” It easily returns the answer “300,208”. But if you ask it ‘What number comes before 1 or after 1 billion?” – it will make something up. This is what they call a hallucination. When a model doesn’t know something, it will make the answer up, from its knowledge base. In this over-simplified case my LNM’s wrong answer will be from its dataset (1 to 1 billion).
In this context, you can fundamentally understand what a LLM chatbot like ChatGPT is doing. Large Language Models are essentially just next word predictors. (Like your phone does when it tries to autocomplete words for you.) Although, a LLM is that system on steroids. Which is fine, until you understand the costs behind that chatbot writing those words back to you.
And that’s why the AI Bubble will pop, because of the unsustainable costs of keeping it running. Note I’m not saying it’s going to go away, I think Google & Microsoft will keep it around in some form, but an LLM will never achieve DOG AGI because it cannot. But this research may at sometime in the future lead to my dream of a droid Corgi.