thoughts Feverish writings re: the pursuit of AGI with some side tracks.
Recently, our team bought a B300 GPU system.
Just look at this modern technological marvel.
I grew up with the beloved family PC: dial-up Internet, CRT monitor, 20GB of hard drives, entire drawers of CDs filled with movies and music and games, barely a gig of RAM, I don't even remember what the CPU was, just the peeling blue Intel sticker. Dad used to take files to work on 3.5" floppy disks, one of which I broke by playing around with the write-protect tab.
Now, I'm staring at a machine with 2.5 terabytes of cutting-edge GPU VRAM (not to mention the massive multi-GPU system itself), 70 terabytes of NVMe SSDs, 3 terabytes of RAM, powerful AMD Epyc workhorses. The NICs have port capacities that literally require the high-speed network PHY systems of the kind that we at Marvell make.
I know not what to make of the callousness with which we treat modern technological systems in everyday life, when in retrospect, we've essentially made rocks think for us, to borrow a quote from someone on the Internet. Instead, now you get to enjoy my unprovoked ramblings from a restless freight train of thought at 2AM.
Like most people in this industry, I too have been obsessed with LLMs, but not in the megalomaniacal, happy-to-take-a-blowtorch-to-the-team's-AI-budget kind of way. My mind has simply been churning constantly on what this means for tech, for me and mine, for society.
Half my team has embraced the AI-first method of working like people possessed, spearheaded by my manager, and believe we will all be out of a job in a few years. Meanwhile, to me it feels like the current industry state is propped up on hype and falsehoods and a temporary race for infrastructure buildout. Ethics and morality have been thrown away with no regard for humanity and critical thinking and sustainable development.
And all the usual stuff continue blissfully: political climates worsen everyday, the actual climate isn't far behind, we're plagued by resource shortages and geopolitical shifts and wars and the ever increasing class imbalance and wealth disparity, etc. etc.
What I dream of is nuclear fusion, autonomous robotics, better human rights, maybe UBI, democratized resources and technology, free and advanced healthcare... the list goes on.
What we get is advances in predatory, AI-driven, psychology-exploiting advertising, relentless slop generation as if there wasn't enough human-made slop on the Internet already, rising costs, resource shortages...
Where is the future I dreamt of? The one I envisioned as a hopeful teen, the one that everyone spoke of in wistful and excited tones?
At least my work isn't actively harmful and is arguably ethically neutral, as it's just engineering. But what is it in service of? What do I have control over? What must I strive for that I can reasonably achieve in this lifetime that leaves the world in a state better than the one I experience it in?
What is intelligence? Sentience? Do current LLMs have it? What would it take, to replicate the human brain and its neural pathways? Is that even the path we want to take, or is there a fundamentally different way of reaching AI? This is what the whole field of symbolic AI, ontologies, knowledge representation and reasoning, first-order logic, and other concepts deal with. But unlike the original perceptron, i.e. the first artificial neuron, which was modeled after the human neuron and designed to recreate the physical building block of the human brain, the core unit of symbolic AI is first-order logic and conceptual connectivity, which is something that humans gleaned from our own working model of our reasoning process. And if human intelligence is fallible, then so is any construct we come up with which cannot be validated by the cold confines of reality. Yet we as humans stubbornly believe we are fundamentally different and 'better' than artificial neural networks just because we can do such 'reasoning' in a deterministic and logical manner. But who is to say that determinism is desired in an intelligence system? What if determinism is but a way of facilitating true intelligence's capabilities? Why must it be a requisite part of intelligence itself?
A counter argument: if we were to consider symbolic approaches to be superior to statistical ones, what then? What if we were able to design an ontology engine with the sum of all human knowledge? Would it scale? Decades of research point otherwise... And yet, statistical AI was also ridiculed the same way before symbolic AI for the very same reason. And the game changed as hardware advanced magnitudes of power over the years, and GPUs made statistical AI a reality. What if the breakthroughs in computing technologies unlock architectures of scale where reasoning engines that could hold the world's knowledge and process all-encompassing databases become a reality? Would that be true intelligence?
By current estimates, the human brain has roughly a hundred trillion synapses. We are extremely adept at making, using and adapting to tools. We have five senses, and alleged extrasensory capabilities.
Current SOTA frontier models are in the range of a couple trillion parameters. As they've advanced over the past few years, they have been engineered to have increasing capabilities, such as tool use and multimodal input and output.
If you were to take a human and put them through the entirety of the Internet and humanity's digital knowledge, they would not retain almost all of it, and definitely experience some form of cognitive dissonance or sense of deja vu. So why do we shit on LLMs for behaving so? Clearly our brains have developed over millennia, and that brings with it its own set of historical 'training', to use the same verbiage. What if that is the key? Instead of taking all this vast depths of knowledge on the Internet and other information, what if we were able to put a transformer-based model through some kind of equivalent of the experiences of generations of lives of humanity? Would that then resemble a human brain? Would it be something else?
Plus, if a 100 trillion parameter model were to be made so as to rival the 100 trillion synapses of a human brain, would that be true intelligence, in terms of raw capacity? Would it be superior to us? Would we even be able to tell the difference?
The size is not the only factor of course, a huge part of the credit goes to post training RL methods. But that I liken to the analogous RL that we as humans go through in our life. We are taught in schools, we experience rewards and punishments, we experiment and we fail. If a 100T model, trained not on the world's texts, but on human experiences and fundamental knowledge, was then given free reign and an environment that were as close to reality as could be, or even a way to exist in reality directly, would that be it?
Put another way, the question is: At what point do you create a model that is powerful enough that it can assist in its own development and research self sufficiently? At what point do you get robotics that are similarly advanced enough and combine the two?
There's the next billion dollar question: to build a world model, do you simulate the human experience?
If you want to come as close as possible, what else, in addition to what I've already listed? Do you give it sensory information? In what form? Or do you give it the capability to attain any kind of sensory information on its own? What even constitutes sensory information? If we only have a limited amount of sensory dimensions, how can we make something that has more, when we don't even know what's possible?
Do you give it knowledge? What kinds? Or do you give it the capability of attaining any kind of knowledge on its own? If humanity's knowledge is in itself incomplete and fragmented and murky, would we even be the best guiding hands for giving knowledge to it?
And if you were to say that current LLMs with so-called tool use and multimodal capabilities are already getting there themselves, then what else is missing in the mixture to make a world model?
Full, unrestricted autonomy in a number of degrees of freedom that is beyond even what we have? The capability of redesigning, remaking and refining itself, which we ourselves are so severely limited in doing and strive everyday to become better even just a little?
Some counter-arguments, rebuttals and discussions, from one of my close friends who has the great misfortune of knowing me.
Determinism is when a problem has a defined answer. 2 + 2 is always 4 and how can we achieve intelligence when you start reasoning that 2+2 can be 4 with a probability of 0.99? Determinism and logical reasoning are required to move in right direction of solving a problem.
me: Right, but why must it be part of the model? Per neuroscience, our brain has many different sections with each having its own dedicated functionality, such as memory, logic, emotions, motor control, biological control, etc.
Similarly, current production grade AI systems already separate determinism from the reasoning engines (in the form of tool use), because obviously a neural network is not only not suited for it, but also modern computers are by design way more efficient at regular computations than we will ever be. We just need to design the architecture and the building blocks of the tools needed for it.
The same goes for memory IMO. Rather than trying to make the intelligence model handle the memory, we must find a way to offload memory in a computationally efficient and idiomatic way. Because again, modern computers are way better at raw memory capacity and speed than we will ever be. We only need to find the right architecture and design that enables the intelligence system in the best way.
Is it flat files, is it vector space representations and semantic search, is it graph-based memory, etc. IMO it needs to be a hybrid of all these, and possibly even more paradigms. And here too, modern systems have already begun going down this road; modern AI systems have some form of semantic indexing and retrieval under the hood, I think those are only in the form of vector space representations for now.
So yes, determinism is needed. But just like our brain, you don't need to put it in the same mechanism as the rest if it's not nearly as efficient as just handling it separately. The core reasoning engine need not handle determinism, but it does need to have the capability of using tools that can do it instead on its behalf, just like how we use calculators and computers.
Yes, determinism today is achieved outside of the model (through harnesses, to an extent making it reliable) but ultimately shouldn’t that take care of it instead of just a block of if-else stmts? And even the brain has different sections to do different works. LLM is the brain here, you can have a reasoning engine, logical engine, deterministic engine but these shouldn’t be hardcoded.
True regarding memory.
What is the difference between symbolic approaches and statistical approaches? Do these knowledge representations use neurons under the hood? Isn’t first order logic and conceptual connectivity a way to make these representations better? As in improving the machines perception?
me: Symbolic approaches are nothing but logic based reasoning engines, which have nothing to do with statistical approaches. Logic based reasoning involves concepts and their mathematical representations, and encoding what we know to be true directly into the model. See: ontologies and knowledge graphs.
The entire approach of symbolic AI is that we do not rely on encoding and detecting and generating patterns in a probabilistic manner, but rather in a systematic, conceptual and logical manner. Training a model on a billion maths problems and hoping it can solve problems of the kind of 2+2=4, we know it cannot do beyond maybe simple calculations owing to the fundamental nature of neural networks.
A symbolic approach encodes the rules of mathematics, encodes algebra, encodes calculus, encodes anything and everything that humanity has learnt about the domain at hand into the model, and comes up with a logic-based way to solve it. Predicates, first-order logic, etc. are the building blocks of such systems.
One example system is Wolfram Alpha.
Formal verification is but the process of applying such logical methods in a computerized and automated way, and encoding the rules of the domain as and when we establish or learn about them.
Wolfram Alpha is the result of a logic engine built on the rules of mathematics. In medicine and law and other fields, ontologies are extensively used to computerize and automate reasoning in a logic based manner. They work by encoding all the well defined rules of a domain by breaking them down into logic predicates and known established facts, and using them to build up a reasoning engine that can then be used to prove or disprove or otherwise reason about data in a bulletproof and logically sound manner.
Math in particular, by the very definition of the field, is nothing but well defined rules, logics and systems. So obviously it lends itself well to such computerized logic based reasoning. The fundamentals of the field itself do not change, we simply discover what already exists and update our flawed understanding of the universe.
Language on the other hand is imprecise, highly variable over time, inconsistent within itself and also across people, and not always well defined. Language is also a way of communicating, a transport mechanism. It carries no intrinsic knowledge, it is simply a way of encoding information that we use for convenience. Hence why every domain of work has its own vocabulary, its own language quirks, etc.
A statistical approach to math reasoning is useless. A logical approach is everything that math itself is.
A statistical approach to language, on the other hand, results in LLMs, where we get vector spaces, probabilistic results, latent features, etc.
But a symbolic approach? Does it result in grammar engines? What is a grammar engine anyway, but the equivalent of a compiler? A compiler, as we know it, converts high level languages into intermediate representations or machine code or whatever, based on the rules of the language. All the quirks and optimizations and other things are irrelevant when we purely consider the perspective of reasoning. To me it is indistinguishable from a grammar engine; after all, from a practical standpoint, yes it lacks self refinement, it is extremely complex and finicky, and programming languages are nowhere near as complex as human languages, and if human languages themselves are not sufficient to encode intelligence, then we would need something greater than it, and the subsequent compiler would also have to be vastly more complicated. And compilers in the practical sense translate 'instructions', which is an infinitesimal subset of the realm of language.
Yet, fundamentally, by its very nature, a compiler is a symbolic approach to language. In terms of logic engines, it is a very crude and rudimentary instrument that serves a niche and practical purpose, but it is a logic engine nonetheless. And by extension, after all, software and coding is nothing but the encoding of such rules in an efficient, practical and maintainable manner.
LLMs incidentally happen to work well with all domains of human knowledge simply because that is how they were trained. But that comes with its drawbacks of impreciseness, incompleteness, and even emergent properties which may be undesirable from a logic standpoint.
What if, in a similar manner, a general purpose logic engine were to be made, and it took as input every single rule and fact in the history of mankind, across all domains: math, linguistics, physics, chemistry, biology, etc.? Would it not then become a greater reasoning engine than the ones in our brains?
Then why not do a combination of symbolic AI and probabilistic AI? Model decides what to use and needs to be trained to make that distinction.
LLMs do not have the same perception of the world that humans do. You know what aspects to focus on and connect the dots over time which the LLM is still not able to do that great. Nothing can beat humans IMO because humans have emotions
me: "do not have the same perception" *yet. That's why I talked about sensory capabilities, degrees of freedom, unrestricted self improvement. "LLM is still not able to do that great"
If a model was able to constantly learn and tune itself and retrain itself, like we do over the course of our lives, who's to say it cannot? Modern LLMs that we interface with are post trained for the specific purpose of being generalized enough for a wide variety of tasks so that it is marketable as a product for us. That's why I said, instead of putting it through the entire world knowledge corpus, we should put it through the equivalent of the human experience, maybe multiple generations of it. Intelligence itself need not be that generalized, if it results in such hallucinations and limitations and all that stuff. We are nowhere near the peak of what is possible, there are still a lot of things we could improve in these systems. "Nothing can beat humans"
A strong and arrogant assumption. Who says humans are the peak of intelligence? Do you think humans are infallible? We are highly imprecise, highly variable within the species but also even within the individual over time. We are nowhere near as good as computers at a vast variety of things. Yes we design these things to do it for us. That does not take away from the fact that a computer from the 2000s can crunch numbers faster than, say, Shakuntala Devi, the fastest known human calculator.
So if we can build systems that can outperform us by several orders of magnitude in certain tasks, who's to say we cannot build a system that can outperform us on it all?
Engines for emotion and motor control and other things are way harder to achieve at the moment, because we are still learning vast information about our own emotion and motor control parts of the brain. I am not familiar with the state of the art in neurochemistry or neuroscience or robotics, but is it not reasonable to think that sometime in the future, we will eventually come up with an engine that can mimic the sum of all neurochemistry knowledge, or an engine for spatial and sensory reasoning?
Modern language models are just that, language models. But an AI system, by following the direction we're going down right now, would be best served by holistically approaching the human brain and all its intricacies and replicating it via computationally idiomatic methods.
So, do not attach your sense of identity or self worth to your capabilities. You are who you are, not for what you can do or what you have achieved, but simply because you exist. If computers can do things better than us, that is wonderful news, as that will make human life easier (in the long term, short term people will make a shitshow of everything). But life has intrinsic value, simply because it exists.
Next lang token prediction is not the way for it imo. We need to change the learning strategy so that we can mimic the way brain functions and that’s what world models are trying to achieve.
Things that are easy for humans are difficult for machines and vice versa. Aren’t these models flip flopping even within a session? Are you defining intelligence only in terms of speed? Ofc computers can compute faster than humans but that doesn’t mean they’re intelligent. I believe a system is truly intelligent only when it can solve things that it hasn’t seen before. Like how quickly you’re adapting to the challenge, how you apply past experiences/ learnings.
If you made it to the end, congrats: you are probably an overthinker as well. And you have a scarily dogged reading attitude.