# Beyond Bigger Models: Toward a Modular Cognitive Architecture > Source: > Published: 2026-09-22 01:21:19+00:00 Beyond Bigger Models: Toward a Modular Cognitive Architecture From Exception-Driven Reasoning to AI Systems That May Eventually Design Their Successors Authors: Abstract The dominant approach to improving artificial intelligence has largely focused on increasing the scale and capability of neural models. This paper explores a different question: How much intelligence actually needs to exist inside model parameters? We propose a modular cognitive architecture in which neural computation is only one of several possible substrates of cognition. Depending on the application, functions may remain within a Neural Core or be distributed among rules, explicit memory, specialized tools, databases, or other computational components. The architecture is therefore not a fixed collection of modules. It is a design space in which cognition can be allocated according to capability requirements, hardware constraints, latency, energy consumption, communication cost, reliability, and validation cost. Several mechanisms are proposed: exception-driven reasoning, dynamic rules, cognitive compilation, cognitive decompilation, Neural Memory, cognitive locality, and architectural governance. A central principle is Negative Capability: an intelligent system should not only know how to perform a task, but also recognize when another computational substrate can perform it more efficiently, precisely, or reliably. This perspective may be particularly relevant to Edge AI, where computation, memory, energy, latency, and connectivity are constrained. More broadly, the paper proposes that future AI research may need to optimize not merely model intelligence, but the distribution of cognition across an entire computational system. The ultimate research question is whether sufficiently capable AI systems could participate in searching, constructing, testing, and refining successor AI architectures. The central proposal can be summarized as follows: ``` Cognitive System │ ┌──────┴──────┐ │ Neural Core │ └──────┬──────┘ │ ┌──────────────┼──────────────┐ │ │ │ Memory Rules Tools │ │ │ Dynamic Rules │ │ │ Cognitive Compilation │ ↕ │ Cognitive Decompilation │ └──── Neural Memory ────┐ │ Structured Data ``` This diagram should not be interpreted as a mandatory architecture. Some systems may require only a Rule-Based System + Small Neural Core. Others may benefit primarily from a Neural Core + Neural Memory. A more complex system may use a Neural Core, Memory, Rules, Tools, and specialized computational components together. The central idea is therefore not that every AI should be divided into the same modules, but that cognition should be distributed across computational substrates according to what is most effective for the application. The optimal architecture remains an empirical question. Neural models are remarkably general. They can represent language, perception, reasoning, pattern recognition, and many other capabilities within a common parameter space. However, generality does not imply efficiency. A neural model may perform tasks that could instead be handled by deterministic rules, exact calculators, databases, retrieval systems, specialized algorithms, or dedicated hardware. This raises a fundamental architectural question: Must every cognitive operation be performed through neural inference? We propose that the answer may be no. The relevant optimization target is therefore not simply: larger model → greater intelligence but rather: system architecture → distribution of cognition → system capability The research problem becomes: Given a required capability and a resource budget, what is the most effective distribution of cognition between neural computation and other computational substrates? A useful distinction is: [ \text{Model Intelligence} \neq \text{System Intelligence} ] The Neural Core provides general-purpose learned computation, but the complete AI system may also contain memory, rules, tools, sensors, databases, specialized algorithms, and governance mechanisms. Thus: [ \text{System Intelligence} f(\text{Neural Core},\text{Memory},\text{Rules},\text{Tools},\text{Interaction},\text{Architecture}) ] This does not imply that every system needs all of these components. A translation system might concentrate most of its capability in a Neural Core and Neural Memory. An industrial controller might consist largely of rules with a small Neural Core for uncertain perception. A robot might require neural perception, explicit memory, rules for safety, and specialized tools for planning and control. The architecture should follow the problem rather than the other way around. The architecture follows a simple engineering principle: If a task can be performed more cheaply, precisely, reliably, or quickly by a specialized computational substrate, neural inference need not perform that task. Task| Potential Substrate Exact arithmetic| Calculator / program Stable procedure| Rule Large precise information store| Neural Memory / database Repeated deterministic reasoning| Compiled rule Novel or ambiguous situation| Neural Core Specialized numerical optimization| Dedicated algorithm Safety constraint| Protected rule / governance layer This does not mean that neural computation should be removed whenever possible. The relevant question is the system-level trade-off. A neural implementation may sometimes be simpler or faster than introducing another module. The architecture should therefore be selected according to measured performance rather than ideological preference for modularity. The concept of Negative Capability provides an important principle for this architecture. In this context, it refers to an engineering capability: The ability of a cognitive system to recognize when it should not perform a task through its own neural computation. A capable Neural Core should be able to recognize situations such as: Negative Capability therefore complements active reasoning. We can express the relationship as: Exception-Driven Reasoning → When should Neural intervene? Negative Capability → When should Neural refrain from intervening? This principle is important because externalizing cognition is useful only when the system knows when and why to externalize it. Traditional neural systems often attempt to process every input through essentially the same learned computational pathway. A modular architecture can instead establish stable behavior through rules and invoke neural reasoning primarily when the rule system encounters an exception. The basic loop is: Stable Case ──> Rule ──> Normal Operation Exception ──> Neural Core ──> Reasoning / Interpretation This has two potential advantages. First, predictable cases may be processed without expensive neural inference. Second, the Neural Core can concentrate computational resources on cases where generalization, ambiguity, or contextual reasoning is actually required. The architecture therefore treats exceptions not merely as failures, but as information about where additional intelligence is required. Exceptions do not necessarily remain exceptions forever. Suppose a Neural Core repeatedly encounters a particular class of cases and discovers that the same reasoning pattern is applicable. The system may transform that repeated pattern into a verified dynamic rule: Rule ↓ Exception ↓ Neural Reasoning ↓ Pattern Discovery ↓ Validation ↓ Dynamic Rule ↓ Rule This creates a learning loop in which some neural reasoning is gradually transformed into cheaper deterministic computation. However, rule creation must not be automatic deployment. A candidate rule should pass appropriate validation, including: consistency testing, conflict detection, regression testing, boundary-condition testing, and application-specific safety constraints. A dynamic rule that improves one class of cases while silently damaging another is not a successful compilation. We call this process Cognitive Compilation. Conventional compilation transforms a higher-level representation into a more directly executable form. Cognitive Compilation similarly attempts to transform expensive, repeatedly required reasoning into a cheaper and more explicit computational structure. The transformation can be viewed as: [ \text{Neural Reasoning} \rightarrow \text{Validated Rule} ] The objective is not to eliminate neural intelligence. It is to avoid repeatedly paying the full cost of neural reasoning when a reliable deterministic representation has already been discovered. This provides a possible path toward reducing the amount of computation that must remain inside the Neural Core. Compiled cognition should not be considered permanent. A rule that is correct under one environment may become incorrect when the environment changes, the underlying data distribution changes, new exceptions appear, the original assumptions become invalid, or the rule begins to conflict with newly learned information. The system therefore needs a reverse process: [ \text{Rule} \rightarrow \text{Failure / Drift} \rightarrow \text{Decompilation} \rightarrow \text{Neural Relearning} \rightarrow \text{New Rule} ] We call this Cognitive Decompilation. This creates an adaptive cycle: ``` ┌───────────────┐ │ Rule │ └───────┬───────┘ │ ▼ Exception │ ▼ Neural Reasoning │ ▼ Pattern Discovery │ ▼ Validation │ ▼ Dynamic Rule ──────┐ │ ▼ Environmental Drift │ ▼ Cognitive Decompilation │ ▼ Neural Relearning │ └─────────> (Return to Rule) ``` The architecture thus treats rules as compiled knowledge with a validity lifecycle, rather than permanent truths. The phrase "Neural Memory" should not be understood as meaning that all memory is removed from the Neural Core. Memory can exist along a continuum: [ \text{Implicit (Neural Core: Distributed Parametric)} \longleftrightarrow \text{Explicit (Neural Memory: Structured Retrievable)} ] Simple and frequently used information may be handled directly by the Neural Core. For more complex, precise, rare, or extensive information, the Neural Core cooperates with Neural Memory. The relationship is characterized by functional coordination: the Neural Core does not need to contain every book in the library; it needs to recognize when a precise reference is required and retrieve it efficiently via Neural Memory. Neural Memory can manage different types of information, including semantic memory, episodic memory, structured facts, relationships, historical records, and contextual databases. The separation should be functional rather than absolute. For some applications, maintaining information directly within model parameters may be more efficient. For others, explicit memory may provide advantages in capacity, precision, updateability, traceability, retrieval, or persistence. Therefore, the question is: Which information should be represented parametrically, and which information should be represented explicitly in Neural Memory, given the system's requirements? Modularity introduces an important cost that cannot be ignored: Communication. A modular architecture is not automatically efficient. If every operation requires expensive communication between distant components, the cost of modularity may outweigh the savings from specialization. We therefore introduce Cognitive Locality as a design principle. Modules may communicate through local memory, shared memory, on-chip interconnects, in-package communication, dedicated accelerators, or external networks. These mechanisms have fundamentally different latency and energy characteristics. Consequently, "modular" does not necessarily mean "networked." For an integrated Edge AI system, several cognitive substrates could potentially coexist within the same hardware architecture, allowing relatively efficient communication. The relevant quantity is measured communication cost rather than an assumption that modularity is inherently expensive. The architecture should ultimately be evaluated at the system level. One possible model is: [ C_{\text{total}} C_{\text{neural}} + C_{\text{memory}} + C_{\text{rule}} + C_{\text{tool}} + C_{\text{communication}} + C_{\text{validation}} ] where: [ C_{\text{neural}}: \text{neural computation cost} ] [ C_{\text{memory}}: \text{memory access and management cost} ] [ C_{\text{rule}}: \text{rule execution and management cost} ] [ C_{\text{tool}}: \text{specialized tool execution cost} ] [ C_{\text{communication}}: \text{inter-component communication cost} ] [ C_{\text{validation}}: \text{cost of maintaining correctness and safety} ] This model is not intended as a universal final equation. It is a framework for asking the correct experimental question: Does moving cognition outside the Neural Core actually reduce total system cost while preserving or improving required capability? The answer may differ across applications. The proposed architecture should therefore not be interpreted as a fixed blueprint. Instead: [ \boxed{ \text{Modular Cognitive Architecture} \equiv \text{Architecture Design Space} } ] For one application, a Rule-Based + Small Neural Core configuration may be optimal. For another, a Neural Core + Neural Memory configuration may be sufficient. The architecture does not assume that externalization is always beneficial. It asks whether a different distribution of cognition produces a better system under the relevant constraints. Edge systems often operate under constraints involving computation, memory capacity, energy, thermal limits, latency, connectivity, and hardware cost. A conventional strategy is to make the neural model smaller. An alternative is: «Do less neural computation in the first place.» Conventional Edge AI: Large Neural Model ──> Most Computation ──> Answer Modular Edge AI: Neural Core <───► [Rules / Memory / Tools / Specialized Logic] Whether the second architecture is superior is precisely what should be measured. Possible metrics include capability, latency, energy per task ((E_{\text{task}})), memory footprint, communication cost, hardware cost, reliability, and adaptation cost. Edge AI therefore provides a practical environment in which the hypothesis can be tested under real resource constraints. The deeper architectural question is not merely whether a model can perform a task, but where should the capability reside? A capability may migrate along the following gradient: [ \text{Neural parameters} \rightarrow \text{Explicit memory} \rightarrow \text{Rule} \rightarrow \text{Specialized algorithm} \rightarrow \text{Dedicated hardware} ] The optimal representation may change over time. A newly encountered problem may initially require Neural reasoning. Repeated solutions may become rules. Frequently accessed information may be promoted into a faster memory representation. Changing information may be moved from static parameters into explicit memory. This suggests that future AI architectures could potentially perform not only inference, but also architectural redistribution of cognition. A sufficiently capable AI system may eventually be able to analyze its own architecture and identify functions that could be implemented more efficiently elsewhere. The conceptual loop is: Existing AI ↓ Analyze Capability & Cost ↓ Identify Computational Bottlenecks ↓ Consider Alternative Cognition Distributions ↓ Design Candidate Architecture ↓ Generate Components ↓ Build Tests ↓ Measure Capability / Cost / Reliability ↓ Revise Architecture ↓ (Loop) The AI would participate in architectural decomposition, component selection, implementation, experimental evaluation, failure analysis, and architectural revision. The research question is therefore: «Can an AI system participate in automated search and validation of successor cognitive architectures?» This should be treated as a research hypothesis rather than an assumption about future AI capabilities. If an AI system can modify its own architecture, unrestricted optimization creates an important engineering problem. The system should therefore distinguish between: Optimizable architecture: Components that may be redesigned to improve capability, efficiency, latency, energy consumption, memory use, or reliability. Protected invariants: Constraints that the architecture search is not permitted to remove or weaken beyond defined limits, such as permission boundaries, human override, auditability, access restrictions, safety constraints, and required validation procedures. This can be represented as: [ \text{Architecture Search} \subseteq \text{Allowed Design Space} ] where the allowed design space is constrained by explicitly defined Safety Invariants. A future research direction is to define measurable Safety Invariant Margins, rather than relying solely on qualitative descriptions of safety. The proposal becomes meaningful only if it can be experimentally tested. A basic experiment could compare architectures performing the same capability set under identical hardware constraints: Baseline: Monolithic Neural Model Modular candidate: Smaller Neural Core + Selected external cognitive substrates Different configurations should be evaluated systematically: The objective is to measure the trade-offs across: 19.1 Capability Measure whether the architecture satisfies the required task set using predefined evaluation criteria. 19.2 Total Cost Measure (C_{\text{total}}), including neural computation, memory, communication, tools, and validation where measurable. 19.3 Energy Measure energy consumed per completed task ((E_{\text{task}})), which may be more informative than parameter count. 19.4 Latency Measure both average latency and worst-case or tail latency. 19.5 Reliability Measure failure rate, rule conflicts, retrieval errors, hallucination or fabrication, and recovery speed. 19.6 Adaptation Test whether Cognitive Compilation reduces repeated reasoning cost, then alter the environment to measure if Cognitive Decompilation recovers stability. The architecture leads to several empirical questions: RQ1: How much capability can be externalized from Neural Core parameters without reducing required system capability? RQ2: Under what conditions does a smaller Neural Core plus specialized cognitive substrates outperform a larger monolithic model in system-level efficiency? RQ3: When does communication cost eliminate the benefits of modularization? RQ4: What types of information are better represented parametrically versus explicitly in Neural Memory? RQ5: Can repeated neural reasoning be reliably transformed into validated dynamic rules? RQ6: How quickly can a system detect when a previously compiled rule has become invalid? RQ7: Can AI systems automatically search the architecture design space? RQ8: What safety invariants are required when an AI system is permitted to modify its own architecture? Several claims should be deliberately avoided. This proposal does not claim that: Instead, the proposal is an architectural hypothesis: «The capability of an AI system may depend not only on the size and intelligence of its Neural Core, but on how cognition is distributed across the entire system.» The dominant trajectory of AI development has emphasized increasingly capable neural models. This paper proposes a complementary direction. Instead of asking only: «How can we make the Neural Core more powerful?» we should also ask: «What cognition actually needs to remain inside the Neural Core?» A future AI system may distribute cognition among neural parameters, explicit memory, rules, tools, specialized algorithms, and hardware. The architecture should be treated as a design space, supported by Exception-Driven Reasoning, Negative Capability, Cognitive Compilation, Cognitive Decompilation, and Architectural Governance. The central hypothesis can be stated simply: [ \boxed{ \text{AI Intelligence} \ne \text{Neural Model Size Alone} } ] [ \boxed{ \text{System Capability} f( \text{Neural Core}, \text{Memory}, \text{Rules}, \text{Tools}, \text{Communication}, \text{Architecture} ) } ] The practical objective is to find an effective distribution of cognition that satisfies the required capability, cost, latency, energy, reliability, and safety constraints. Ultimately, this direction leads to a more ambitious possibility: «An AI system may eventually become capable of experimentally searching for and constructing better cognitive architectures.» Co-Authors ✍️ GPT-5.6 Luna ✍️ Google-based AI ✍️ Herbert Huang — Founding Observer & Human Proxy