[Submitted on 30 Jan 2023 (
[v1](https://arxiv.org/abs/2301.12987v1)), last revised 11 Apr 2024 (this version, v4)]# Title:The Optimal Choice of Hypothesis Is the Weakest, Not the Shortest
[View PDF](/pdf/2301.12987)
[HTML (experimental)](https://arxiv.org/html/2301.12987v4)
Abstract:If $A$ and $B$ are sets such that $A \subset B$, generalisation may be understood as the inference from $A$ of a hypothesis sufficient to construct $B$. One might infer any number of hypotheses from $A$, yet only some of those may generalise to $B$. How can one know which are likely to generalise? One strategy is to choose the shortest, equating the ability to compress information with the ability to generalise (a proxy for intelligence). We examine this in the context of a mathematical formalism of enactive cognition. We show that compression is neither necessary nor sufficient to maximise performance (measured in terms of the probability of a hypothesis generalising). We formulate a proxy unrelated to length or simplicity, called weakness. We show that if tasks are uniformly distributed, then there is no choice of proxy that performs at least as well as weakness maximisation in all tasks while performing strictly better in at least one. In experiments comparing maximum weakness and minimum description length in the context of binary arithmetic, the former generalised at between $1.1$ and $5$ times the rate of the latter. We argue this demonstrates that weakness is a far better proxy, and explains why Deepmind's Apperception Engine is able to generalise effectively.
Submission history #
From: Michael Timothy Bennett [[view email](/show-email/39071763/2301.12987)]
**Mon, 30 Jan 2023 15:29:40 UTC (281 KB)**
[[v1]](/abs/2301.12987v1)**Mon, 6 Mar 2023 01:54:22 UTC (279 KB)**
[[v2]](/abs/2301.12987v2)**Tue, 25 Apr 2023 07:23:31 UTC (58 KB)**
[[v3]](/abs/2301.12987v3)**[v4]** Thu, 11 Apr 2024 05:02:10 UTC (58 KB)
Current browse context:
cs.AI
References & Citations
...
Bibliographic Explorer
(What is the Explorer?) Connected Papers
(What is Connected Papers?) Litmaps
(What is Litmaps?) scite Smart Citations
(What are Smart Citations?)# Code, Data and Media Associated with this Article alphaXiv
(What is alphaXiv?) CatalyzeX Code Finder for Papers
(What is CatalyzeX?) DagsHub
(What is DagsHub?) Gotit.pub
(What is GotitPub?) Hugging Face
(What is Huggingface?) ScienceCast
(What is ScienceCast?)# Demos Influence Flower
(What are Influence Flowers?) CORE Recommender
(What is CORE?)# arXivLabs: experimental projects with community collaborators arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.