• Images
• Videos
• Documents
• Books
• Articles

Overview

Technically, simplicity corresponds in a drop in Kolmogorov complexity, which means that, for an observer, the shortest description of the situation is shorter than anticipated. For instance, the description of a consecutive lottery draw, such as 22-23-24-25-26-27, is significantly shorter than a typical one, such as 12-22-27-37-38-42. The former requires only one instantiation (choice of the first lottery number), whereas the latter requires six instantiations.

Simplicity theory makes several quantitative predictions concerning the way atypicality,⁷ distance, recency or prominence (places, individuals)⁸ influence interestingness.

Formalization

The basic concept of simplicity theory is unexpectedness, defined as the difference between expected complexity and observed complexity:

U = C exp − C obs . {\displaystyle U=C_{\text{exp}}-C_{\text{obs}}.}

This definition extends the notion of randomness deficiency.⁹ In most contexts, C exp {\displaystyle C_{\text{exp}}} corresponds to generation or causal complexity, which is the smallest description of all parameters that must be set in the "world" for the situation to exist. In the lottery example, generation complexity is identical for a consecutive draw and a typical draw (as long as no cheating is imagined) and amounts to six instantiations.

Simplicity theory avoids most criticisms addressed at Kolmogorov complexity by considering only descriptions that are available to a given observer (instead of any imaginable description). This makes complexity, and thus unexpectedness, observer-dependent. For instance, the typical draw 12-22-27-37-38-42 will appear very simple, even simpler than the consecutive one, to the person who played that combination.

Connection with probability

Algorithmic probability is defined based on Kolmogorov complexity:¹⁰ complex objects are less probable than simple ones. The link between complexity and probability is reversed when probability measures surprise¹¹ and unexpectedness:¹² simple events appear less probable than complex ones. Unexpectedness U {\displaystyle U} is linked to subjective probability P {\displaystyle P} as

P = 2 − U . {\displaystyle P=2^{-U}.}

The advantage of this formula is that subjective probability can be assessed without necessarily knowing the alternatives. Classical approaches to (objective) probability consider sets of events, since fully instantiated individual events have virtually zero probability to have occurred and to occur again in the world. Subjective probability concerns individual events. Simplicity theory measures it based on randomness deficiency, or complexity drop. This notion of subjective probability does not refer to the event itself, but to what makes the event unique.

External links

• A tutorial on Simplicity Theory
• Juergen Schmidhuber's page on interest and low complexity

References

1. Chater, N. (1999). "The search for simplicity: A fundamental cognitive principle?" The Quarterly Journal of Experimental Psychology, 52 (A), 273–302. http://www.dectech.org/publications/LinksNick/FoundationsTheoryAndMethodology/The%20Search%20for%20Sim%20plicity%20A%20Fundamental%20cognitive%20principle.pdf ↩

2. Chater, N. & Vitányi, P. (2003). "Simplicity: a unifying principle in cognitive science?". [Trends in Cognitive Sciences], 7 (1), 19–22. http://www.dectech.org/publications/LinksNick/FoundationsTheoryAndMethodology/simplicity,%20unifying%20principle%20in%20cognitive%20science.pdf ↩

3. Feldman, J. (2004). "How surprising is a simple pattern? Quantifying 'Eureka!'". Cognition, 93, 199–224. https://www.researchgate.net/profile/Jacob_Feldman/publication/8529241_How_surprising_is_a_simple_pattern_Quantifying_Eureka/links/5c6d761792851c1c9defe771/How-surprising-is-a-simple-pattern-Quantifying-Eureka.pdf ↩

4. Dessalles, Jean-Louis (2008). La pertinence et ses origines cognitives. Paris: Hermes-Science Publications. ISBN 978-2-7462-2087-4. 978-2-7462-2087-4 ↩

5. Dessalles, J.-L. (2013). "Algorithmic simplicity and relevance". In D. L. Dowe (Ed.), Algorithmic probability and friends - LNAI 7070, 119-130. Berlin, D: Springer Verlag. https://www.dessalles.fr/papers/Dessalles_11061001.pdf ↩

6. Schmidhuber, J. (1997). "What’s interesting?" Lugano, CH: Technical Report IDSIA-35-97. ftp://ftp.idsia.ch/pub/juergen/interest.ps.gz ↩

7. Maguire, P., Moser, P. & Maguire, R. (2019). "Seeing patterns in randomness: a computational model of surprise". Topics in Cognitive Science, 11 (1), 103-118. https://onlinelibrary.wiley.com/doi/pdf/10.1111/tops.12345 ↩

8. Dessalles, J.-L. (2013). "Algorithmic simplicity and relevance". In D. L. Dowe (Ed.), Algorithmic probability and friends - LNAI 7070, 119-130. Berlin, D: Springer Verlag. https://www.dessalles.fr/papers/Dessalles_11061001.pdf ↩

9. Maguire, P., Moser, P. & Maguire, R. (2019). "Seeing patterns in randomness: a computational model of surprise". Topics in Cognitive Science, 11 (1), 103-118. https://onlinelibrary.wiley.com/doi/pdf/10.1111/tops.12345 ↩

10. Solomonoff, R. J. (1964). "A Formal Theory of Inductive Inference. Information and Control, 7 (1), 1-22. http://world.std.com/~rjs/1964pt1.pdf ↩

11. Maguire, P., Moser, P. & Maguire, R. (2019). "Seeing patterns in randomness: a computational model of surprise". Topics in Cognitive Science, 11 (1), 103-118. https://onlinelibrary.wiley.com/doi/pdf/10.1111/tops.12345 ↩

12. Dessalles, J.-L. (2013). "Algorithmic simplicity and relevance". In D. L. Dowe (Ed.), Algorithmic probability and friends - LNAI 7070, 119-130. Berlin, D: Springer Verlag. https://www.dessalles.fr/papers/Dessalles_11061001.pdf ↩