Entropy Estimators for Markovian Sequences: A Comparative Analysis

• DOI: 10.3390/e26010079
• ArXiv number: 2310.07547

Entropy estimation is a fundamental problem in information theory that has applications in various fields, including physics, biology, and computer science. Estimating the entropy of discrete sequences can be challenging due to limited data and the lack of unbiased estimators. Most existing entropy estimators are designed for sequences of independent events and their performances vary depending on the system being studied and the available data size. In this work, we compare different entropy estimators and their performance when applied to Markovian sequences. Specifically, we analyze both binary Markovian sequences and Markovian systems in the undersampled regime. We calculate the bias, standard deviation, and mean squared error for some of the most widely employed estimators. We discuss the limitations of entropy estimation as a function of the transition probabilities of the Markov processes and the sample size. Overall, this paper provides a comprehensive comparison of entropy estimators and their performance in estimating entropy for systems with memory, which can be useful for researchers and practitioners in various fields.

• Full paper

Related research projects

MdM-IFISC-2

Maria de Maeztu 2023-2026

P.I.: Ernesto Estrada, Ingo Fischer, Emilio Hernández-García, Rosa Lopez, Claudio Mirasso, Jose Javier Ramasco, Raúl Toral, Roberta Zambrini

After 15 years of its existence, IFISC can point to a proven track record of impactful research. The previous 2018-2022 MdM award has significantly enhanced the institute's capabilities, as demonstrated by an …

APASOS-UIB

A Physics approach to sociotechnical systems: from theory to data analysis.

P.I.: Maxi San Miguel, Raúl Toral

APASOS objective is to use mathematical and computational methods combined with data and physics thinking to model complex socio-technical systems. APASOS is organized into two workpackages (WP). WP 1 focuses on models …