Presentation 1-2, 13:40~14:10.
Speaker
Takato Hashino (Kyushu University)
Title
The Shannon Entropy Estimation via the Pitman--Yor Process
Abstract
The Shannon entropy, also known as the Shannon--Wiener index, is a fundamental measure for quantifying diversity and model complexity in various fields such as information theory, ecology, and genetics. While many existing studies assume that the true number of species is known, this assumption is often unrealistic in practice. Some recent works relaxed this constraint. In this work, we build upon these developments and propose a new entropy estimation method based on the Pitman--Yor process, a representative approach in Bayesian nonparametrics. By approximating the true distribution with an infinite-dimensional distribution, our method enables stable estimation even when the observed number of species is smaller than the true number. This approach provides a principled way to handle uncertainty in species richness and improves the reliability of entropy-based diversity assessments.