Languages carry information. To fulfil this purpose, they employ a multitude of coding strategies. This book explores a core property of linguistic coding - called lexical diversity. Parallel text corpora of overall more than 1800 texts written in more than 1200 languages are the basis for computational analyses. Different measures of lexical diversity are discussed and tested, and Shannon's measure of uncertainty - the entropy - is chosen to assess differences in the distributions of words. To further explain this variation, a range of descriptive, explanatory, and grouping factors are considered in a series of statistical models. The first category includes writing systems, word-formation patterns, registers and styles. The second category includes population size, non-native speaker proportions and language status. Grouping factors further elicit whether the results extrapolate across - or are limited to - specific language families and areas. This account marries information-theoretic methods with a complex systems framework, illustrating how languages adapt to the varying needs of their users. It sheds light on the puzzling diversity of human languages in a quantitative, data driven and reproducible manner.

• Preface
• Contents
• Abbreviations
• List of Tables
• List of Figures
• 1. Introduction
• 2. Languages as Adaptive Systems
• 3. Language Change and Population Structure
• 4. Lexical Diversity across Languages of the World
• 5. Descriptive Factors: Language “Internal” Effects
• 6. Explanatory Factors: Language “External” Effects
• 7. Grouping Factors: Language Families and Areas
• 8. Predicting Lexical Diversity: Statistical Models
• 9. Explaining Diversity: Multiple Factors Interacting
• 10. Further Problems and Caveats
• 11. Conclusions: Universality and Diversity
• 12. Appendix A: Advanced Entropy Estimators
• 13. Appendix B: Multiple Regression Assumptions
• 14. Appendix C: Mixed-effects Regression Assumptions
• Bibliography
• Index