Pages: 508
ISBN: 978-1-80061-675-2 (hardcover)
Markov Decision Processes (MDPs) form a cornerstone of applied probability, with over 50 years of rich research history. Throughout this time, numerous foundational books and thousands of journal articles have shaped the field. The central objective of MDP theory is to identify the optimal control strategy for Markov random processes with discrete time. Interestingly, the best control strategies often display unexpected or counterintuitive behaviors, as documented by a wide array of studies.
This book gathers some of the most compelling examples of such phenomena while introducing new ones. By doing so, it serves as a valuable companion to existing textbooks. While many examples require little to no prior knowledge, others delve into advanced topics and will primarily interest specialists.
In this second edition, extensive revisions have been made, correcting errors and refining the content, with a wealth of new examples added. The range of examples spans from elementary to advanced, requiring background knowledge in areas like measure theory, convex analysis, and advanced probability. A new chapter on continuous time jump processes has also been introduced. The entire text has been reworked for clarity and accessibility.
This book is an essential resource for active researchers and graduate students in the field of Markov Decision Processes.
Preface
Finite-Horizon Models
Homogeneous Infinite-Horizon Models: Expected Total Cost
Homogeneous Infinite-Horizon Models: Discounted Cost
Homogeneous Infinite-Horizon Models: Average Cost and Other Related Criteria
Continuous-Time Markov Decision Processes
Appendix A Proofs of Auxiliary Statements
Afterword
Bibliography
Index
Notation
List of the Main Statements
Most of examples are of academic nature, useful and interesting to active researchers in the field, from graduate students to international experts. Some undergraduate students can use this book as the complement to textbooks on Markov decision processes. Several subsections (e.g., on gambling, queues, games) can be interesting for a more general audience.
Pages: 248
ISBN: 978-981-98-0168-8 (hardcover)
This book presents an overview of the most recent research and findings in the field of approximation and regularisation methods for operator-functional equations, and explores their applications in electrical and power engineering. It presents the state of the art in building operator theory, regularised numerical methods, and the verification of mathematical models for dynamical models based on integral and differential equations. Special attention is paid to Volterra models, a powerful tool for modelling hereditary dynamics.
This book begins by exploring the solvability of singular integral equations and moves on to study approximation methods for linear operator equations and nonlinear integral equations. Following this, it examines loaded equations and bifurcation analysis, before concluding with an investigation of the applications of the contents of the book in electrical engineering and automation. Each chapter provides an overview and analysis of the relevant problem statements, outlines current methods within the field, and identifies future directions for research.
With an interdisciplinary approach, this book is essential reading for anyone interested in operator-functional equations. Graduate students and professors in the fields of applied mathematics, physics, materials science, and numerical analysis will find this work insightful and valuable, as will industry professionals in related fields.
Introduction
Solvability of Singular Integral Equations on Banach Function Spaces
Approximation Methods for Linear Operator Equations and Nonlinear Integral Equations
Loaded Equations and Bifurcation Analysis
Applications in Electrical Engineering and Automation
This book is suitable for both graduate students and professors in applied mathematics, physics, material science, and numerical analysis. It is also suitable for industry professionals in the fields of electrical and thermal engineering, combustion, and biomass.
Pages: 408
ISBN: 978-981-12-9065-7 (hardcover)
This is the first book on the problems of mathematical modelling that combines a generalised approach to the problem of mathematical modelling processes and phenomena in the real world with developing modelling methods in relation to specific objects and processes. Some of the problems discussed here include the stability and bifurcation of simplified models of supply and demand, and resultants of various approaches, such as standardisation, unification, partial ordering and data packaging.
As well as touching upon the general principles of modelling, the book studies the stability of mathematical models using the generalised direct Lyapunov method based on matrix-valued auxiliary functions. It has significant potential for application in the design of new moving objects (nano-robots, manipulators, drones, etc.) and processes related to artificial intelligence (multidimensional interaction of related data arrays) that can be described by differential equations of different types.
Written by two respected specialists in the field of the qualitative theory of equations and problems of modelling real-world phenomena, this book is primarily targeted at specialists in the field of engineering sciences, but is also suitable for graduate courses.
Model and Modelling, General Remarks
Sets of Numbers and Modelling
Orders, Rankings,...?
Average Rank Methods
Modelling Elements: First Patterns
Econometric and Economic Models
Mathematical Modelling of a Coronavirus
Facts about Population Growth and Mathematical Models
Modelling Errors
Principle of Comparison in Stability Theory
Stability Analysis of Nonlinear Systems via Matrix-Valued Functions
Stability of Singularly Perturbed Systems via Matrix-Valued Functions
Stability of Stochastic Systems via Matrix-Valued Functions Method
Stability in Models of Real-World Phenomena
The primary target audience for this book are specialists in engineering and the natural sciences who are directly involved in modelling real-world objects and phenomena. The book is also suitable for senior undergraduate and graduate courses specialising in mechanics and applied mathematics, systems analysis and cybernetics.
Pages: 250
ISBN: 978-981-98-0182-4 (hardcover)
Why are there more poor people with small bank accounts than rich people with big bank accounts? Why are there more small planets and stars than big ones in the cosmos? And why are there more small rivers than big rivers, and more harmless tremors than devastating earthquakes? Empirical examinations of real-life data overwhelmingly confirm the existence of such uneven size proportions in favor of the small, leaving us only the question: Why?
This law of nature carries through a vast list of topics and disciplines, confirming it to be nearly universal. In the biological world there are only approximately two million whales and over three billion birds; in number theory there are more small prime numbers than there are big ones; in census data there are more villages than towns, more towns than cities, and more cities than metropolises; in history there have been more minor conflicts than great wars.
In a landmark, first-ever study on the distribution of relative sizes, this book discusses several real-life case studies such as those above in extensive detail, and presents three distinct explanations for the phenomenon. Readers of all disciplines and levels of expertise will find an easily accessible yet original exploration that concludes with a numerical quantification demonstrating precisely by how much the relatively small is more numerous than the relatively big.
Empirical Evidence of the Small is Beautiful Phenomenon:
The Small is Beautiful Phenomenon
Small Life Forms Decisively Outnumber Big Creatures
Small Molecules Outnumber Big Molecules in the Chemical World
Small Atoms Outnumber Big Atoms in the Composition of Chocolate
Slow Spinning Pulsars Outnumber Fast Ones Throughout the Universe
Small Planets Outnumber Big Planets in the Milky Way Galaxy
Small Rivers Outnumber Big Rivers Worldwide
Small Stars Outnumber Big Stars in the Universe
Small Cities and Towns Outnumber Big Metropolises
Small Wars Outnumber Big Wars in History
The Poor Vastly Outnumbers the Rich in Global Wealth Distribution
Small Companies Outnumber Big Corporations
Small Bills Outnumber Big Bills in Revenue and Expense Data
Cheap Items Outnumber Expensive Items in Catalogs and Price Lists
Small House Numbers Outnumber Big House Numbers in Address Data
Global Carbon Dioxide Pollution Data in Favor of the Small
Small Families Outnumber Big Families Regarding Number of Children
Small Meteorites Outnumber Big Meteorites
Small Volcano Eruptions Outnumber Big Volcano Eruptions
Small Countries Outnumber Big Countries
Small Prime Numbers Slightly Outnumber Big Prime Numbers
Four-Dice Multiplication Game at the Casino in Favor of the Small
Three-Dice Selection Game at the Casino in Favor of the Small
Proper Criteria for Sizes Balancing the Chances of Big and Small
Exceptions and Counter Examples to the Phenomenon
Seven-Dice Addition Game at the Casino in Favor of the Medium
Beauty is in the Eye of the Beholder
Causes and Explanations of the Phenomenon:
Partitioning as a Cause of the Small is Beautiful Phenomenon
Random Dependent Partition is Always in Favor of the Small
Random Real Partition is Always in Favor of the Small
Numerous & Distinct Parts in Partitions are Necessary Conditions
Conclusion: Random Partitions and the Small is Beautiful Phenomenon
Multiplication Processes Lead to the Small is Beautiful Phenomenon
Exponential Growth Series and the Small is Beautiful Phenomenon
Explanation of the Small is Beautiful Phenomenon in Growth Series
The Fibonacci Series and the Small is Beautiful Phenomenon
Growth Model for Planets, Stars, Cities, and Bank Accounts
Data Aggregation Leads to the Small is Beautiful Phenomenon
Three-Dice Selection Game as Data Aggregation or Probability Scheme
Thirty-Dice Selection Game is Consistently in Favor of the Small
Chains of Statistical Distributions & the Small is Beautiful Phenomenon
The Prevalence of the Lognormal Distribution Favoring the Small
The Prevalence of the Exponential Distribution Favoring the Small
Discovery of Size Pattern: The General Law of Relative Quantities:
The Quest for Numerical Consistency in All Data Sets
Division of Data along Small, Medium, and Big Sizes
Constructing a Set of Infinitely Expanding Histograms
Numerical Consistency in Bin Schemes for 15 Real-Life Data Sets
The Quest for the General Mathematical Expression for All D&F Cases
The Postulate on Relative Quantities
Application of the Postulate Via Generic Bin Schemes on K/X
The General Law of Relative Quantities (GLORQ)
Saturation in GLORQ-Inducing Processes Precludes Extreme Skewness
GLORQ is Number System Invariant
Benford's Law as Direct Consequence of the General Law:
The Physical Clues Leading to the Discovery of Benford's Law
The First Digit on the Left Side of Numbers
Benford's Law and the Predominance of Low Digits
Empirical Tests and Confirmation of Compliance with Benford's Law
The Theoretical Clues Leading from Benford's Law to GLORQ
The Base Invariance Principle in Benford's Law
The Scale Invariance Principle in Benford's Law
Integral Powers of Ten (IPOT)
Benford's Law as a Special Case and Direct Consequence of GLORQ
Benford & GLORQ as a Subset of the Small is Beautiful Phenomenon
Physical Order of Magnitude of Data (POM)
A Robust Measure of Physical Order of Magnitude (CPOM)
Two Essential Requirements for Benford and GLORQ Behavior
Sum of Squares Deviation Measure (SSD)
A Critique on the Mixture of Distribution Model in Benford's Law
The Random and the Deterministic Flavors in Benford's Law
Tugs of War between Addition and Multiplication of Random Variables
Quantitative Partition Models and Benford's Law
Chains of Statistical Distributions are Nearly Always Benford
Development Pattern within the Expanding Histograms of Benford
Appendices:
Conceptual Explanation of Histograms
Conceptual Explanation of Powers and Logarithms
Glossary of Frequently Used Abbreviations
Bibliography
The book is suitable for professionals and experts in the fields of mathematics, statistics, and the sciences, as well as university students of these disciplines. It is also suitable for the interested layman, non-expert, and educated general public.