Statistical Invariance of Betti Numbers

# .orange[Statistical Invariance of Betti Numbers .tiny[In the thermodynamic regime]]

### Siddharth Vishwanath .small[.grey[With Kenji Fukumizu, Satoshi Kuriki & Bharath Sriperumbudur]]

---
class: inverse, center, middle
count: false

# Statistics + Topology

---

# Sufficient Statistics may not suffice

<img src="images/anscombe.png" width="650">
.center[Anscombe's quartet]
</div>

---
# Sufficient Statistics may not suffice

<div class="centered">
<img align="centered" class="animated-gif" src="images/DinoSequential.gif" width="120%">
.center[Anscombe's quartet on steroids]
</div>

---
layout: false
class: inverse, center, middle
count: false

# Preliminaries
## .orange[Topological Ingredients]

---
layout: true
class: split-five with-border border-black

.column[.content[
.split-five[
.row.bg-main1[.content.center.vmiddle[
# Space
]]
.row.bg-main2[.content.center[
## Circle
]]
.row.bg-main3[.content.center[
## Sphere
]]
.row.bg-main4[.content.center[
## Torus
]]
.row.bg-main5[.content.center[
## `$3d_{z^2}$`
]]
]]]
.column[.content[
.split-five[
.row.bg-main1[.content.center.vmiddle[
# Shape
]]
.row.bg-main2[.content.center[
<img src="images/circle.svg" width="100",height="100">
]]
.row.bg-main3[.content.center[
<img src="images/sphere.svg" width="100",height="100">
]]
.row.bg-main4[.content.center[
<img src="images/torus.svg" width="100",height="100">
]]
.row.bg-main5[.content.center[
<img src="images/orbital.png" width="100",height="100">
]]
]]]
.column[.content[
.split-five[
.row.bg-main1[.content.center.vmiddle[
# `$\beta_0$` 
]]
.row.bg-main2[.content.center[
## 1
]]
.row.bg-main3[.content.center[
## 1
]]
.row.bg-main4[.content.center[
## 1
]]
.row.bg-main5[.content.center[
## 1
]]
]]]
.column[.content[
.split-five[
.row.bg-main1[.content.center.vmiddle[
# `$\beta_1$`
]]
.row.bg-main2[.content.center[
## 1
]]
.row.bg-main3[.content.center[
## 0
]]
.row.bg-main4[.content.center[
## 2
]]
.row.bg-main5[.content.center[
## 1
]]
]]]
.column[.content[
.split-five[
.row.bg-main1[.content.center.vmiddle[
# `$\beta_2$` 
]]
.row.bg-main2[.content.center[
## 0
]]
.row.bg-main3[.content.center[
## 1
]]
.row.bg-main4[.content.center[
## 1
]]
.row.bg-main5[.content.center[
## 3
]]
]]]

---
class: fade-row2 fade-row3 fade-row4 fade-row5 gray-row2 gray-row3 gray-row4 gray-row5

---
count: false
class: fade-row3 fade-row4 fade-row5 gray-row3 gray-row4 gray-row5

---
count: false
class: fade-row2 fade-row4 fade-row5 gray-row2 gray-row4 gray-row5

---
count: false
class: fade-row2 fade-row3 fade-row5 gray-row2 gray-row3 gray-row5

---
count: false
class: fade-row2 fade-row3 fade-row4 gray-row2 gray-row3 gray-row4

---
layout: false
class: inverse, center, middle
count: false

# Preliminaries
## .orange[Probabilistic Ingredients]

---
# Random Topology

* Given a probability space `$(\Omega,\mathcal{F},\mathbb{P})$` and some metric-space `$\mathcal{X}$`

* `$\mathbb{X}_n = \{ \boldsymbol{X}_1, \boldsymbol{X}_2, \dots \boldsymbol{X}_n \} \sim \mathbb{P}$`

- A fixed probability measure, i.e., observed i.i.d.
  - A random field, e.g., Poisson Process

--
* A simplicial complex, `$\mathcal{K}(\mathbb{X}_n,r)$`, is a random-variable measurable w.r.t. `$\mathbb{P}^{\otimes n}$`

--
* The Betti Numbers, `$\beta_k \left( \mathcal{K}(\mathbb{X}_n,r) \right) : \mathcal{X}^n \rightarrow \mathbb{N}$`, are topological summaries

--

* What are the properties of these **.purple[random]** topological summaries?
`\begin{align}
\text{(LLN)} & & \lim\limits_{n\rightarrow \infty}\frac{1}{n}\beta_k\left( \mathcal{K}(\mathbb{X}_n,r) \right) = \color{red}{\gamma_k(\mathbb{P})} \ \ \text{a.s.} \hspace{2cm}\\ \\
\text{(CLT)} & & \lim\limits_{n\rightarrow \infty}\frac{\beta_k\left( \mathcal{K}(\mathbb{X}_n,r) \right) - \mathbb{E}(\beta_k\left( \mathcal{K}(\mathbb{X}_n,r) \right))}{\sqrt{n}}  \sim \mathcal{N}(0,\sigma^2)
\end{align}`

.tiny[.caption[Bobrowski and Kahle (2018); Kahle and Meckes (2013); Yogeshwaran, Subag, and Adler (2017)]]

---
layout: true
class: center,split-three

# .left[Asymptotic Regimes]

In the simplicial complex `$\mathcal{K}\left( \mathbb{X}_n, {r_n} \right)$`, `$r_n$` depends on `$n$`

.column.bg-main1[.content[
 
.center[
Dense
<img src="images/dense.gif" width="300",height="300">]
`$nr_n^d \rightarrow \infty$`
]]

.column.bg-main1[.content[
 
.center[
Sparse
<img src="images/sparse.gif" width="300",height="300">]
`$nr_n^d \rightarrow 0$`
]]

.column.bg-main1[.content[
 
.center[
Thermodynamic
<img src="images/thermodynamic.gif" width="300",height="300">]
`$nr_n^d \rightarrow t \in (0,\infty)$`
]]

---

---

---

---
class: show-111
count: false

---
layout: false
count: false
class: inverse, center, middle

# Statistical Invariance
## .orange[Characterization]

---
layout: false

# Statistical Invariance of Betti Numbers

* Consider a family of distributions* `$\mathcal{P} = \{\mathbb{P}_\theta : \theta \in \Theta \}$`

* Given `$\mathbb{X}^\theta_n = \{\mathbf{X}^\theta_1,\mathbf{X}^\theta_2,\dots,\mathbf{X}^\theta_n\} \sim \mathbb{P}_\theta$`, for `$\theta \in \Theta$`

.content-box-purple[
`$\mathbf{S}(\mathbb{P}_{\theta}^{\otimes n})\!:= \mathbf{S}(\mathbb{X}^\theta_n)\!= \frac{1}{n} \Big( \beta_0\big( \mathcal{K}(\mathbb{X}^\theta_n,r_n) \big), \beta_1\big( \mathcal{K}(\mathbb{X}^\theta_n,r_n) \big), \dots , \beta_d\big( \mathcal{K}(\mathbb{X}^\theta_n,r_n) \big) \Big)$`
]

* **.purple[Invariance.]**

.center[<body> `$\bbox[20px, border: 2px solid orange]{ \text{For } \theta_1,\theta_2 \in \Theta, \text{ can we have that } \lim\limits_{n\rightarrow \infty}\mathbf{S}(\mathbb{P}_{\theta_1}^{\otimes n}) {=} \lim\limits_{n\rightarrow \infty}\mathbf{S}(\mathbb{P}_{\theta_2}^{\otimes n}) \text{ ? } }$`</body>]

--
* **Example (1).**

Consider `$\color{red}{\mathcal{P} = \{ \mathcal{N}(\theta,\mathbf{I}_d) : \theta \in \mathbb{R}^d \}}$` and `$\color{green}{\mathbf{S}(\mathbb{X}_n) = \bar{\mathbf{X}}_n}$`
  
      `$\hspace{3cm} \lim\limits_{n\rightarrow \infty}\bar{\mathbf{X}}^{\theta_1}_n = \theta_1 \neq \theta_2 = \lim\limits_{n\rightarrow \infty}\bar{\mathbf{X}}^{\theta_2}_n$`

---
layout: false
count: false

# Statistical Invariance of Betti Numbers

* Consider a family of distributions* `$\mathcal{P} = \{\mathbb{P}_\theta : \theta \in \Theta \}$`

* Given `$\mathbb{X}^\theta_n = \{\mathbf{X}^\theta_1,\mathbf{X}^\theta_2,\dots,\mathbf{X}^\theta_n\} \sim \mathbb{P}_\theta$`, for `$\theta \in \Theta$`

* **.purple[Invariance.]**

* **Example (2).**

Consider `$\color{red}{\mathcal{P} = \{ \mathcal{N}(\mathbf{0},\boldsymbol{\theta}) : \boldsymbol{\theta} \in \mathcal{S^d_{++}} \}}$` and `$\color{green}{\mathbf{S}(\mathbb{X}_n) = \bar{\mathbf{X}}_n}$`
  
      `$\hspace{3.5cm} \lim\limits_{n\rightarrow \infty}\bar{\mathbf{X}}^{\theta_1}_n = 0 = \lim\limits_{n\rightarrow \infty}\bar{\mathbf{X}}^{\theta_2}_n$`

---
layout: false
count: false

# Statistical Invariance of Betti Numbers

* Consider a family of distributions* `$\mathcal{P} = \{\mathbb{P}_\theta : \theta \in \Theta \}$`

* Given `$\mathbb{X}^\theta_n = \{\mathbf{X}^\theta_1,\mathbf{X}^\theta_2,\dots,\mathbf{X}^\theta_n\} \sim \mathbb{P}_\theta$`, for `$\theta \in \Theta$`

* **.purple[Invariance.]**

* **Example (3).**

Consider `$\color{red}{\mathcal{P} = \{ \mathcal{N}(\mathbf{0},\boldsymbol{\theta}) : \boldsymbol{\theta} \in \mathcal{S^d_{++}} \}}$` and `$\color{green}{\mathbf{S}(\mathbb{X}_n) = \text{Cov}(\mathbb{X}_n)}$`
  
      `$\hspace{2cm} \lim\limits_{n\rightarrow \infty}\text{Cov}(\mathbb{X}^{\theta_1}_n) = \boldsymbol{\theta}_1 \neq \boldsymbol{\theta}_2 = \lim\limits_{n\rightarrow \infty}\text{Cov}(\mathbb{X}^{\theta_2}_n)$`

---
layout: false
count: false

# Statistical Invariance of Betti Numbers

* Consider a family of distributions* `$\mathcal{P} = \{\mathbb{P}_\theta : \theta \in \Theta \}$`

* Given `$\mathbb{X}^\theta_n = \{\mathbf{X}^\theta_1,\mathbf{X}^\theta_2,\dots,\mathbf{X}^\theta_n\} \sim \mathbb{P}_\theta$`, for `$\theta \in \Theta$`

* **.purple[Invariance.]**

* **Assumptions:** 
1. .small[<body> `$\text{dim}(\mathcal{X}) = d \le D$`. If `$d < D$` then `$\mathcal{X}$` is a compact, `$d$`-dimensional `$\mathcal{C}^1$` manifold </body> ]

1. .small[<body> `$\mathbb{P}_\theta$` admits a density `$f_\theta$` such that `$f_\theta \in L_p(\mathcal{X})$` for all `$p \in \mathbb{N}$` </body> ]

1. .small[<body> `$\mathcal{K}(\mathbb{X}^\theta_n,r_n)$` is the Čech complex constructed on `$\mathbb{X}^\theta_n$` with `$r_n = r(n)$` </body> ]

---
layout: false
count: false

# Statistical Invariance of Betti Numbers

* Consider a family of distributions* `$\mathcal{P} = \{\mathbb{P}_\theta : \theta \in \Theta \}$`

* Given `$\mathbb{X}^\theta_n = \{\mathbf{X}^\theta_1,\mathbf{X}^\theta_2,\dots,\mathbf{X}^\theta_n\} \sim \mathbb{P}_\theta$`, for `$\theta \in \Theta$`

* **.purple[Invariance.]**

* **.purple[Definition.]**

As `$n \rightarrow \infty$` and `$nr_n^d \rightarrow t$`, the **.purple[thermodynamic limit]** is the functional 
`\begin{align}
\mathbf{S}(\mathbb{P}_\theta; t) = \lim_{n \rightarrow\infty}\mathbf{S}(\mathbb{P}_{\theta}^{\otimes n})
\end{align}`

---
layout: false

# `$\beta$`-equivalence

.content-box-purple[
**.purple[Definition.]**  `$\mathcal{P}$` admits `$\beta$`-equivalence if `$\mathbf S(\mathbb{P}_\theta; t) = \eta(t)$` for all `$\theta \in \Theta$`
]

* Let `$\color{purple}{\mathcal{F}}$` be the family of probability density functions associated with `$\color{purple}{\mathcal{P}}$`

.content-box-purple[**.purple[Proposition.]** Suppose `$\mathcal{F}$` is a family of distributions such that: 
for all `$f,g \in \mathcal{F}$` with `$\mathbf{X} \sim f$` and `$\mathbf{Y} \sim g$`, it holds that `$f(\mathbf{X}) \stackrel{d}{=} g(\mathbf{Y})$`. 
Then, `$\mathcal{F}$` admits `$\beta$`-equivalence.]

--
* **.purple[Remark.]** Invariance extends to persistent Betti numbers and Euler characteristic

---
layout: false
count: false
class: inverse, center, middle

# Statistical Invariance
## .orange[via Topological Groups]

---
layout: true
class: center,split-two

# .left[Statistical Invariance - I]

.column.bg-main1.top[.content[
 
.center[<img src="images/group.svg" width="400",height="400">]]
]

.column.top[.content[
 
.left[.small[
* `$\Psi:\mathcal{X} \rightarrow \mathcal{Y}$` is differentiable 
* `$\mathcal{G} = \{ g_\theta : \theta \in \Theta \}$`: group acting bijectively on `$\mathcal{Y}$` 
* Consider `$T: \mathcal{Y} \rightarrow \mathcal{T}$`

- `$T$` is `$\color{purple}{\mathcal{G}}$`.purple[-invariant] if it is constant on orbits, i.e., `$T(g_\theta\cdot\mathbf{x}) = T(\mathbf{x})$`
 
 - `$T$` is `$\color{red}{\mathcal{G}}$`.red[-maximal invariant] if it is constant **only** on orbits, i.e.,.center[ <body> `$T(\mathbf{x}) = T(\mathbf{y})$` if and only if `$\mathbf{y} \in \mathcal{G}\mathbf{x}$` </body>]
* Define `$f_\theta(x) := \phi\big( g_\theta \circ \Psi(x) \big)$`

- `$\phi$` just ensures `$f_\theta$` is a valid density
]]]]

---

---

.content-box-purple[.small[
**.purple[Theorem.]** `$\hspace{2cm}$` `$\mathcal{F}$` admits `$\beta$`-equivalence if and only if  `$\exists \hspace{0.1cm}\zeta:\mathcal{T} \rightarrow \mathbb{R}$` `$\hspace{2cm}$`
`\begin{align}
\text{det}(J_{\Psi^{-1}}(y)) = \zeta(T(y))
\end{align}`
]]

---
layout: false
count: true
class: left

# Statistical Invariance - I : .red[Example]

.content-box-red[
$$
f_\theta(x_1,x_2) = \big( \cos(\theta) \Phi^{-1}(x_1) + \sin(\theta )\Phi^{-1}(x_2) \big)^2 \hspace{0.5cm} \mathbf{1}(0 \le x_1,x_2 \le 1)
$$
]

---
layout: false
count: false
class: inverse, center, middle

# Statistical Invariance
## .orange[via Excess Mass]

---
layout: true
class: center,split-two

# .left[Statistical Invariance - II]

* `$\mathscr{X}=(\mathcal X, \pi, \mathcal Y, \mathcal Z)$` is a smooth fiber bundle with local trivialization `$\{ U_\alpha,\psi_\alpha \}$`

.column.bg-main1[.content[
 
.center[<img src="images/bundle.svg" width="400",height="400">]]
]

.column[.content[
 
.left[.small[
* `$\mu:$` A probability measure on `$\mathcal Z$` 
* `$\nu:$` Measure on `$\mathcal Y$` with modular character `$\Psi$` 
* `$\mathbb P:$` Distribution on `$\mathcal Y$` with density `$g$` w.r.t `$\nu$` 
* `$\phi:\mathcal Y \times \mathcal Z \rightarrow \mathcal Y$` is a map such that
 * `$\phi(\cdot,z) \in \text{Diff}(\mathcal Y)$` for each `$z \in \mathcal Z$` 
* For each `$x \in \mathcal X$` with local coordinates `$x=(z,y_z)$`, `$y=\psi^{-1}_{\alpha,z}(y_z)$` 
* Define `$f_\phi(x) = g(\phi(z,y))$`
]]]]

---

---

.content-box-purple[.small[
**.purple[Theorem.]** `$\hspace{2cm}$` `$\{ f_\phi : \phi \in \Phi \}$` admits `$\beta$`-equivalence for `$\hspace{5cm}$`
`$\Phi := \Big\{ \phi : \displaystyle\int_{\mathcal Z}{\Psi\left( \big\lvert \text{det}J_{\phi_z^{-1}} \big\rvert  \right) \mu(dz)} = 1 \Big\}$`
]]

---
layout: false

# Statistical Invariance - II : .red[Example]
.small[ <body> Given a density `$g$` on `$\mathbb{R}_+$` and `$\Theta = \{ (a,b) : \frac{1}{a} + \frac{1}{b} = 1 \}$`, define `$\{ f_\theta\!: \theta\!\in\!\Theta \}$` on `$\mathcal{X} = \mathbb{R}$` </body>]
.small[.center[.content-box-red[
$$
f_\theta(x) = \color{red}{g(-bx) \mathbf{1}(x < 0)} + \color{blue}{g(ax) \mathbf{1}(x \ge 0)}
$$
]]]
.center[<img src="images/Gamma.gif" width="450",height="450">]

.small[
 
`$\phi_1(x) = ax \text{ and } \phi_2(x) = -bx,$` then `$\Psi(|\text{det}J_{\phi^{-1}_1}|)+\Psi(|\text{det}J_{\phi^{-1}_2}|) = \frac 1 a + \frac 1 b = 1.$`
]

---
layout: false

# Verifying `$\beta$`-equivalence

* Given a family of distributions `$\mathcal F$`, can we check if `$\mathcal F$` admits `$\beta$`-equivalence?

--
.content-box-purple[
**.purple[Theorem.]**
Given `$\mathcal F = \{ f_\theta : \theta \in \Theta \}$` where `$\Theta$` contains an open set in `$\mathbb R^p$`.

`$\mathcal F$` admits `$\beta$`-equivalence if and only if for all `$k \in \mathbb N$`
`\begin{align}
\nabla_{\theta} \left(\int_{\mathcal X}{f_\theta^{k+1}(\mathbf{x}) d\mathbf{x}}\right) = \mathbf 0
\end{align}`
]

--
* Under standard stochastic regularity assumptions, it leads to a geometric constraint on `$f_\theta$` 
.content-box-purple[
`\begin{align}
\big\langle S_{\theta},f_\theta^k \big\rangle_{L_2(\mathcal X)} = \mathbf 0,
\end{align}`
where `$S_{\theta} = \nabla_{\theta}\log f_\theta$` is the **score function** used in statistical inference
]

---
layout: false
class: inverse, center, middle

# Thank you!