gNE

This is work in progress, the description below will likely be subject to substantial changes.

This project introduces and explores geometric Neighbour Embeddings (gNE – /ˈdʒ:ɪni/, like Genie from Aladdin), a method for dimensionality reduction that aims to preserve geometric structures more faithfully than standard methods.

Similar to traditional Neighbour Embeddings like t-SNE and UMAP, gNE seeks to maintain the neighbour relationships present in high-dimensional data. However, gNE modifies these approaches in two ways: It extends beyond pairwise affinities and instead aims to preserve geometric properties like distances, areas, and volumes of higher-order neighbour relations. The resulting embeddings are intended to be more geometrically interpretable, such that they can facilitate downstream tasks on the low-dimensional representation, like dynamical inference.

An early implementation of these ideas is available as python package gNE on GitHub.

Let $X$ be a finite subset of a Riemannian manifold $(M, g_{M})$ and let $(N, g_{N})$ be a chosen target Riemannian manifold.

We then want to find an embedding

f : X ↪ (N, g_{N})

that tries to preserve the geometric properties of ( $k$ -nearest) neighbourhoods in $X$ .

The higher-order neighbour relations in $X$ and $Y = f (X)$ are modeled by \textit{weighted simplicial complexes} $(K_{X}, w_{X})$ and $(K_{Y}, w_{Y})$ . The weight functions $w_{X}$ and $w_{Y}$ associate geometric quantities, as determined by $g_{N}$ and $g_{M}$ , to each simplex. For example distances for 1-simplices (edges), areas for 2-simplices (triangles), volumes for 3-simplices (tetrahedra).

In complete analogy to standard Neighbour Embeddings, gNE then factors through a comparison of $(K_{X}, w_{X})$ and $(K_{Y}, {\tilde{w}}_{Y})$ . The general structure of the loss function is given by a simplex-wise comparison of weights:

L (K_{X}, K_{Y}) = λ_{0} \sum_{i} L_{0} (w_{i}, {\tilde{w}}_{i}) + λ_{1} \sum_{i, j} L_{1} (w_{i j}, {\tilde{w}}_{i j}) + λ_{2} \sum_{i, j, k} L_{2} (w_{i j k}, {\tilde{w}}_{i j k}) + \dots

Here $λ_{i} \in R$ are hyperparameters that fix the relative importance of the simplices of various dimensionalities, $L_{i}$ are loss functions (e.g. $L^{2}$ ), and we used abbreviations of the form $w_{i j} = w_{X} (x_{i}, x_{j})$ and ${\tilde{w}}_{i j} = w_{Y} (y_{i}, y_{j})$ .

Further Ideas

For future purposes want to implement Hamiltonian dynamics on a given geometry. For this need tensors and differential forms.
parametric gNE, i.e. use $(X, d_{X}) \subset (M, g_{M})$ to learn a map $f : M \to N$ such that one can calculate $f (x)$ for any $x \in M$ that was not originally in $X$ . As far as I understand it, this is usually done with neural nets and relies on a completely different Ansatz. Currently I do not plan to follow up on this idea.

Further Ideas

References

2022

2021

2018

2017

2008

2002