TopoFormer
Topology Meets Attention for Graph Learning
MD Joshem Uddin★, Astrit Tola★, Cuneyt Gurcan Akcora, Baris Coskunuzer.
International Conference on Learning Representations (ICLR), 2026.
★ Equal contribution.
Overview
TopoFormer is a lightweight and scalable framework for graph representation learning that encodes topological structure into attention-friendly sequences. At its core is Topo-Scan, a novel module that decomposes a graph into a short, ordered sequence of topological tokens by slicing over node or edge filtrations. These sequences capture multi-scale structural patterns — from local motifs to global organization — and are processed by a Transformer to produce expressive graph-level embeddings.
Unlike traditional persistent homology pipelines, Topo-Scan is parallelizable, avoids costly diagram computations, and integrates seamlessly with standard deep learning architectures. The result is a class of models that:
- Bypass persistence diagrams — directly extract topological sequences from sliced subgraphs.
- Avoid early saturation — a sliding-window slicing strategy preserves late-emerging structural features that standard sublevel filtrations miss.
- Plug into transformers natively — short, 1D, fixed-width topological token sequences fit attention's native regime, no graph-specific machinery required.
- Come with theoretical guarantees — we prove stability of the topological encodings under graph perturbations.
- Achieve state-of-the-art performance — matching or exceeding strong GNN and topology-based baselines on graph classification and molecular property prediction benchmarks.
We view TopoFormer as a step toward topology-aware graph foundation models — bridging topological data analysis and modern attention-based architectures in a unified, scalable framework.
Method Highlights
🔬 Topo-Scan
Decomposes a graph into a sequence of overlapping subgraph slices along a filtration function, then computes compact topological tokens per slice.
🧠 Topology + Attention
Ordered topological token sequences are processed by a Transformer encoder, letting self-attention capture multi-scale structural dependencies natively.
📐 Theoretical Stability
We prove that TopoFormer's encodings are stable under graph perturbations — a desirable property inherited from persistent topology.
⚡ Scalable & Parallel
Slice extraction is embarrassingly parallel and avoids global persistence computation, making the pipeline practical for large-scale graph learning.
Links & Resources
Citation
If you use TopoFormer in your research, please cite:
@inproceedings{uddin2026topoformer,
title = {TopoFormer: Topology Meets Attention for Graph Learning},
author = {MD Joshem Uddin and Astrit Tola and Cuneyt Gurcan Akcora and Baris Coskunuzer},
booktitle = {International Conference on Learning Representations (ICLR)},
year = {2026},
url = {https://openreview.net/pdf?id=xanpQt9YZh}
}