Graph Context-Aware Positional Encoding for Graph Representation Learning

Abstract

Transformer architectures have shown great potential in graph learning, where positional encoding (PE) play a crucial role in capturing each node’s structural information. However, existing PE methods face two major challenges: First, they typically rely on single-pattern structural features—such as shortest-path distances or Laplacian eigenvectors—facing a fundamental trade-off between locality and stability that limits their comprehensive understanding of graph topology. Second, structural and contextual information are typically derived independently and merely summed to compute attention weights, limiting the discriminative power for structurally similar but contextually different node pairs. To address these challenges, we propose Graph Context-aware Positional Encoding (GCPE), a framework that improves the discriminative power of Graph Transformers through multi-pattern, context-aware PE. GCPE introduces two key innovations: (1) a multi-head structural encoding mechanism that unifies diverse PE types (spatial, spectral, and subgraph-based) by leveraging their respective structural patterns in PE computation, and (2) a gating-based context-aware module that dynamically adapts the relative PE between node pairs based on their context, integrating node semantic information to distinguish structurally similar patterns. Extensive experiments demonstrate that GCPE achieves consistent performance improvements across diverse graph-level prediction benchmarks. On molecular property prediction tasks (PCQM4Mv2, MOLHIV, MOLPCBA, ALCHEMY-12K, DrugOOD), GCPE achieves state-of-the-art performance using only 2D molecular information and exhibits superior out-of-distribution generalization capability. On general graph structure learning benchmarks (Peptides-func, Peptides-struct, MNIST, CIFAR10), GCPE achieves state-of-the-art or competitive results, validating its effectiveness and generalizability across different graph characteristics and node feature types.

Keywords

Graph Transformer / Positional Encoding / Graph Representation Learning / Graph Property Prediction

RIGHTS & PERMISSIONS

Higher Education Press 2026

Just Accepted

This article has successfully passed peer review and final editorial review, and will soon enter typesetting, proofreading and other publishing processes. The currently displayed version is the accepted final manuscript. The officially published version will be updated with format, DOI and citation information upon launch. We recommend that you pay attention to subsequent journal notifications and preferentially cite the officially published version. Thank you for your support and cooperation.