PyG 2.6.0
We are excited to announce the release of PyG 2.6 πππ
PyG 2.6 is the culmination of work from 59 contributors who have worked on features and bug-fixes for a total of over 238 commits since torch-geometric==2.5.0.
Highlights
PyTorch 2.4 Support
PyG 2.6 is fully compatible with PyTorch 2.4, and supports the following combinations:
| PyTorch 2.2 | cpu |
cu118 |
cu121 |
cu124 |
|---|---|---|---|---|
| Linux | β | β | β | β |
| macOS | β | |||
| Windows | β | β | β | β |
You can still install PyG 2.6 with an older PyTorch release up to PyTorch 1.13 in case you are not eager to update your PyTorch version.
GNNs+LLMs
In order to facilitate further research on combining GNNs with LLMs, PyG 2.6 introduces
- a new sub-package
torch_geometric.nn.nlpwith fast access toSentenceTransformermodels and LLMs - a new model
GRetrieverthat is able to co-trainLLAMA2withGATfor answering questions based on knowledge graph information - a new example folder
examples/llmthat shows how to utilize these models in practice
Index Tensor Representation
Similar to the EdgeIndex class introduced in PyG 2.5, torch-geometric==2.6.0 introduces the Index class for efficient storage of 1D indices. While Index sub-classes a general torch.Tensor, it can hold additional (meta)data, i.e.:
dim_size: The size of the underlying sparse vector, i.e. the size of a dimension that can be indexed viaIndex. By default, it is inferred asdim_size=index.max() + 1is_sorted: Whether indices are sorted in ascending order.
Additionally, Index caches data via indptr for fast CSR conversion in case its representation is sorted. Caches are filled based on demand (e.g., when calling Index.get_indptr() or when explicitly requested via Index.fill_cache_(), and are maintained and adjusted over its lifespan.
from torch_geometric import Index
index = Index([0, 1, 1, 2], dim_size=3, is_sorted=True)
assert index.dim_size == 3
assert index.is_sorted
# Flipping order:
index.flip(0)
assert not index.is_sorted
# Filtering:
mask = torch.tensor([True, True, True, False])
index[:, mask]
assert index.is_sortedEdgeIndex and Index will interact seamlessly together, e.g., edge_index[0] will now return a Index instance.
This ensures optimal computation in GNN message passing schemes, while preserving the ease-of-use of regular COO-based PyG workflows. EdgeIndex and Index will fully deprecate the usage of SparseTensor from torch-sparse in later releases, leaving us with just a single source of truth for representing graph structure information in PyG.
Breaking Changes
- Allow
Noneoutputs inFeatureStore.get_tensor()-KeyErrorshould now be raised based on the implementation inFeatureStore._get_tensor()(#9102) cugraph-based GNN layers such asCuGraphSAGEConvnow expectEdgeIndex-based inputs (#8938)
Features
Examples
- Added a multi-GPU example for training GNNs on the PCQM4M graph-level regression task (#9070)
- Added a multi-GPU
ogbn-mag240mexample (#8249) - Added support for
cugraphdata loading capabilities in thepapers100mexamples (#8173) - Improved the hyper-parameters of the [single-node](ogbn-papers100m example
](https://github.com/pyg-team/pytorch_geometric/blob/master/examples/ogbn_papers_100m.py) and [multi-node](https://github.com/pyg-team/pytorch_geometric/blob/master/examples/multi_gpu/papers100m_gcn_cugraph_multinode.py)ogbn-papers100m examples, and added evaluation on all ranks (#8823, #9386, #9445)
EdgeIndex and Index
- Added
torch_geometric.Index(#9276, #9277, #9278, #9279, #9280, #9281, #9284, #9285, #9286, #9287, #9288, #9289, #9296, #9297) - Added support for
EdgeIndexinMessagePassing(#9007, #9026, #9131) - Added support for
torch.compilein combination withEdgeIndex(#9007) - Added support for
EdgeIndex.unbind()(#9298) - Added support for
EdgeIndex.sparse_narrow()(#9291) - Added support for
EdgeIndex.sparse_resize_()(#8983)
torch_geometric.nn
- Added the
GRetrievermodel (#9480) - Added the
ClusterPoolinglayer (#9627) - Added the
PatchTransformerAggregationlayer (#9487) - Added a
residualoption inGATConvandGATv2Conv(#9515) - Added a
nlp.LLMmodel wrapper (#9462) - Added a
nlp.SentenceTransformermodel wrapper (#9350) - Added the heterogeneous
HeteroJumpingKnowledgemodule for applying jumping knowledge in heterogeneous graphs (#9380) - Added the
VariancePreservingAggregationlayer (#9075) - Added approximate
faiss-based KNN-search capabilities viaApproxKNN(#8952, #9046)
torch_geometric.metrics
- Added the
LinkPredMRRmetric (#9632)
torch_geometric.transforms
- Added the
RemoveSelfLoopstransformation (#9562)
torch_geometric.utils
- Added
normalize_edge_index()for symmetric/asymmetric normalization of graph edges (#9554) - Added
from_rdmol/to_rdmolfunctionality (#9452) - Added ONNX export for
scatterwith min/max reductions (#9587)
torch_geometric.datasets
- Added the
WebQSPDataset(#9481) - Added the
OPFDataset(#9379) - Added
CornellTemporalHyperGraphDatasethypergraph dataset (#9090) - Added option to pass custom
from_smilesfunctionality toPCQM4Mv2andMoleculeNet(#9073)
torch_geometric.loader
- Added support for negative sampling in
LinkLoaderacccording to source and destination node weights (#9316)
Bugfixes
- Fixed
VirtualNodetransform for empty edge indices (#9605) - Fixed an issue where the import order in the multi-GPU
cugraphexample could cause anrmmerror (#9577) - Fixed
load_state_dictbehavior with lazy parameters inHeteroDictLinear(#9493) Sequentialmodules can now be properly pickled (#9369)- Fixed
pickle.loadfor jittableMessagePassingmodules (#9368) - Fixed batching of sparse tensors saved via
data.edge_index(#9317) - Fixed arbitrary keyword ordering in
MessagePassing.propagate()(#9245) - Fixed the node mapping in the
RCDDdataset (#9234) - Fixed incorrect treatment of
edge_labelandedge_label_indexinToSparseTensortransform (#9199) - Fixed
EgoDataprocessing inSnapDatasetin case filenames are unsorted (#9195) - Fixed empty graph and isolated node handling in
to_dgl()function (#9188) - Fixed bug in
to_scipy_sparse_matrix()when CUDA is set as default torch device (#9146) - Fixed the
MetaPath2Vecmodel in case the last node is isolated (#9145) - Ensured backward compatibility in
MessagePassingviatorch.load()(#9105) - Prevented model compilation on custom
MessagePassing.propagate()functions (#9079) - Ignore
self.propagateappearances in comments when parsingMessagePassingimplementation (#9044) - Fixed
OSErroron read-only file systems withinMessagePassing(#9032) - Fixed metaclass conflict in
Dataset(#8999) - Fixed import errors on
MessagePassingmodules with nested inheritance (#8973) - Fixed TorchScript compilation error for
MessagePassing._check_input()on older torch versions (#9564)
Changes
- Use
torch.load(weights_only=True)by default (#9618) - Allow optional but untyped tensors in
MessagePassing(#9494) - Added support for modifying
filenameof the stored partitioned file inClusterLoader(#9448) - Support other than two-dimensional inputs in
AttentionalAggregation(#9433) - Added the
fmtargument toDataset.print_summary()(#9408) - Skip zero atom molecules in
MoleculeNet(#9318) - Ensure proper parallelism in
OnDiskDatasetfor multi-threadedgetcalls (#9140) - Allow mini-batching of uncoalesced sparse matrices (#9099)
- Default to
scatter()operations inMessagePassingin casetorch.use_deterministic_algorithmsis not set (#9009) - Added XPU support to basic GNN examples (#9421, #9439)
New Contributors
- @arthurdjn made their first contribution in #8918
- @project-delphi made their first contribution in #8946
- @arlesniak made their first contribution in #8978
- @simon-forb made their first contribution in #9057
- @rf523281 made their first contribution in #9075
- @BenediktAlkin made their first contribution in #9076
- @luckynozomi made their first contribution in #9113
- @MatthieuMelennec made their first contribution in #9116
- @brenting made their first contribution in #9146
- @drivanov made their first contribution in #9178
- @Akkete made their first contribution in #9204
- @1taroh made their first contribution in #9230
- @nelsonaloysio made their first contribution in #9334
- @chaojun-zhang made their first contribution in #9396
- @zhouyu5 made their first contribution in #9407
- @mzgubic made their first contribution in #9379
- @guanxingithub made their first contribution in #9386
- @devanshamin made their first contribution in #9445
- @ihkao made their first contribution in #9436
- @oiao made their first contribution in #9452
- @kano5266 made their first contribution in #9473
- @andyhuang-kumo made their first contribution in #9487
- @bryceForrest made their first contribution in #9533
- @ECMGit made their first contribution in #9543
- @nilserranestle made their first contribution in #9499
- @hnsgrvr made their first contribution in #9498
- @Linnore made their first contribution in #9515
- @alexbarghi-nv made their first contribution in #9541
- @mfairley made their first contribution in #9564
- @viktor-ktorvi made their first contribution in #9569
- @lee-clement-oxb made their first contribution in #9587
- @Danial-sb made their first contribution in #9554
- @DonBoscoBlaiseA made their first contribution in #9611
- @kaarthiksundar made their first contribution in #9621
- @rhjohnstone made their first contribution in #9632
Full Changelog: 2.5.0...2.6.0