Skip to content

Conversation

@andyhuang-kumo
Copy link
Contributor

@andyhuang-kumo andyhuang-kumo commented Jul 4, 2024

Adding patch transformer architecture from dynamic graph learning literature as a PyG aggregation module.
Mostly based on "Simplifying Temporal Heterogeneous Network for Continuous-Time Link Prediction"

  • added patch transformer aggregation module in torch_geometric/nn.aggr.patch_transformer.py
  • added all related util functions in pt_utils.py
  • passed precommit hooks
  • added test_patch_transformer_aggr.py in `test/nn/aggr/ , test passed
  • add documentation for helper functions
  • integrate into workflow
  • test on dataset

Notes:

  • PatchTransformerAggregation requires edge features(edge_feats) and edge timestamps(edge_ts)
  • PatchTransformerAggregation primarily focus on aggregating edge information for each node based on their past interactions, the embedding of edges are fed into the patch transformer blocks which learns interaction between these edges and by pooling over multiple patches, the representation of each node is generated. Lastly, this representation is merged with the input node features (x) to produce an output.
  • PatchTransformerAggregation do not run self.to_dense_batch() at the moment thus do not reshape the input x tensor.
  • the output dimension is set via the out_dim argument

@andyhuang-kumo andyhuang-kumo reopened this Jul 5, 2024
@rusty1s rusty1s changed the title adding Patch Transformer Adding PatchTransformerAggregation Jul 7, 2024
Copy link
Member

@zechengz zechengz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general LGTM. Maybe @rusty1s can help to make another review when available.

@rusty1s rusty1s marked this pull request as ready for review July 19, 2024 17:11
@rusty1s rusty1s enabled auto-merge (squash) July 19, 2024 17:11
@rusty1s rusty1s merged commit 9ac9aa3 into pyg-team:master Jul 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants