Skip to content

Add ComplexWebQuestions (CWQ) dataset #9950

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
puririshi98 merged 3 commits into from
Jan 23, 2025
Merged

Add ComplexWebQuestions (CWQ) dataset #9950

puririshi98 merged 3 commits into from
Jan 23, 2025

Conversation

@Kh4L
Copy link
Contributor

@Kh4L Kh4L commented Jan 15, 2025

This PR adds the CWQ dataset, which is similar to WebQSP but larger and featuring more complex multi-hop questions

Comparing the datasets:

Datasets #Train #Test Max #hop
WebQSP 2,826 1,628 2
CWQ 27,639 3,531 4

GRetriever performance:

LLM: meta-llama/Llama-3.1-8B-Instruct

Hit@1: 0.5675
Precision: 0.5444
Recall: 0.5435
F1: 0.5310
Total Training Time: 4337.614916s

@Kh4L Kh4L requested a review from wsad1 as a code owner January 15, 2025 12:36
@puririshi98 puririshi98 self-requested a review January 23, 2025 03:55
puririshi98
puririshi98 approved these changes Jan 23, 2025
View reviewed changes
Copy link
Contributor

@puririshi98 puririshi98 left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, good expansion of g-retriever.

@puririshi98 puririshi98 merged commit 9b794b6 into pyg-team:master Jan 23, 2025
15 of 16 checks passed
thomasbauer76 added a commit to thomasbauer76/pytorch_geometric that referenced this pull request Jan 23, 2025
@thomasbauer76
Merge pull request #2 from pyg-team/master
309f898 
Add ComplexWebQuestions (CWQ) dataset (pyg-team#9950)
thomasbauer76 added a commit to thomasbauer76/pytorch_geometric that referenced this pull request Jan 23, 2025
@thomasbauer76
Merge pull request #3 from pyg-team/master
740071f 
Add ComplexWebQuestions (CWQ) dataset (pyg-team#9950)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@puririshi98 puririshi98 puririshi98 approved these changes

@wsad1 wsad1 Awaiting requested review from wsad1 wsad1 is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@Kh4L @puririshi98