Skip to content

Conversation

@Kh4L
Copy link
Contributor

@Kh4L Kh4L commented Jan 13, 2025

This PR:

  • Introduces llm_model_name argument in g_retriever.py to allow specifying the LLM model
  • Make num_params optional in the LLM constructor, automatically determining it using huggingface_hub metadata if not provided
  • Change the default LLM model to the more recent meta-llama/Meta-Llama-3.1-8B-Instruct

Previous model meta-llama/Llama-2-7b-chat-hf metrics:

Hit: 0.6966
Precision: 0.6250
Recall: 0.5344
F1: 0.5405
Total Training Time: 556.111935s

Newer model meta-llama/Llama-3.1-8B-Instruct metrics:

Hit: 0.7629
Precision: 0.7145
Recall: 0.6027
F1: 0.6190
Total Training Time: 572.248117s

Copy link
Contributor

@puririshi98 puririshi98 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great improvements, thanks.

@puririshi98
Copy link
Contributor

please get the CI green so its mergeable

@Kh4L Kh4L force-pushed the improve_llm_num_param branch from f29bbc6 to 2b01c81 Compare January 14, 2025 07:55
@puririshi98 puririshi98 merged commit 4406bd7 into pyg-team:master Jan 14, 2025
16 checks passed
self,
model_name: str,
num_params: int,
num_params: int = None,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Kh4L The annotated type is no longer correct. Mind sending a quick PR?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants