Skip to content

Detailed error message for noise sensitivity metric #1986

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

KalyanKS-NLP
Copy link
Contributor

Absence of any of the required inputs (user_input, response, reference or retrieved_contexts) in the test sample results in KeyError. For example, the absence of user_input in the test sample results in the error message "KeyError: user_input".

The error message "KeyError: user_input" is too abstract. I included the following lines of code in the _ascore() function definition which will display a clear error message in the absence of any of the required inputs.

if "reference" not in row or not row["reference"]:
      raise ValueError("reference is missing in the test sample. Please add reference to the test sample.")

 if "user_input" not in row or not row["user_input"]:
      raise ValueError("user_input is missing in the test sample. Please add user_input to the test sample.")

 if "response" not in row or not row["response"]:
       raise ValueError("response is missing in the test sample. Please add response to the test sample.")

 if "retrieved_contexts" not in row or not row["retrieved_contexts"]:
       raise ValueError("retrieved_contexts is missing in the test sample. Please add retrieved_contexts to the test sample.")

@dosubot dosubot bot added the size:S This PR changes 10-29 lines, ignoring generated files. label Mar 29, 2025
@shahules786 shahules786 self-requested a review April 4, 2025 02:40
Copy link
Member

@shahules786 shahules786 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks bro

@shahules786 shahules786 merged commit 23c86a3 into explodinggradients:main Apr 4, 2025
17 checks passed
@KalyanKS-NLP KalyanKS-NLP deleted the feature/clear_error_message branch April 5, 2025 16:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size:S This PR changes 10-29 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants