Skip to content

fix(eval): handle None score in LocalEvalSampler to prevent TypeError crash#5420

Open
nileshpatil6 wants to merge 2 commits intogoogle:mainfrom
nileshpatil6:fix/eval-sampler-none-score
Open

fix(eval): handle None score in LocalEvalSampler to prevent TypeError crash#5420
nileshpatil6 wants to merge 2 commits intogoogle:mainfrom
nileshpatil6:fix/eval-sampler-none-score

Conversation

@nileshpatil6
Copy link
Copy Markdown

Problem

When a metric evaluation fails (transient API error, rate limiting, JSONDecodeError), local_eval_service.py catches the exception and returns an EvaluationResult with score=None. However, local_eval_sampler.py line 292 then calls round(eval_metric_result.score, 2) unconditionally, crashing with:

TypeError: type NoneType doesn't define __round__ method

Fixes #5403

Fix

Guard the round() call with a None check:

"score": round(eval_metric_result.score, 2) if eval_metric_result.score is not None else None,

@nileshpatil6
Copy link
Copy Markdown
Author

I have read the Contributor License Agreement and I hereby agree to its terms.

@adk-bot
Copy link
Copy Markdown
Collaborator

adk-bot commented Apr 20, 2026

Response from ADK Triaging Agent

Hello @nileshpatil6, thank you for submitting this pull request!

To help us review it more efficiently, could you please add a testing plan section to your PR description explaining how you tested these changes?

Additionally, since this is a bug fix, could you please provide logs or a screenshot demonstrating that the TypeError is resolved after your changes?

You can find more details in our contribution guidelines. Thanks!

@adk-bot adk-bot added the eval [Component] This issue is related to evaluation label Apr 20, 2026
@nileshpatil6
Copy link
Copy Markdown
Author

Testing Plan

Manual verification:

from unittest.mock import MagicMock
from google.adk.optimization.local_eval_sampler import LocalEvalSampler

# Simulate a metric result with None score (happens when evaluation fails)
mock_metric_result = MagicMock()
mock_metric_result.score = None
mock_metric_result.metric_name = 'test_metric'
mock_metric_result.eval_status.name = 'NOT_EVALUATED'

mock_per_invocation = MagicMock()
mock_per_invocation.eval_metric_results = [mock_metric_result]

mock_eval_result = MagicMock()
mock_eval_result.eval_metric_result_per_invocation = [mock_per_invocation]

Before fix:

TypeError: type NoneType doesn't define __round__ method

at round(eval_metric_result.score, 2) when score is None.

After fix: The None score is passed through as-is (None) instead of crashing, consistent with how the upstream local_eval_service.py already uses None to represent a failed evaluation.

Reproducing the original crash (as described in issue #5403): trigger a transient API error or JSONDecodeError during metric evaluation via adk optimize — the sampler now handles it gracefully instead of crashing.

@rohityan rohityan self-assigned this Apr 20, 2026
@rohityan rohityan added the request clarification [Status] The maintainer need clarification or more information from the author label Apr 20, 2026
@rohityan
Copy link
Copy Markdown
Collaborator

Hi @nileshpatil6 , Thank you for your contribution! We appreciate you taking the time to submit this pull request. Please fix formatting errors by running autoformat.sh

@nileshpatil6
Copy link
Copy Markdown
Author

Hi @nileshpatil6 , Thank you for your contribution! We appreciate you taking the time to submit this pull request. Please fix formatting errors by running autoformat.sh

Thank you for responding , I have Fixed the issue, Ran the formatting fix -- the ternary expression is now wrapped across multiple lines per pyink style. Pushed in fe19f31.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

eval [Component] This issue is related to evaluation request clarification [Status] The maintainer need clarification or more information from the author

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TypeError in LocalEvalSampler when metric evaluation fails

3 participants