-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Description
Hi,
I have been trying to apply qwen3-coder to a mathematical reasoning use case, since as far as my understanding goes, it should be better in this task than a standard qwen model.
As a first step I am trying to make the model extract variables with their values from a given prompt. I am using a structured outputs approach, where i give a pydantic model's json schema with the structure of the variable. One of the fields is a float value, representing the value of the variable. You can see the schema below:
class Variable(BaseModel):
symbol: str = Field(
description="Short algebraic symbol used inside equations, e.g. 'v', 'm', 'KE'."
)
description: str = Field(
description="Human-readable description of what this variable represents."
)
unit: Optional[str] = Field(
default=None,
description="Physical or financial unit, e.g. 'm/s', 'kg', 'USD'. Omit if dimensionless."
)
known_value: Optional[float] = Field(
default=None,
description=(
"Numeric value if the problem statement provides it. Do not add trailing zeros - e.g. write 0.5 instead of 0.500"
"Set to null if this variable is unknown and must be derived."
)
)Whenever I try this approach with the Qwen3-Coder-Next model, it is outputting an infinite list of zeros for the known_value field. I have tried to solve it with prompting, but I have not been successful. I find it very curious, since when testing other models of the Qwen3 family (more specifically Qwen3-VL-30B-A3B-Instruct) it has worked flawlessly, and this coding model that should be better at mathematics related tasks is hallucinating a lot. You can see a snippet of the output I get below:
Example [0]: Kinematics – final velocity
Problem:
A car starts from rest and accelerates uniformly at 3 m/s² for 8
seconds. What is the final velocity and how far does it
travel?
Streaming model output (raw JSON tokens):
────────────────────────────────────────────────────────────────────
{
"problem_summary": "A car starts from rest and accelerates uniformly at 3 m/s² for 8 seconds. Find final velocity and distance traveled.",
"variables": [
{
"symbol": "u",
"description": "Initial velocity",
"known_value": 0.0000000000000000e+000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000Is there any known solution to this kind of problem? Thanks in advance.