feat(ext): Speculative Reasoning Execution (SRE) for DeepSeek-R1 / NVIDIA NIM

[Yash3561]

Why are these changes needed?

Reasoning models like DeepSeek-R1 introduce significant "Reasoning Latency" (10s-60s) during their Chain-of-Thought (CoT) phase. Currently, AutoGen agentic loops are sequential, leaving compute resources idle while waiting for the <think> block to terminate.

This PR introduces Speculative Reasoning Execution (SRE), allowing AutoGen to parallelize model "thought" with tool "action."

Technical Changes

• NvidiaSpeculativeClient: A specialized extension for autogen-ext that peeks into the reasoning stream to identify tool-call intents.
• ReasoningSniffer: A high-speed heuristic engine that detects high-confidence intents within streaming tokens.
• SpeculativeCache: A thread-safe result vault for pre-warmed tool outputs, enabling 0ms latency on formal tool requests.

Real-World Benchmarks (NVIDIA A100-80GB)

Validated on institutional HPC hardware using DeepSeek-R1-Distill-Qwen-8B:

• Baseline (Sequential): 13.4s Time-to-Action.
• SRE (Speculative): 1.6s Time-to-Action.
• Achievement: 85% reduction (11.8s saved) in wait-time by parallelizing I/O pre-warming with model reasoning.

Checks

• 26 comprehensive unit/integration tests passing (pytest).
• Fully backward-compatible wrapper for existing model clients.
• Linting and formatting (Black/Ruff) verified.

[ashaffir]

Impressive PR.
The 85% reduction in Time-to-Action is a massive improvement for interactive agent workflows. The ReasoningSniffer is a clever way to front-load I/O.
I'm curious how its heuristic accuracy holds up under different sampling temperatures. It might be a useful diagnostic to run a sweep across a range of temperature values. Tracking hits, misses, and misfires against temperature could reveal the robustness of the approach when the model's output is more varied. The performance gains are very compelling.

[Yash3561]

Great point @ashaffir. High-variance CoT streams at higher temperatures are exactly why we implemented the heuristic buffering in the ReasoningSniffer. I am currently running a temperature sweep (0.0 to 1.0) on an NVIDIA A100 cluster to quantify the 'Misfire Rate' vs. 'Latency Gain'. I’ll update the PR with the robustness data shortly. Preliminary results suggest the regex-based intent capture remains stable at temp=0.7 due to the semantic anchor points we've targeted.