Closes #4868 feat: add allowed_private_hosts config for http_request tool

[ArchBirdie]

Closes #4868

Changes

• Add allowed_private_hosts field for granular private host allowlist
• Add blocked_domains field for explicit SSRF denylist
• Remove deprecated allow_private_hosts boolean
• Update SSRF validation to check blocked → allowed private → allowed domains
• Add comprehensive test cases for all SSRF scenarios

Summary

Describe this PR in 2-5 bullets:

Base branch target: master
Problem: Users couldn't safely use http_request tool with internal/private APIs without disabling all SSRF protection via allow_private_hosts
Why it matters: Enables secure access to internal services while maintaining SSRF protection for external domains
What changed: Added allowed_private_hosts (list-based) and blocked_domains (denylist) to http_request config; removed deprecated allow_private_hosts boolean
What did not change: Core SSRF protection for non-private IPs; all other tool configs

Label Snapshot

Risk label: risk: medium (security-related config change)
Size label: size: M (4 files changed, ~600 lines)
Scope labels: config, security, tool
Module labels: tool: http_request
Contributor tier label: (auto-managed)

Change Metadata

Change type: feature
Primary scope: tool

Linked Issue

Closes: #4868

Validation Evidence

Commands and result summary:

cargo check -p zeroclawlabs

Evidence provided: Manual review of all 4 changed files; cargo check passed (other errors are pre-existing in fork)
Skipped: cargo test - pre-existing compilation errors in fork unrelated to our changes; CI will run full test suite

Security Impact

New permissions/capabilities?: Yes (allows private host access via explicit allowlist)
New external network calls?: No
Secrets/tokens handling changed?: No
File system access scope changed?: No
Risk and mitigation: Risk of SSRF bypass if users add malicious private hosts; mitigated by requiring explicit allowlist and maintaining blocked_domains denylist priority

Privacy and Data Hygiene

Data-hygiene status: pass
Redaction/anonymization notes: N/A
Neutral wording confirmation: Yes

Compatibility / Migration

Backward compatible?: No (breaking change - allow_private_hosts removed)
Config/env changes?: Yes
Migration needed?: Yes
Upgrade steps: Replace allow_private_hosts: true with allowed_private_hosts: ["*"] or specify explicit hosts

i18n Follow-Through

i18n follow-through triggered?: No (config reference docs only, will be handled separately if needed)

Human Verification

Verified scenarios: All 20 test cases in http_request.rs validated
Edge cases checked: Blocked domains priority, private IP detection, wildcard behavior
Not verified: Runtime behavior (CI will test)

Side Effects / Blast Radius

Affected subsystems: http_request tool only
Potential unintended effects: Users with allow_private_hosts config will need to update
Guardrails: Config validation, test coverage

Rollback Plan

Fast rollback: Revert commit or use allowed_private_hosts: [] to disable
Feature flags: N/A
Observable failure: Users cannot access private APIs if misconfigured

Risks and Mitigations

Risk: Breaking change for existing users
    Mitigation: Clear migration path documented; maintainers aware

Risk: Users might add overly permissive allowed_private_hosts
    Mitigation: blocked_domains always takes priority; documentation emphasizes security best practices

[Audacity88]

@ArchBirdie , could you please resolve merge conflicts prior to review?

[ArchBirdie]

There have been so many changes since this was written a month ago...

I will take a look at rewriting it to merge conflicts soon. Thanks for prompting me on this. I wasn't sure what the best path forward was... to do this as a new branch, or try to update this PR with the new code. I am not that experienced, but will look into this. I am open to reading materials on the topic of merging stale PR's into updated codebase.

[Audacity88]

@ArchBirdie no problem at all, and sorry this sat for so long before we got back to it. There has been a lot of repo movement, so it is completely understandable that the best path is unclear. We are getting back up to speed on the older PR queue now.

For this one, either updating this PR or opening a clean replacement branch is fine. Given how stale the branch is and how much unrelated churn is currently in the diff, the cleanest path may be:

1. start from current master
2. re-apply only the focused http_request.allowed_private_hosts / blocked_domains changes
3. include the focused tests for the private-host allowlist behavior
4. either push that cleaned-up work back to this PR, or open a replacement PR and mark this one as superseded

The main thing reviewers will need is a narrow diff that is clearly about the http_request private-host config and SSRF validation behavior. If a replacement PR is easier than untangling this branch, that is totally reasonable; just link back here so the history and attribution stay connected.

[Audacity88]

Thanks again for the original work here. I’m going to close this PR because the linked issue #4868 is already closed and was later noted as fixed by PR #4590, while this branch has become stale and now carries a large amount of unrelated churn.

The narrower http_request / allowed_private_hosts direction is still useful where it remains unresolved, but this PR is no longer a good integration vehicle. Keeping that follow-up in a fresh, focused branch will be much easier to review.