Data scrubbing changes never reach Relay: taskworker rebuilds project configs from stale organization options #127554

Environment

self-hosted (https://develop.sentry.dev/self-hosted/)

Steps to Reproduce

1. In Organization Settings > Security & Privacy, add a new entry to Global Sensitive Fields (for example my_probe) and save.
2. Wait a few seconds. organizationoption saving queues invalidate_project_config, and the taskworker runs it: every cached project config in projectconfig_cache gets a new rev and lastFetch about 8 seconds after the save.
3. Read a cached config: datascrubbingSettings.sensitiveFields does not contain my_probe. Send an event with extra.my_probe set: it is stored unscrubbed. We kept checking for 8 minutes, past Relay's project expiry.
4. In a fresh sentry shell, in the web container or in the taskworker's own container, compute_configs(organization_id=...) returns configs that do contain my_probe. Running invalidate_project_config(organization_id=...) inline there fixes the cache.
5. Queue another refresh later, from schedule_invalidate_project_config(trigger=..., organization_id=...) inside a committed transaction. The taskworker runs it within seconds (rev changes on every key), but it still writes configs without my_probe.

Expected Result

A refresh run by the taskworker builds project configs from the organization options as saved, so new data scrubbing settings reach Relay within seconds of saving them.

Actual Result

The taskworker keeps rebuilding project configs from the organization options it read earlier. A settings change never takes effect until each long-running worker process is replaced (--max-child-task-count=10000). Worse, any refresh queued later by a process that still holds the old options writes them back, even after a manual inline rebuild.

Likely cause. OptionManager._option_cache (src/sentry/db/models/manager/option.py) is a process-local cache (_local_cache). It is cleared only by request_finished.connect(self.clear_local_cache), at the end of a web request. Nothing clears it between taskworker tasks: in 26.7.2, clear_local_cache has no call site in the taskworker. So a task reads organization and project options as they were when its worker process first read them.

Workaround. After a settings change, restart the taskworker container, then run invalidate_project_config(organization_id=...) inline in sentry shell.

First filed as getsentry/self-hosted#4561, where we were told it belongs here.

Product Area

Settings - Security & Privacy

Link

No response

DSN

No response

Version

26.7.2 (self-hosted)