[Serve] fix _to_object_ref memory leak

[GeneDer]

Why are these changes needed?

_PyObjScanner/ CloudPickler is designed to pin object refs into memory so they don't escape the scope of existing Ray session. This PR allows ObjectRef and ObjectRefGenerator to also escape the Ray memory pin so they won't cause memory leaks.

Related issue number

Closes #43248

Checks

• I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've added any new APIs to the API Reference. For example, if I added a
    method in Tune, I've added it in doc/source/tune/api/ under the
    corresponding .rst file.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[GeneDer]

Also sanity checked on workspace and seeing the fixed worked 😄

After the fix

Before the fix

[shrekris-anyscale]

Nice work tracking this down! What was the root cause of the leak? Why were ObjectRefs getting pinned by the CloudPickler in the first place?

[shrekris-anyscale]

Ah never mind, I took a closer look at the initial issue. Looks like the workload involved explicitly passing ObjectRefs around.

[GeneDer]

Credit to @edoakes! Couldn't have found the root cause without pairing 😄

[edoakes]

Nice work tracking this down! What was the root cause of the leak? Why were ObjectRefs getting pinned by the CloudPickler in the first place?

It's because when we actually cloudpickle object refs, we want to pin them (because we don't know their lifecycle anymore). The _PyObjScanner uses the cloudpickler so as a side effect it was also pinning 🤦. We avoid this by not calling into the CloudPickler for object refs by adding it to the target type.

[GeneDer]

@edoakes this is ready for merge