iOS: Filament rendering freezes after app backgrounding/foregrounding

[mekami-io]

iOS: Filament rendering freezes after app backgrounding/foregrounding

Problem Description

On iOS, the 3D rendering occasionally freezes after the app is sent to background and then returned to foreground. This issue occurs sporadically (not every time) and seems to be related to timing conditions during the app lifecycle transition.

Symptoms

• 3D Scene Frozen: The Filament 3D rendering becomes completely static/frozen
• React Callbacks Continue: RenderCallbackContext.useRenderCallback worklets continue to execute normally
• Data Updates Working: Debug logs show that data is still being updated in the background
• UI Remains Functional: The rest of the React Native app works perfectly fine
• Only Filament Affected: Only the 3D rendering portion is frozen

Steps to Reproduce

It's kind of hard to reproduce, since it only occurs occasionally. But it would be:

1. Create a React Native app using react-native-filament
2. Set up a 3D scene with animation (using useRenderCallback)
3. Run the app on iOS device/simulator
4. Send the app to background (home button/app switcher)
5. Wait for some time (can be seconds, minutes or hours)
6. Return to the app
7. Sometimes (not always) the 3D rendering will be frozen

Expected Behavior

The Filament 3D rendering should resume normally when returning to the app, just like it was before backgrounding.

Actual Behavior

The 3D scene becomes completely frozen while all other app functionality continues to work. The only way to recover is to completely restart the app.

Environment

• Platform: iOS (device and simulator)
• React Native: 0.74, 0.75, 0.76
• react-native-filament: Happens on all versions up to 1.6.1
• iOS Version: Tested on iOS 17 and 18
• Devices: Tested on multiple iOS devices

Technical Analysis

After investigation, this appears to be related to iOS Metal framework behavior during app lifecycle transitions:

Root Cause

1. App Backgrounding: When the app goes to background, iOS may invalidate Metal resources including CAMetalLayer drawables
2. Resource Invalidation: The CAMetalLayer.nextDrawable method may return nil after foregrounding
3. Filament Context Loss: Filament's Metal backend loses its rendering context but doesn't properly recover
4. No Recovery Mechanism: There's currently no automatic recovery when Metal resources become invalid

Evidence

• Debug logs show RenderCallbackContext.useRenderCallback continues executing
• Data updates are processed normally in the render callbacks
• The freeze only affects the actual 3D rendering output
• Issue is specific to iOS (Metal backend)
• Timing-dependent (doesn't happen every time)
• Calling choreographer.start() programmatically does not resolve the freeze
• Complete unmounting and remounting of the FilamentScene (including FilamentView) does recover the rendering, but this is not a practical solution for production apps

Current Workarounds

• Complete app restart (only reliable solution)
• Complete unmounting/remounting of FilamentScene components (works but destroys user experience since it's currently not possible to detect if the rendering is frozen)
• No practical programmatic workaround currently available

Status

I'm currently working on a fix for this issue and have implemented a potential solution that addresses the Metal resource invalidation during iOS app lifecycle transitions. The fix involves:

• Proper handling of iOS app lifecycle notifications
• Metal drawable validation and recovery mechanisms
• Integration with the existing DisplayLink management
• Automatic Metal layer reset when drawables become invalid

Initial testing shows promising results, but since this issue occurs sporadically and is timing-dependent, I want to test the fix thoroughly over a longer period before submitting a pull request. I'm currently running extended tests to ensure the solution is robust and doesn't introduce any regressions. Once I'm confident the fix is stable and reliable, I'll submit a pull request with the complete solution.