See evaluation.spec.js "should await promise from popup". The only reason it worked before was that the promise was resolved by the time it was awaited. I suspect that the issue is that the debugger of this page never received onPromiseSettled events of a new page. The test never worked and recent changes to the evaluation / boxing surfaced it.