[Feature] `launchPersistentServer` Launches persistent storage browser server

[roblav96]

I love the launchServer feature, without it my janky 2014 MacBook Pro (with scissor switch keyboard 😇) battery would die instantly from relaunching the browser process (headless: false) every time I make a change to the code.

Problem

As of playwright@0.12.1, can't call launchServer that uses persistent storage located at userDataDir.

Any attempt throws Playwright manages remote debugging connection itself.

Solution

Either or,

1. Expose an option on the launchServer method to define userDataDir persistent storage.

2. Expose an option on the launchPersistentContext method to set the LaunchType to server.

[dgozman]

Could you tell us more about your usecase? It seems like you are using persistent context - what for? Context creation is very fast, so perhaps you could just setup a new context in a way you want it, instead of launching a browser?

[roblav96]

@dgozman I have two scripts,

1. launch.ts

This will launch the browser instance and act as the master process.

const { browserServer } = await (playwright.chromium as any)._launchServer(
	{ headless: false },
	'server',
	path.resolve(process.cwd(), `.chromium-data-dir`),
	9222,
)
console.log('browserServer.wsEndpoint() ->', browserServer.wsEndpoint())

I was able to get a persistent storage BrowserServer instance with this, and logged the browserServer.wsEndpoint().

2. index.ts

This will use browserType.connect({ wsEndpoint: '...' }) to connect to the launched browser instance from launch.ts.

I'd like to be able to control the launched browser from launch.ts in non-headless mode for debugging purposes, navigate to a page I'd like to run my index.ts on, and be able to access all the context.Pages().

const remote = await browser.connect({ wsEndpoint: '...' })
console.log(`remote.isConnected() ->`, remote.isConnected()) // -> true
const context = (remote as any)._defaultContext as playwright.BrowserContext
console.log('context.pages().length ->', context.pages().length) // -> []

I'd like to have the context.pages() method return all browser tabs currently open. This will allow for a far more efficient developer experience without having to restart a Chromium.app process every time a code change happens.

[dgozman]

Thank you for the details! Could you please clarify the reason you need a persistent browser context? Perhaps, a regular context could work for you?

I'd like to be able to control the launched browser from launch.ts in non-headless mode for debugging purposes, navigate to a page I'd like to run my index.ts on, and be able to access all the context.Pages().

I am also interested in why would you want to manually navigate instead of issuing page.goto() in the index.ts file.

Overall, this scenario is not currently on our radar, so I'd really like to understand it better before proposing any changes, to ensure the best fit.

[sbekrin]

Just to add my 2c: this is very helpful when browser instance is launched "on-demand" with a warmed up temp. user profile. Currently, I'm using the internal _launchServer as a workaround as @roblav96 has mentioned.

[char101]

To launch persistent server

server.js

const path = require('path');
const fs = require('fs');
const { chromium } = require('playwright');

(async function() {
  const options = {};
  const profilePath = path.join(__dirname, 'profile');
  const { browserServer: server } = await chromium._launchServer(options, 'server', profilePath);

  const endpoint = server.wsEndpoint();
  console.log(endpoint);
  fs.writeFileSync('endpoint.txt', endpoint);
})();

client.js

const fs = require('fs');

(async function() {
  async function connect(options) {
    const crBrowser = require('playwright-core/lib/chromium/crBrowser');
    const transport = require('playwright-core/lib/transport');
    const logger = require('playwright-core/lib/logger');
    return await transport.WebSocketTransport.connect(options.wsEndpoint, transport => {
      return crBrowser.CRBrowser.connect(transport, true, new logger.RootLogger(options.logger), options);
    });
  }

  const endpoint = fs.readFileSync('endpoint.txt', { encoding: 'utf8' });
  const browser = await connect({
    wsEndpoint: endpoint,
    timeout: 1000,
  });

  const context = browser._defaultContext;

  const page = await context.newPage();
})();

[char101]

In my case a persistent context is required because the site I am automating sends a notification every time a new device is connect and old devices are detected based on a persistent cookie.

[char101]

context.pages() in this case returns an empty array because the context is compared using object equality here https://github.com/microsoft/playwright/blob/master/src/chromium/crBrowser.ts#L310.

To retrieve the existing pages we need to read browser._crPages

for (const p of browser._crPages) {
  const page = p._initializedPage;
}

[pavelfeldman]

We seemed to have dropped the ball on this one.

[dgozman]

Thank you everyone for the details! Unfortunately, this feature does not fit well with Playwright's focus on testing and isolation.

We are working on making non-persistent contexts suit everyone's needs better, so that you don't have to use persistent context at all. For example, it is possible to save all cookies using context.cookies() and then initialize context with cookies using context.addCookies(). This way, you don't need a persistent context, don't have to keep userDataDir around, and can get as many isolated contexts that have desired cookies as you'd like.

[windbridges]

Thank you everyone for the details! Unfortunately, this feature does not fit well with Playwright's focus on testing and isolation.

We are working on making non-persistent contexts suit everyone's needs better, so that you don't have to use persistent context at all. For example, it is possible to save all cookies using context.cookies() and then initialize context with cookies using context.addCookies(). This way, you don't need a persistent context, don't have to keep userDataDir around, and can get as many isolated contexts that have desired cookies as you'd like.

Unfortunately, persistent context is not just a saved cookies, but cache, storages, user preferences and more. In addition, this is a little illogical: the API exposes two methods of browser creation - launch() and launchServer(). The one of them has an option to launch persistent context, while another hasn't. It looks a little confusing and violates the integrity of the API.

Actually, it's great that API has launchPersistentContext method, but it's bad that we can not to retrieve the wsEndpoint from returned context to connect to it from another application process.

And yes, in this case launchPersistentServerContext (or smth like that) will require passing userDataDir. Of course, we should not be able to pass the userDataDir there, since server path management should be done by the server only. But maybe we can use unique session identifiers instead? Actually we even don't want to know where exactly the data is stored, we just want to differ one profile from another.

In a nutshell, I would like to emphasize the importance to not just to launch a persistent contexts, but to connect to them (from another Node Js process for example). Currently we can connect to isolated conexts only. Why are permanent ones worse?

So, maybe you could reconsider your opinion about this?

[x0day]

@pavelfeldman could you reconsider this issue?