Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -539,7 +539,7 @@ persistent. See also [`BrowserType.launch_persistent_context`](https://playwrigh
Note that persistent contexts are launched independently from the main browser
instance, hence keyword arguments passed in the
[`PLAYWRIGHT_LAUNCH_OPTIONS`](#playwright_launch_options)
setting do not apply.
setting do not apply.

### Creating a context during a crawl

Expand Down Expand Up @@ -592,14 +592,22 @@ def parse(self, response):
async def parse_in_new_context(self, response):
page = response.meta["playwright_page"]
title = await page.title()
await page.close()
await page.context.close()
return {"title": title}

async def close_context_on_error(self, failure):
page = failure.request.meta["playwright_page"]
await page.close()
await page.context.close()
```

### Avoid race conditions & memory leaks when closing contexts
Make sure to close the page before closing the context. See
[this comment](https://github.com/scrapy-plugins/scrapy-playwright/issues/191#issuecomment-1548097114)
in [#191](https://github.com/scrapy-plugins/scrapy-playwright/issues/191)
for more information.

### Maximum concurrent context count

Specify a value for the `PLAYWRIGHT_MAX_CONTEXTS` setting to limit the amount
Expand Down
1 change: 1 addition & 0 deletions examples/contexts.py
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,7 @@ async def parse(self, response):
page = response.meta["playwright_page"]
context_name = response.meta["playwright_context"]
storage_state = await page.context.storage_state()
await page.close()
await page.context.close()
return {
"url": response.url,
Expand Down
2 changes: 1 addition & 1 deletion tests/test_browser_contexts.py
Original file line number Diff line number Diff line change
Expand Up @@ -160,8 +160,8 @@ async def test_contexts_startup(self):

page = resp.meta["playwright_page"]
storage_state = await page.context.storage_state()
await page.context.close()
await page.close()
await page.context.close()
cookie = storage_state["cookies"][0]
assert cookie["name"] == "foo"
assert cookie["value"] == "bar"
Expand Down