Extracting hrefs

[darasus]

What would be the best approach to extracting hrefs?

[anerli]

Hey - currently our extract implementation is based on whats visible in the screen and content in the accessibility tree. Including DOM content quickly bloats the context window - we're working on ways to preprocess the DOM and scrape more comprehensive data.

[anerli]

Hey @darasus - newest version overhauls extract so it does a much better job of picking up stuff like hrefs. If you update to the latest version you should find it works a lot better!

[darasus]

@anerli what would be your recommendation in of instruction and provided schema?

[anerli]

Depends what links you are trying to pull - if you're just trying to get all of them I would just try something like:

const links = agent.extract('list all links', z.array(z.string()));

[darasus]

I'm trying to extract list of profile hrefs from a social media site.

Do you not need to specify .url() in zod schema?

I've tried different instructions and schemas but didn't have much luck so far.

[anerli]

Hey, what version are you on? String should be fine if instructions are clear, not sure if having url in the zod schema would actually help or not

[anerli]

Also - do note that the content to be extracted has to be on the active page - if its included in the DOM at the time extract() is called, the LLM should be able to see it.