Scrapy, a fast high-level web crawling & scraping framework for Python.
-
Updated
Apr 9, 2025 - Python
#
web-scraping-python
Here are 155 public repositories matching this topic...
Python APIs for web automation, testing, and bypassing bot-detection.
python webdriver selenium test-automation pytest web-scraping chromedriver webkit pytest-plugin cdp behave bot-detection web-automation python-scraper selenium-python e2e-testing cloudflare-bypass seleniumbase anti-detection web-scraping-python
-
Updated
Apr 20, 2025 - Python
🕷️ An undetectable, powerful, flexible, high-performance Python library that makes Web Scraping simple and easy again!
python crawler data automation ai scraping crawling web-scraper python3 web-scraping selectors xpath data-extraction stealth webscraping hacktoberfest crawling-python playwright web-scraping-python ai-scraping
-
Updated
Apr 17, 2025 - Python
The All in One Framework to Build Undefeatable Scrapers
anti-bot web-crawling bot-detection python-scraper anti-detect undetected scraping-framework undetectable python-web-scraper scraping-tool cloudflare-bypass scraping-python python-web-scraping anti-detection cloudflare-scrape bypass-cloudflare web-scraping-python undetected-chromedriver antidetect-browser anti-detect-browser
-
Updated
Apr 13, 2025 - Python
AgentQL is a suite of tools for connecting your AI to the web. Featuring a query language and Playwright integrations for interacting with elements and extracting data quickly, precisely, and at scale. Includes REST API, Python and JavaScript SDKs, browser debugger.
javascript python agent automation web ai scraping web-scraping web-scrapping rpa playwright web-scraping-python web-scraping-javascript aiagent webagent web-scraping-colabs
-
Updated
Mar 26, 2025 - Python
Scalable Python web scraping scripts for +40 popular domains
python crawler scraper automation spider web-crawler scraping crawling webscraper proxies web-scraping webscraping twitter-scraper datascraping python-scraper captcha-bypass antibot scraping-python crawling-python web-scraping-python
-
Updated
Mar 27, 2025 - Python
In this Python Web Scraping Tutorial, we will outline everything needed to get started with web scraping. We will begin with simple examples and move on to relatively more complex.
python crawler scraping web-scraping python-web-crawler webscraping web-crawler-python python-web-scraper python-projects web-scraping-python github-python web-scraping-api scraper-python amazon-scraper-python json-database-python
-
Updated
Apr 9, 2025 - Python
A simple and easy to use web crawler for Python
python crawler scraping crawling web-scraping python-web-crawler python-package web-crawler-python web-scraping-python
-
Updated
Aug 27, 2024 - Python
Web scraping framework built for AI applications. Extract clean, structured content from any website with dynamic content handling, markdown conversion, and intelligent crawling capabilities. Perfect for RAG applications and AI training data pipelines. Features async processing, browser management, and Prometheus monitoring.
-
Updated
Feb 26, 2025 - Python
TLS Requests is a powerful Python library for secure HTTP requests, offering browser-like TLS client, fingerprinting, anti-bot page bypass, and high performance.
anti-bot-page anti-bot python-web-crawler python-crawler python-spider web-crawler-python web-spider python-scraper tls-client python-web-scraper cloudflare-bypass scraping-python python-web-scraping crawling-python web-scraping-python anti-bot-detection cf-clearance cloudflare-scraper web-scraping-api python-tls-client
-
Updated
Mar 9, 2025 - Python
Web scraping API for building AI applications.
python scraping crawling web-scraping chatbots web-crawling scraping-python crawling-python web-scraping-python
-
Updated
Jan 24, 2024 - Python
CobWeb is a Python library for web scraping. The library consists of two classes: Spider and Scraper.
python crawler scraper web-crawler scraping web-scraper web-crawler-python cli-tool web-scraping-python
-
Updated
Nov 28, 2023 - Python
A short introduction to scraping with Python with given steps and an example scraper script.
python learning crawler data-science data-mining scraping web-scraping beautifulsoup python-web-crawler webscraping web-crawler-python python-web-scraper python-projects web-scraping-python github-python web-scraping-api scraper-python json-database-python
-
Updated
Apr 19, 2024 - Python
Builds a Tagalog dictionary by collecting Tagalog words from tagalog.pinoydictionary.com
python scraper database dictionary scraping web-scraper beautiful-soup web-scraping tagalog tagalog-dictionary philippines web-scrapping philippines-json web-scraping-python philippines-dataset
-
Updated
Feb 19, 2023 - Python
WatchTower - A platform to save your valuable time while staying updated in the Cyber realm.
digitalocean pandas python3 requests cybersecurity scrapy bash-scripting vercel-deployment web-scraping-python
-
Updated
Mar 8, 2025 - Python
The News Headlines Tracker application collects the latest news headlines from major news sources such as CNN, BBC, and The New York Times.
-
Updated
Jul 15, 2024 - Python
-
Updated
Oct 7, 2023 - Python
The Facebook Comments Crawler is an unofficial tool for extracting comments from Facebook posts using Selenium in Python. It's designed to aid in academic and personal research. #Facebook comments scaper #Facebook comments crawler
facebook selenium facebook-crawler web-scraping-python facebook-comments-scraper crawl-facebook facebook-comment-crawler
-
Updated
Oct 30, 2024 - Python
"instagram-post-fetcher" is a Python module leveraging Selenium to extract Instagram post details, including account username, descriptions, media URLs, and post timestamps. Simplifying access to Instagram data for analytics and research.
python pictures instagram data-mining automation web-scraping instagram-scraper videos data-extraction data-analysis instagram-metadata osint-python instagram-cli web-scraping-python
-
Updated
Jul 2, 2024 - Python
Repository designed to help freshers easily grasp the basics of web scripting, offering simple guides and examples to build a strong foundation.
python scraping web-scraper python3 web-scraping scrape web-scrapping web-scraping-tutorials scrap-data python-web-scraper scraping-python scraping-images scraping-web web-scrapper-python scraping-data web-scraping-python web-scraping-project simple-scraping
-
Updated
Jan 28, 2025 - Python
Improve this page
Add a description, image, and links to the web-scraping-python topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the web-scraping-python topic, visit your repo's landing page and select "manage topics."