The Scrapingdog Blog
Designed with you in mind, our web scraping tutorials cater to any skill level. Each guide is straightforward and packed with useful insights.
Web scraping using Puppeteer and Nodejs
TL;DR Hands-on tutorial: scrape with Puppeteer (render) + Cheerio (parse) in Node.js . Demo targets a product page and extracts title , price , and stock…
Aug 23, 2024 7 min read4 Best Python HTML Parsers
TL;DR Compares 4 Python HTML parsers: BeautifulSoup , lxml , html5lib , PyQuery . BS4 : easiest; handles messy HTML. lxml : fast with XPath / XSLT; great…
Aug 23, 2024 10 min read5 Best Free Proxy Lists for Web Scraping
TL;DR 5 free proxy lists; many free proxies are unsafe ,only 21% allowed HTTPS in a study. Bench: 500 requests each to Google / Amazon / eBay /…
Aug 23, 2024 9 min read8 Tips To Avoid Getting Blocked While Web Scraping (Updated)
TL;DR Know the traps: robots.txt , IP bans, CAPTCHAs, honeypots, JS-rendered pages. Rotate IPs (residential / mobile) and User-Agent ; send real headers /…
Aug 22, 2024 10 min readHow To Scrape Idealista.com with Python
TL;DR Python guide to scrape Idealista with requests + BeautifulSoup . Parses listing cards to get title, price, area, description, and property link.…
Aug 15, 2024 8 min readWeb Scraping with jQuery
TL;DR Client-side jQuery scraping hits CORS; do it server-side instead. In Node.js , use jsdom + jQuery : fetch with $.get , select elements, extract…
Aug 15, 2024 15 min readWeb Scraping with Node-Fetch
TL;DR Node.js tutorial: use node-fetch + Cheerio to grab HTML, parse data, and write JSON . Flow: install deps, enable ESM, fetch, select / loop items,…
Aug 14, 2024 16 min read
No articles found
Try a different keyword or category.
Try Scrapingdog for Free!
Get 200 free credits to spin the API. No credit card required!
Try Scrapingdog for Free!
Get 200 free credits to spin the API. No credit card required!