This course was created with the
course builder. Create your online course today.
Start now
Create your course
with
Autoplay
Autocomplete
Previous Lesson
Complete and Continue
Web Scraping for Data Science-Python: BS4, Selenium & Scrapy
Introduction
What is the best web scraping library? BS4 vs Selenium vs Scrapy (3:30)
Read This Before You Start
Installing Python and PyCharm (4:57)
Python Basics for Web Scraping (30:31)
Web Scraping Cheat Sheet for The Course (Don't skip it)
HTML for Web Scraping: Tags and Elements (7:49)
HTML for Web Scraping: Tree Structure (3:52)
Introduction to Beautiful Soup
Section Overview
Installing Beautiful Soup (3:13)
Beautiful Soup Basics (4:01)
How To Get The HTML from a Website (4:41)
How to Scrape a Single Page (10:34)
Exporting Data to a TXT File (3:20)
Scraping Multiple Pages with Beautiful Soup
Section Overview
Scraping Multiple Links within The Same Page (16:27)
Pagination (18:27)
XPath
Important Note (Don't skip it!)
XPath Syntax, Functions and Operators (3:41)
Test Your XPath (5:29)
Special Characters and Syntax (4:37)
Introduction to Selenium
Section Overview (+ Important Note)
How to Identify Java Script Driven Websites (3:25)
Installing Selenium and ChromeDriver (2:19)
Importing Libraries and Creating The Driver (3:37)
How to Find Elements with Selenium (3:41)
Clicking on a Button (6:36)
Extracting Data from a Table (12:26)
Exporting Data to a CSV File with Pandas (5:14)
Selecting Elements within a Dropdown (7:41)
Project #1: Dealing with Multiple Pages using Selenium (Amazon Audible Bot)
Important Note (Update)
Building the Bot (20:42)
Headless Mode (5:04)
Handling Pagination (17:05)
Implicit vs Explicit Waits (9:03)
Project #2: Infinite Scrolling & Login (Twitter Bot)
Important Note
How to Login to Websites (11:51)
Building a Function (22:17)
Infinite Scrolling (8:33)
Scraping the Website with Infinite Scrolling (12:28)
Introduction to Scrapy
Section Overview
Scrapy Installation (2:23)
Setting up Scrapy with Pycharm (2:39)
Note: How to Show Hidden Folders on Mac (1:07)
Scrapy's Commands (4:13)
Creating our First Project and Spider (6:00)
Scrapy Templates and How to Find Elements with Scrapy (3:21)
Scrapy's Shell Command (4:08)
Building a Spider (2:59)
Getting links listed in a website (6:19)
Relative vs Absolute Links (5:10)
Scraping Data from Multiple Links (10:53)
Dealing with Multiple Pages using Scrapy
Building the Spider (First Page) (14:24)
Pagination (6:22)
Changing User-Agent (7:03)
Project #3: Building Crawlers with Scrapy
Crawler - Rules (11:25)
Crawler - Pagination (5:10)
Crawler - Changing User Agent (4:30)
Exporting Data to a Database: MongoDB and SQLite
Pipelines (6:00)
MongoDB (9:29)
SQLite3 (9:37)
Scraping APIs & Login to website with Scrapy
Scraping APIs - Building the Spider (11:34)
Scraping APIs - Dealing with Pagination (5:31)
Login to website - Building the Spider (10:04)
Splash Basics
Section Overview
Setting up Splash with Docker (4:30)
Introduction to Splash (6:02)
How to Find Elements with Splash (5:10)
Project #4: Scraping JavaScript websites using Scrapy/Splash
Building the Script in Splash (8:15)
Using Splash with Scrapy (6:45)
Parsing (5:45)
Changing User-Agent (Splash) (4:28)
How to Make Money Using Web Scraping
Section Overview
How I Monetized My Web Scraping Skills Over The Past 2 Years (14:45)
Getting Started - Part 1 (13:20)
Getting Started - Part 2 (20:17)
ChatGPT for Web Scraping
Section Overview
Web Scraping with ChatGPT - Part 1 (11:48)
Web Scraping with ChatGPT - Part 2 (14:57)
Congratulations!
Scraping Data from Multiple Links
Lesson content locked
If you're already enrolled,
you'll need to login
.
Enroll in Course to Unlock