Block 136: Selenium: Form Filling & Data Extraction
Use Selenium to interact with forms and dynamic content.
Concepts
- send_keys() to type in fields
- select element handling with Select class
- Extracting text from elements after JavaScript renders
- Switching between windows and frames
Code Examples
See exercise below.
Exercise
Automate filling in a search form, submitting, and extracting the first 5 results. Loop through multiple pages of results and collect all titles.
Homework
When is Selenium necessary vs BeautifulSoup? What does 'dynamic content' mean in this context? Thursday