Advanced API Web Scraping in UiPath
Intro When we do web scraping, we can take advantage of hidden APIs. For example, many companies load JSON or XML from their backend to their website via API calls. In this use case, we’ll find the hidden APIs and use the information in them. Learning Objectives Using exposed API’s to scrape the web with […]
Table of Contents
Intro
When we do web scraping, we can take advantage of hidden APIs. For example, many companies load JSON or XML from their backend to their website via API calls. In this use case, we’ll find the hidden APIs and use the information in them.
Learning Objectives
- Using exposed API’s to scrape the web with UiPath
- How to do API calls/HTTP requests
- Interpret the results
- JSON parsing
- Advanced JSON parsing
Description
- Download the Excel data file
- For each of the players on the sheet “3. Hidden APIs” get all their stats from nhl.com
Hints
- Navigate to https://www.nhl.com/player and do a search for e.g. Sidney Crosby
- Open the Developer Tools (F12)
- Refresh the page and look at Network
- Find the API call with the player stats (look at the responses)
- Use that endpoint URL to create HTTP Requests in UiPath and interpret the result with JSON parsing