What is Web Scraping?
When you think of web scraping, you may think of a person in a dark room with a hoodie on frantically clicking while they ravage the internet for all its data. This may be the picture some of the time. The reality is, if data is publicly available from URL found on a search engine, then there’s nothing shady actually going on. Web Scraping is simply taking the data from a page and transforming it into data which can be used for a specific purpose. In this 2-part blog, I will be walking you through the steps to understand Web HTML from a website (Part 1) so it can be scraped in Alteryx (part 2).
Inspecting your website
For this example, we are going to be using a website with football/soccer results from the last 10 season in England https://footballrates.com/lasttenseasons. The first thing you need to do before evening opening Alteryx is to inspect the website structure aka inspect elements. On most computers, you can right-click and select inspect to open this inspect element pane. For window the shortcut is CTRL+SHIFT+C and for Mac its CMD+OPT+I. Once this pane is open you will be able to see the html structure of the website in the top half of the pane. This is where you will find the elements you need to use to scrape the data.
At first, this may seem daunting and for some, this may be their preferred method of learning, sifting through the code to find the elements they need. For me, I like to use the hover feature. This is that icon you can see in the little red box I’ve drawn in the Image to the left. What this allows you to do is hover over an item on the actual page and it takes you straight to the point in the html where this item is- as seen in the example below.
What this allows you to do is find the structure of the table. Which is generally wrapped in ‘<Table> (Table Content) </Table>’. This is the part which will be parsed later when we download the data. The second element you’re looking for is your rows which is expressed with ‘<tr> (Row Content) </tr>’. This is where you’ll tokenize your table into rows. The last element you need is your columns which is expressed with ‘<td> (Column Content) </td>. For this example, we’re actually able to tokenize the data from the headers at the same time as our column data so the structure will be slightly different. The structure of this table can be seen below.
Now that you have found the HTML elements you need, you will be able to scrape the data using Alteryx.
In the next part I will be walking through the tools you will need to build your workflow and what your outputs should be looking like.
You can find part 2 Here.
