Why web scrape?
That’s may seem like a silly question, though we have many uses for web scraping, so I will break down by item:
1) Company or personal interests.
2) ML
3) BI
- Company: This first item is obvious too generic, but lets make an imagination exercise. What if every company had the power to scrape properly tons of data from internet, for example: An e-commerce company know exactly how amazon system works, this surely will be a great advantage against its competitors, of course, it depends on sector in which the company operates.
Personal interest: Imagine that you want to buy a cheap flight ticket to London, how do you do to know the best price, in which day or hour will it be available to sell? - With the target/company/website defined, you can build your model with the data and answer that you got, and open the world of ML.
- Why use BI-Business Intelligence after all the work to scrape, build de ML model (or even put it to production) and refine the model? In most cases, of course, if this is not a personal project, you’ll have to show what you did to your boss or whoever you’re trying to convince that your model or idea is good.
So here we have some good options for data visulization, like power bi, looker, tableau, etc.
I recommend power bi, as it has the most prizes in “data viz” competitions.
My web scraping code was based on personal interests, I always liked the idea of investing in some “safe” business models, so I choose franchises.
I will not go into deeper details as this is intend to newers. The part II may cover these details.