The 80/20 rule also known as the Pareto Principle states that approximately 80% of the effort comes from 20% of the causes. The name comes from the economist Vilfredo Pareto who showed that about 80% of the land in Italy was owned by 20% of the population. In data science this translates to 80% of the work data scientist do is related to preparing data while only 20% of their time is used on analysis.


Why is this such a problem?

Well with the growing need for data scientist they are becoming more scarce and their time more valuable. If every week they must spend four out of every five days preparing data and not finding actionable insights, then that is a massive waste of time and money. The reason you want to build data services into a business is to gain a return on investment. From my perspective enjoyment for the data scientist comes from the analysis not the tedious cleaning of data.


My experiences of the this.

We have been having many different instructors to come in over our first five weeks of learning and one thing that is consistent between them is the 80/20 rule, that we will spend most of the time preparing the data in Alteryx and then quickly push out dashboards in Tableau. While these tools have made ratio between the two more even it is still heavily in the favor of data processing.


Our first Client Project

With our first client project we experienced this. We got our data Monday and went straight into exploring it and trying to cleanse it into a format that could easily be represented in Tableau. This was no easy task, we came up against many problems along the way. Even when we thought we were done we would find a new problem while using it in Tableau. So, come Thursday we have finally done all the data cleaning and barely started our dashboards for the final presentation on Friday.  Everyone in a bit of a panic about how we are going to get soo much done in such a short period of time. But with a combine effort we all worked hard and got everything done with a few only a bugs in our final dashboards.


Summing up

While being told all about the 80/20 rule from many different sources it was hard to imagine till I experienced it for myself. Having not much to show for all the work till the very end is a very daunting problem to manage, but becoming part of this industry I will have to get used to it and I am sure I will through the many client projects to come in the rest of the training