Preamble

As I’m writing this I’m sitting outside the Powercor offices waiting for the pizzas I ordered for my team (graciously paid for by MIP on account of the “work late on project week Thursdays, dinner’s on us” rule). It’s 6:50pm and the team’s still going strong.

Without going into too much detail, we’ve been working all week to provide Powercor with functionality they’ve never had before; the ability to go back in time. I want you to leave this blog post with a handful of key takeaways you can use in your next data project. This was my first experience leading a project like this and to say I learned a lot would be an understatement.

12 Key Takeaways

  1. Set Out Time for Sensemaking as a Team; Ideally a Day.
  2. Know Who Your Stakeholders Are, Be Crystal Clear About What They Want and Talk to Them.
  3. “Clean” Data Does Not Imply Ease of Understanding.
  4. Split Your Team into Smaller Units and Assign Portfolios
  5. Bring in any Outside Knowledge Your Team Has; Put it to Work.
  6. Take Care of Your Team; They’ll Take Care of You.
  7. Performance is a Real Thing Outside the Training Room.
  8. Your Insights are Valuable, Your Perspective Will Be Fresh; Back Yourself.
  9. Beware Duplication of Work.
  10. Ignore the Previous Rule; Sometimes.
  11. Work in Parallel as Much as Possible.
  12. Set Time Aside for Preparing Your Presentation; Then Double It.

1. Set Out Time for Sensemaking as a Team; Ideally a Day.

    • We received the data at about 10am on Monday morning, although until 4:30pm we had no idea what we were looking at, or if the data was even correct. There were multiple columns that seemingly had the same data, and it took many back and forth conversations with our contact at Powercor to make the numbers work. While this was happening progress had completely stalled, and a feeling of dread was falling over the team. However, we were all looking into and pouring over the data and eventually Stuart found the golden combination that worked, and we immediately felt better.
    • For a week long project, spend the first day on the data, put together exploratory visualisations, ask questions, sanity check the numbers, and find out what’s possible together, as a team. The gains you make in collective understanding will be invaluable as you progress through the week, and is easily worth the time investment at the start.

2. Know Who Your Stakeholders Are, Be Crystal Clear About What They Want and Talk to Them.

    • Everyone’s opinion is important, but in a client project there are some voices in the room that hold more weight. We had multiple visits over the weeks from some of the higher ups who took a great interest in what we were doing. In the end, it was those people that provided the most important feedback on what we were doing. We asked them as many questions as we could, and showed them what we were working on. What they had to say was golden and made sure that we were on the right track come presentation time; at the end of the day, it’s them that will use our dashboards to make more informed decisions in their business.

3. “Clean” Data Does Not Imply Ease of Understanding.

    • At the start of the week we were promised the holy grail of analytics; “clean data.” I wouldn’t go so far as to say that was a lie—the data was after all quite clean—but we had to pour hours into understanding what the columns referred to, and even how the related to each other. Making sense of clean data can still be a messy process and it almost derailed the project right at the beginning.

4. Split Your Team into Smaller Units and Assign Portfolios

    • I chose to split our team of seven (six + me) into two groups of three; a data team, and a viz team. The viz team would be in charge of ideating, and creating concepts and a skeleton dashboard, liaising with the data team wherever necessary. The data team would continue the process of understanding the data, and began to cut and combine the data to create insights.
    • The data team was also charged with collating an “Insight Inventory” which became a valuable reference during the week, and the basis for assigning tasks to individuals. The inventory was a numbered list of each of the insights that we thought were applicable to the problems we were trying to solve. Then we met as a team to rate each one on the MoSCoW scale; focussing on Must-have, Should-have, and Could-have. Once we triaged these ideas together, I assigned each one as a work item on the SCRUM board, and the team got to work.

5. Bring in any Outside Knowledge Your Team Has; Put it to Work.

    • We have a diverse team, bringing a myriad of experience into the room. Powercor brought us in as “a shot of adrenaline to the team,” a crack team of analysts to inject new ideas and possibilities to inspire them. Regardless of our experience in the power industry, and in data analytics in general, we were able to leave them with valuable insights that were actionable right from the get go.
    • I personally spent most of the week working on a collection of Custom SQL Scripts leveraging the native calculation ability of SAP HANA’s SQL interface. Because we were working on a table with roughly ~75 million rows, bringing that into Alteryx, or Tableau Prep wasn’t a viable option. Using some creative SQL queries, I was able to run calculations that took seconds, read each of those 75 million rows, and then only sent back the data that we needed.

6. Take Care of Your Team; They’ll Take Care of You.

    • My colleagues in Cohort 17 are the best group of people that I’ve ever worked with. I don’t believe that the week would’ve run so smoothly (relatively) or been so successful if I didn’t have such capable people in the team. We were able to deliver what they asked for early, and we didn’t stop creating insights.
    • If I could offer any advice to a prospective Project Manager; make yourself available to your team, and check in with them as often as they’ll let you. The confidence that you can instill in your team will take them further than you can imagine. Make sure they’re watered, fed, caffeinated, and well rested at all times.

7. Split Your Team into Smaller Units and Assign Portfolios

    • I chose to split our team of seven (six + me) into two groups of three; a data team, and a viz team. The viz team would be in charge of ideating, and creating concepts and a skeleton dashboard, liaising with the data team wherever necessary. The data team would continue the process of understanding the data, and began to cut and combine the data to create insights.
    • The data team was also charged with collating an “Insight Inventory” which became a valuable reference during the week, and the basis for assigning tasks to individuals. The inventory was a numbered list of each of the insights that we thought were applicable to the problems we were trying to solve. Then we met as a team to rate each one on the MoSCoW scale; focussing on Must-have, Should-have, and Could-have. Once we triaged these ideas together, I assigned each one as a work item on the SCRUM board, and the team got to work.

8. Bring in any Outside Knowledge Your Team Has; Put it to Work.

      • We have a diverse team, bringing a myriad of experience into the room. Powercor brought us in as “a shot of adrenaline to the team,” a crack team of analysts to inject new ideas and possibilities to inspire them. Regardless of our experience in the power industry, and in data analytics in general, we were able to leave them with valuable insights that were actionable right from the get go.
      • I personally spent most of the week working on a collection of Custom SQL Scripts leveraging the native calculation ability of SAP HANA’s SQL interface. Because we were working on a table with roughly ~75 million rows, bringing that into Alteryx, or Tableau Prep wasn’t a viable option. Using some creative SQL queries, I was able to run calculations that took seconds, read each of those 75 million rows, and then only sent back the data that we needed.

9. Take Care of Your Team; They’ll Take Care of You.

    • My colleagues in Cohort 17 are the best group of people that I’ve ever worked with. I don’t believe that the week would’ve run so smoothly (relatively) or been so successful if I didn’t have such capable people in the team. We were able to deliver what they asked for early, and we didn’t stop creating insights.
    • If I could offer any advice to a prospective Project Manager; make yourself available to your team, and check in with them as often as they’ll let you. The confidence that you can instill in your team will take them further than you can imagine. Make sure they’re watered, fed, caffeinated, and well rested at all times.

10. Ignore the Previous Rule; Sometimes.

    • However, if you’re working on a difficult problem, with several approaches to solving it; especially if one of those solutions is experimental and untested, it might be a good idea to assign it to multiple people. In my case, I was working on SQL scripts that I believed in, but wasn’t certain that I could make them work. I had my colleague Key work on an Alteryx workflow that mimiced the functionality I was trying to achieve. The scripts ended up working, and they were relatively performant, but had I put all my eggs in one basket and it didn’t work, then I would’ve wasted a lot of peoples time, instead of just Key’s.

11. Work in Parallel as Much as Possible.

    • You have to tackle a big project one bit at a time, and without parallelisation of the workload, progress would’ve been slow. By splitting the group into two portfolios—a data team, and a viz team—everyone had something that they could work on individually or amongst their mini-team. We were able to deliver so much more to the client because we had successfully distributed the workload amongst each member of the team.

12. Set Time Aside for Preparing Your Presentation; Then Double It.

    • I’d say that we put together a good presentation, but not a great one. The story telling was mostly missing, and we were underprepared to present our work to the stakeholders. Honestly, we had a rehearsal booked that would’ve been invaluable, but I kept pushing it back and eventually we just ran out of time. If I had another shot at running this project, I would’ve liked us to spend the whole of Friday just putting together the presentation. As it was, we were essentially putting together our dashboards right up until, and a little after, the final siren.
    • The presentation is what most people will see, they might not have the context of the week that was, they might not know what valuable insight you’ve got tucked in those sheets, unless you tell them about it. Without storytelling, and the right preparations, you can end up leaving a lot on the table. Luckily, we had a friendly crowd, and they were so happy with what we put together; and I couldn’t be more proud of the team.

There you go, 12 key takeaways from my first experience as a Data Project Lead. I’m proud of the team, and I’m proud of myself. We did what we set out to do, and we left nothing out on the field. I learned so much about myself in this exercise, not only about SQL, but about being a good project manager. It’s a skill I’m really looking forward to building upon, and I’m confident that be able to tackle the next project I run even more successfully.

Thanks for taking this journey with me, and making it to the end of this retrospective. I hope you were able to extract a couple of gems to spruce up your next project.

With Love,
Dan

Daniel Lawson
Author: Daniel Lawson

Right off the bat I can tell you that I’m not your average data analyst. I’ve spent most of my career running my own business as a photographer and videographer, with a sprinkling of Web Development and SEO work as well. My approach to life and work is very T-shaped, in that I have a small set of specific skills complemented by a very broad range of interests; I like to think of myself as a dedicated non-specialist. Data Analytics, and Programming, started as a hobby that quickly grew into a passion. The more I learned the more I looked for opportunities to pull, manipulate, and join data from disparate sources in my life. I learned to interact with REST APIs for services I used, personal data from services I use like Spotify, and health data captured by my devices. I learned SQL to create and query databases, as well as analyse SQLite files containing my iMessages and Photos data on my Mac. Every technique I learned opened up more possibilities; now I’m hooked and there’s no turning back. Learn More About Me: https://danlsn.com.au