Datamartist gives you data profiling and data transformation in one easy to use visual tool.

« | »

The top 3 reasons to automate your data preparation

Robot and Graph 800 wideAlmost everyone has “that data crunching” that they do once a month, or once a quarter or (these are often the really brutal ones) once a year.

You go get the raw data, then you painfully, step by step, work your way through it- copy paste, cut and move, formula after formula in your spreadsheet until you get the report that everyone knows you deliver every period.

Every time you clean up the same mess, you do the same steps (perhaps with a tiny variation, but fundamentally the core work is repeated again and again).

There are three BIG reasons that you should automate

  1. You need to spend less time doing mindless work. It’s painful.
  2. Manual steps create errors, errors create pain, pain leads to suffering… via just plain wrong business decisions.
  3. You are leaving huge piles of money on the table because your analysis cycle is too slow.

Big Piles of Money?

Ok, so 1 and 2 make sense- no one likes doing mindless work and its a waste of your time and means you can’t be doing interesting analysis, and errors are a big problem- garbage in garbage out.

But whats this talk of piles of money??

One of the things we sometimes don’t think about when we are doing things manually is that because they are manual, how often we do them is reduced.

That report that takes 2 hours to do? You can’t do that every day- you’d spend 25% of your time on it. So you do it once a month.

That analysis of product performance that takes 3 days of number crunching and copy-paste? Only once a quarter.

That in depth analysis that actually cross checks customer profitability- that takes takes a week- so we do it just once a year.

But think about it- if you do the analysis once a quarter and find something that would improve results- then you have lost up to 3 months of the new improved results! Even if its only a few percentage points- whats 5% for 3 months? Its likely a pile of money.

But you say- automation is hard- I have to wait for the IT department and by the time they get back to me the problem has changed. Its impossible.

Its not impossible- you might just be using the wrong tool. Spreadsheets are fine- but unless you are a programmer they don’t really automate all that well.

But with a visual data preparation tool that lets you draw your data transformations, then run them with a single click, it is possible for an analyst to automate their own reporting.

It takes a while to learn it, it takes longer to automate something than to do it once manually too, but think about it- say it takes 3 times as long to automate something than do it manually. Say you spend a day on something now, and it will take 3 days to automate it. That means if you are doing that thing monthly now, in 3 months you’ll be ahead- AND you’ll have that report every week if you want it- or every day. Not just every quarter.

Invest in automation. Invest in tools beyond the humble spreadsheet. It will make you a better analyst, and deliver results to the bottom line.

Every day you keep manually preparing everything, you make your lost time bigger and bigger. Every day you wait before doing the analysis because you can’t do it frequently enough, an optimisation is sitting there in the raw data, ready to be discovered, but not being taken advantage of.

When spreadsheets were a new technology, the manual cut and paste approach was state of the art. Now, new visual block and connector tools let analysts automate things quickly- without the need for technical resources.


« | »

Leave a Response