Why your data analysis is not at all 'analytical'​ and what to do about it

Author: Jonny Longden from CRO Agency Journey Further

This is, today, the unfortunate reality of the vast majority of what goes by the name 'data analysis:

Business person: Can you give me a report on XYZ?

Analyst: Sure, here you go

Business person: hmm, can you add XYZ and break it down by ABC?

Analyst: Sure, here you go

Business person: Interesting, can you now add blah blah blah...

Ad infinitum!

The business person is ultimately trying to solve a business problem or answer a question, but they lack sufficient skills with data and so believe they are utilising an 'analyst' to help solve their problem.

Make no mistake there is no 'analysis' going on in this situation at all. All that's happening here is someone looking at data in a highly inefficient way using an analyst as a kind of machine to help them extract the data in a certain way.

It's highly unlikely that whatever question the business person had is going to be answered satisfactorily via this method, and they may end up thinking that the analyst is not a very good one because they haven't been able to give them the data in the right way, but in reality, neither of them are approaching the problem analytically.

So what has gone wrong and how can you put real analytics into the heart of your decision-making?

The true definition of analytics

The origin of the word 'analysis', going back to its ancient Greek foundations, means to loosen, break up or set free. What this relates to, for example in the works of Aristotle, is the resolution of a complex problem, via the use of logic, by specifically breaking down and dissecting the issue into more simple components.

Today the more common term for this mental process is critical thinking, and because the word analysis has also come to be associated with data, a proper definition might look something like this:

Critical thinking employed to solve business problems, supported by the interrogation and interpretation of data.

Now here's the problem: the business world seems to have decided that analysis means, quite simply, the interrogation of data. A person with the job title analyst who slices and dices data is more than enough for a business to consider itself as doing analytics.

What they have completely overlooked though, in many cases, is both the critical thinking and the business problem. The interrogation of data is, in many ways, the least important part of the whole process, as it is really just a tool that ought to support the logical process.

Where analytics goes astray

What people think of incorrectly as analytics specifically goes astray in a number of ways:

Failure to properly articulate the business problem

In the above example, the business person will no doubt have a latent awareness of the problem they are trying to solve, but they have not formulated this problem or articulated it in a way that guides what they need to do next. The proper formulation of the right problem is a strategic and even philosophical task that requires business acumen and knowledge of the business vision.

Failure to communicate and collaborate on the business problem

Despite the fact that the business problem has not been formulated, it is not communicated at all. In my example, the business person asks the analyst for data without even remotely explaining what it's for. The analyst does not ask for this information.

One of the many issues this causes is extreme inefficiency. The business person doesn't understand the data and so is using trial and error to try and get something meaningful from it. Had they just explained the business problem, the analyst would have thought about how best to present the data and pulled that the first time. In a big organisation, this practice can waste a monumental amount of resources and money.

Diving straight into the data

If the starting point of your approach is diving into your data, then the only answer you can possibly come up with will be derived from that specific data. But what if you don't have the data required? If you start out with a formulation of the business problem and critical thinking about the approach to a solution, you may realise that you need to go and source the data you need to support that process. But by starting with the data you are skewing your answer to only one possible range of answers and also pre-determining a potential solution based on something completely arbitrary (the data you happen to have).

Lack of critical thinking

You can't conduct critical, analytical thinking without data. Analytics is critical thinking, but how many people doing analysis today could really claim to be applying critical thinking skills to solving business problems? Breaking down a problem into component parts; identification of lines of enquiry; design of analyses and experiments; recognition and avoidance of cognitive bias; etc - is this really happening or is it just the production of reports?

It's impossible to even look at a piece of data without interpretation being involved. Critical thinking is the structured process of interpretation; without it, you are allowing your mind uncontrolled freedom to interpret in highly unstructured ways.

How to regain the true meaning of analytics for your business

Whilst it is not my intention within this post to teach the art of critical thinking, there are some concrete things you can do to try and put these valuable skills into the heart of your business processes and the way people make decisions.

1. Take the time to properly formulate your business problems

There are only a finite number of crucial, macro business problems your business is trying to solve at any one time. Most of the smaller questions people have are just component parts of a bigger problem. Taking the time to understand this strategically not only helps guide data analysis and create real solutions, but it also organises and aligns your entire business around what is important.

This article I wrote explains a particular process of aligning performance to profit strategy for eCommerce businesses. It's a good example of both a piece of critical thinking but also the way business problems can be identified and formulated.

2. Train your business in the art of critical thinking

In the example I cited at the beginning of this article, the outcome would have been incredibly different if both the business person and the analyst had been trained in the art of critical thinking and collaborated on the process.

It's not particularly difficult to do critical thinking and mostly the reason people don't do it is they don't really know what it is, so formal training can go a long way without being too onerous.

3. Democratise data and create data literacy in your business units

Many business people are already reasonably adept in critical thinking, whether they know it or not. The issue these people come up against is that they don't understand data well enough to be able to utilise it effectively in their critical thinking.

Whether or not you want to actually put data into the hands of your business users, or just create better working relationships between them and your analysts, it starts with the business users having a better appreciation of data. This means that they understand what data you have but also the methods that can be employed to interrogate that data.

4. Ensure your analyst have commercial- and business acumen and are involved in the business

It's not uncommon to find analytics or data-science teams within businesses full of incredibly intelligent and technically-proficient people, but who are nevertheless alienated from the wider business.

If analytics as a function is to be responsible for analytics in the true sense of my definition, then the senior people who lead that function need to also be central to the wider strategy and running of the business.

They need to be business partners and consultants. This means a) hiring the right people who are capable of that and b) ensuring a culture that allows this to thrive.

5. Consider the role of a critical thinker

I'm not suggesting you should have someone with the job title critical thinker, but it is nevertheless worth considering creating a role that performs this and only this function.

Another way to think of this is what you might call a data translator (also not meant to be a job title).

The difficult part of getting analytics right is that, on the one hand, you have business people who are not comfortable in the world of data and, on the other, technical analysts who are not comfortable in the world of business. Furthermore, these are generally two very different types of individuals, with different temperaments and world views, who often find it hard to communicate.

A data translator is, conceptually, a person who can straddle both of these worlds comfortably without necessarily being embedded in either, and who performs the critical thinking aspect of analytics by understanding both how to dissect and break down the business problem, and how to brief what is required to data teams in order to get the best output.

This is in fact what I have done for much of my career.

Summary

What people do when they think they are doing analytics is often not actually analytical at all.

They misunderstand analytics as meaning the interrogation of data, which is only actually one part of the picture.

The main task of analytics is critical thinking focused on the resolution of business problems by dissecting the problem into component parts that can be treated differently and brought back together to answer the main question.

These skills and also the culture which respects and allows them to flourish, are lacking in many businesses.

In order to be analytical you must focus on bringing critical thinking into your organisation.

About me:

My name is Jonny Longden, Digital Experience Director at Journey Further. I help businesses become data-driven, and to put research and experimentation at the heart of everything they do.

Over the last 13 years, I have helped the likes of Sky, Visa, Nike, O2, Mvideo, Principal Hotels and Nokia put analytics and experimentation at the heart of their digital innovation and engineering. Doing a 'bit of CRO' is not enough - every decision you make about development without experimentation has a 90% chance of wasting your money. The time is right to embrace experimentation and I can help you get there.

If you don't already, please follow me on LinkedIn to see more content such as this.