Sherri Hammons
Big Data: Humans Requiredmy binary life
Conversation overheard:
Technology: We have built the perfect technology that will answer any question you ask.
Business: Excellent. How can we increase revenue?
Technology: The question needs to be more detailed.
Business: How can we increase revenue for Product A?
Technology: You aren’t getting it. We need more details around what exactly you are asking. We can give you a spreadsheet that outlines how many units shipped month over month, and another around costs associated with the units, and we can even give you another spreadsheet that tells you how many complaints we found on social media sites.
Business: Will that tell me how we can increase revenue for Product A?
Technology: How the hell should I know?
Can Big Data technology actually create answers to business problems without human touch?
I sat in a meeting a few months ago with some amazing leaders in education. They had created a board focused on improving education through data insight. The tech people had been working for months to gather information for the board and on that day had brought in mounds of paper containing information on all the data elements around the state pertaining to education. In all, there were 15,000 pieces of data with names like “dm_Education_Level”, “sm_Ed_Type”.
I watched these leaders suffer through the paperwork, brows furrowed, eyes glazed. I finally asked the question that hung like a pink elephant in the room.
“What do you want to know?”
One educator looked haggardly at me. “Which curricula are working.”
These simple examples outline the heart of the problem with data: interpretation. Data by itself is of little value. It is only when it is interpreted and understood that it begins to become information. GovTech recently wrote an article outlining why search engines will not likely replace actual people in the near future. If it were merely a question of pointing technology at the problem, we could all go home and wait for the Answer to Everything. But, data doesn’t happen that way. Data is very much like a computer: it will do just as it’s told. No more, no less. A human is required to really understand what data makes sense and what doesn’t. But, even then, there are many failed projects.
Why is that?
Some fallacies around Big Data include:
Data Scientists will be the answer
If you read the current thinking about the Data Scientist role, you will probably find a living unicorn before you find a Data Scientist. Skilled in business, language, math, statistics, and computer science, these Super Techno Business Data Geek Scientists could potentially rule the world. If you can find one.
There needs to be a business case
Guess what? There is always a business case. What company doesn’t want to increase revenue, reduce costs, build a better product? What governmental agency doesn’t want to know if their programs and services are making a difference in citizen’s lives? What educational system doesn’t want to know they are creating curriculum that gives their students the right toolset for success?
The data must be clean
It won’t be. Ever. As soon as you clean it up, it gets messy again. Just like your stove.
The business is much too complicated and unique
Not true. Most businesses/government entities are extremely similar. Think of how we are grouping concepts around tags and keywords online. We’re building out communities of information that make sense to everyone. Organically.
If I could go back to the education board, I would turn the conversation around a bit.
Professor: “What curricula are working?”
Me: “What are some keywords that would make sense of the term ‘working’ for the curriculum?”
My guess is we would come up with things like:
Same with the business example.
Business: “What will increase revenue?”
Data Scientist: “What keywords influence increased revenue?”
If this looks a lot like what you learn in Business School, you are right. And your business already has a boatload of these people. A Data Scientist just needs to use these subject matter experts to understand the keywords and consult with them as they find interesting patterns and anomalies. This is where it gets interesting.
Data Scientist: Product A sales dipped last December by 20%.
Business: Yes, it’s because we had that manufacturing problem, remember? But, they rose back up the next month and have remained steady.
Data Scientist: Right, but conditions exist similar to those that caused the manufacturing problem. Might be time to find a new vendor.
Now, that’s the right conversation.
Sherri Hammons 
Sherri, thanks so much for the link. Great insight and thank you for the conversation today. I look forward to a future partnership in sharing ideas.
LikeLike
Sherri – This is a great point; one that I believe can be applied to situations outside of data analysis as well. Conversation, in general, is full of misdirection. Too much content. “Incorrect” questions. Communication is very convoluted, and as such, we are left trying to decipher what the real meat of conversations are, and how to get our own points across without incorrectly (or inefficiently) communicating. This is something I struggle with every day in my efforts to link OIT to our customers, and vice versa. Thanks for bringing this topic to the top.
LikeLike