Big data is a buzzword that is ambiguous and often misunderstood. But it also feels like it could be the next big thing. To open up my perspective I found myself stepping outside the echo chamber of Silicon Valley. I enrolled in a class at the Aspen Institute called Socrates. They pull together a group of around 20 people from diverse backgrounds to spend a couple days digging into a topic and sharing their experiences.
No one in my group was a specialist in big data – we had a person who worked for a family office, a mayor’s office, an investment bank, a major consulting firm, an online advertising company, a hospital, the DEA, and of course I was there as the token enterprise software guy. Prior to the seminar, we had all studied a set of fascinating readings that ranged from the story of Socrates struggling with the advent of the written word, to the implications of a world where every detail is known.
There were two general directions the conversation went. One centered on the negative externalities brought on by big data. Things like information overload, dependency on technology, and privacy risks. The other direction focused on the incredible opportunities that big data was opening up, like:
1. Autonomous, Ubiquitous Presence
iwatchAll kinds of services are working for us, even when we’re not thinking about them. Your LinkedIn profile is generating value by connecting you with career opportunities. Having Siri on your wrist means you’ve got an assistant listening for your cues and monitoring your health. And for businesses, it is all about listening to your customers so you can better anticipate their needs.
Transforming Chaotic Environments into a Self-Healing Society
the future, problems will be predicted and resolved before we even think about them. Self-driving cars will navigate around traffic. The airline industry will adjust bookings and communications based on weather and flight delays. And many business transactions will move from console transactions typed into a computer to programmatic transactions, which involve weighing many signals before automatically optimizing for good outcomes.
3. Deadweight Loss Redistribution
There are massive inefficiencies we can eliminate by listening to data, which can in turn free up and repurpose energy for the most critical endeavors. As an example, 30-50% of water and electricity use in cities could be cut with big data projects. And since 50% of crime takes place on less than 5% of street segments, it’s relatively easy to reduce crime with predictive policing.
For me, this discussion clarified that big data is about way more than just the exponential growth of sites like Twitter or the 1.3 billion transactions logged into Salesforce each day. Those apps just capture the perception we actively want to create in a small snapshot of time. The really big data is the product of passive listening and actual behavior. It’s data from the web, your credit card transactions, your mobile phone, and your medical records. It is going to come from a new breed of connected, wearable devices that monitor our environments.
Our cars, our home thermostats, and our clothing are all becoming “aware.” These devices are able to measure location, speed, sound, temperature, light, vital signs, and any number of other inputs from our environment. With all the new sources of always-on information pouring in, there’s clearly an immense amount of data being collected. The questions become:
How will we make sense of it all?
What are big data’s practical applications?
Who is going to emerge as winners and losers?
And ultimately how will it change society?
As I’ve pondered these observations and questions in the context of my work in the enterprise software arena, I’ve identified three key takeaways for the average business:
1. Get on Board
Big data is a mega-trend every bit as powerful as the alphabet, the printing press, or the Internet itself. It will inevitably transform the competitive landscape, and it will happen faster than you think. It took only 2.5 years to sell 100 million iPads, so how long will it take before 100 million people are wearing an Apple iWatch or Google Glass? And how long before your industry feels the effects of the shift from console transactions to programmatic transactions?
2. We’re in an Arms Race Powered by Data
To remain competitive you need to be relentless in your pursuit of data. Is there data you can collect about your prospects, customers, products or employees that’ll give you an edge over the competition? And just as important, do you have a real-time framework to turn that data into action, so you’re able to programmatically increase the probability of good outcomes for your business, while decreasing bad outcomes?
3. Partner with Vendors Who Understand Predictive
While some companies have the resources to collect big data and build their own proprietary applications, it is likely that best-of-breed vendors have better end-to-end solutions. We’ve already seen this for A/B Testing, Search Optimization, Display Retargeting, Lead Scoring, and Product Adoption. In each of these categories, companies can ride the big data wave without having to hire a single data scientist.
We’re living in an exciting time, especially for those who can embrace being awash in data. We need to find ways to profit from it rather than drown in it. By eliminating the deadweight in your business, you can shift energy to more productive initiatives and blow past the competition. Being a data-driven business means more quickly anticipating customer needs, and finding new ways to meet them so you can guarantee the most profitable outcomes.
So whether we like the term big data or not, there is little doubt that it will fundamentally change how businesses operate.
