How to thrive in the age of Big Data by ignoring it altogether

Algorithms. Artificial intelligence (AI). Machine learning.  

Conventional wisdom would say that each of the above requires a massive amount of big data and breathtaking computational power to achieve world-changing outcomes. But a growing chorus in the technology sector, and across industries, suggests that small data may have an even bigger, broader upside, at least in the near future.  

Pop culture’s obsession with artificial intelligence has raised expectations beyond what the technology is currently capable of. Between the science-fiction movies of robots taking over the human population, and the buzzy news stories of AI programs winning at game shows like Jeopardy, the broader public is led to believe that artificial intelligence should, and can, answer all questions and perform all actions. This kind of technology would require enormous amounts of big data. But the reality is far different. The effective AI programs today are those that focus on automating specific tasks and functions—the “range” of actions they can perform is actually very limited.  

Kaveh Waddell writes in Axios, “Today’s best AI algorithms are one-trick ponies that have each been taught an extremely useful, single trick. But that’s not something to sneer at, because these ponies have already created more than a trillion dollars of business value, according to Gartner—with many trillions more up for grabs for those who can figure out how to apply the technology.” 

In the Gartner report cited by Waddell, John-David Lovelock, research vice president, said, “One of the biggest aggregate sources for AI-enhanced products and services acquired by enterprises between 2017 and 2022 will be niche solutions that address one need very well.” 

Big data versus small data  

Data scientists say big data must have the three Vs: Variety, volume, and velocity of processing. Small data is defined as being accessible, actionable, and information-rich. 

The thing is, while we’re more familiar with the term big data—it’s now a buzzword overused in the context of business and technology—small data is the far more effective predecessor. Long before big data was gracing the headlines as the enabler of powerful technology, small data was instrumental in the advancement of humanity.  

“Everything was small data before we had big data,” as Margaretta Colangelo, partner at Deep Knowledge Ventures, reminds us in a recent article published in the MIT Technology Review. “The scientific discoveries of the 19th and 20th centuries were all made using small data. Darwin used small data. Physicists made all calculations by hand, thus exclusively using small data. And yet, they discovered the most beautiful and most fundamental laws of nature.” 

It was small data that showed us that the sky was no longer the limit. The Apollo team put humankind on the moon, essentially with blackboards and slide rules—i.e., small data. There is probably more computational horsepower in a smartwatch than NASA had for the entirety of the Apollo program.  

Despite the power and storied success of small data, it’s the concept of big data that we see and hear about the most. A Google Trends search shows the immediate and drastic spike in big data’s popularity. Meanwhile, small data has garnered much less attention but has arguably played a more significant role in the world of technology and business. 

Unlocking the power of small data . . . today 

Who’s putting small data to work right now? You are. It’s your train schedule, your Excel spreadsheet, your electric bill. All small, single-purpose data. Your local post office uses it too. Praful Sakani, writing for the Harvard Business Review, asks us to “consider how the United States Postal Service [USPS] automates mail sorting. With the help of machines and advanced optical recognition [OCR] technology, USPS can process upwards of 36,000 pieces of mail per hour.”  

Data-driven marketers are seeing a huge upside in the application of small data. Ahmed Banafa, writing for DataFloq, said, “Small data is at the center of the new Customer Relationship Management (CRM): Social CRM used to create a complete picture of customers, their segments, influencers, and even competitors; we need to combine insights from social channels and campaigns with Web analytics and transactional data. Small data is the key to building these rich profiles that will be the center of the new CRM solutions.”

Customization and personalization—where else could those capabilities pay dividends?  

In a piece called The Power of Small Data in Education, Trenton Goble wrote, “Kids aren’t data points, or trends, or objectives; they’re individuals with unique strengths, weaknesses and interests, and they deserve individual support. The conversation we need to be having is around the role that technology can play in empowering teachers to bring more personalized instruction to each and every student in the classroom. That’s where the battle for the hearts and minds of our kids is being fought.” 

The healthcare arena has something of a different take. Writing for Information Week, Larry Stofko says both big data and small data play important roles. “Big data is a big deal for healthcare. It will help uncover correlations that can lead to cures and treatments for disease. However, small data is important, too–information from individuals can ultimately contribute to big data and lead to important discoveries.” 

He continues with a prediction: “The company that can save patients and healthcare providers money while also improving patients’ health will be king: Billions of dollars are at stake if we can connect the dots between the small data that can maximize an individual’s personal care and the big data that can uncover solutions that can have a global impact.” 

Less is more when it comes to data

Pundits across industries have long been obsessed with deciphering and analyzing the vast amounts of big data they harvest. And even with all the data already at their disposal, they’re continually hunting for more. This was the case even before big data was a well-known concept.    

But this obsession with harvesting and analyzing more could be what’s holding businesses back. There comes a time when “more” doesn’t automatically mean better. There needs to be a point when businesses start putting to work what they have readily available to them. Often, that’s small data, not big data. In addition, experts will tell you that super-human intelligence via the power of big data is still too far in the future to be applicable today.   

Small data, and the tiny clues and pieces of information it reveals about customers, businesses, or industries may be the answer to navigating an increasingly unpredictable marketplace. From marketing to education to healthcare—and many more industries across the globe—small data is making a huge impact right now.