“Since that time [the Dec. 7 attack on Pearl Harbor] the practice of intelligence in the United States has improved greatly, but the opportunities for self-deception are at least as great as ever. Machinery has been built up where there was almost none before. Its effectiveness, however, still depends on the human element. No one has yet found a cure for our tendency to believe what we find most congenial and reject what seems repugnant.” –A. R. Northridge, Central Intelligence Agency
Today marks the 70th anniversary of one of the most tragic days in American history – the Japanese surprise attack on Pearl Harbor. And, as most folks know, the devastation wreaked by the Japanese on that day resulted in part from an information and military intelligence failure on the part of the U.S.
While information gleaned from numerous sources indicated the Japanese were planning some sort of attack – warranting (thankfully) the dispersal of the U.S. Pacific fleet’s valuable aircraft carriers – a variety of missteps left Pearl Harbor unprepared for the dive bombers and attack planes that roared out of that clear-blue Sunday sky seven decades ago.
As the quote above from A. R. Northridge’s article about the intelligence failures leading up to the Pearl Harbor surprise attack indicates, it’s not just about getting one’s hands on the right information at the right time. It’s also about making sure our inherent human flaws don’t obscure the picture data is often trying to paint for us.
This is just as true for businesses as it is for military organizations by the way – something a recent study by global information technology (IT) provider EMC Corp. brought home to me.
EMC’s Data Science Study surveyed 500 data scientists and professionals in the U.S., the United Kingdom, France, Germany, India and China working in disciplines such as data analysts, data specialists, business intelligence analysts, information analysts and data engineers globally, all of whom have IT decision-making authority.
EMC’s poll found that only one-third of the respondents said their companies were able to effectively use new data to assist their business decision-making, gain competitive advantage, drive productivity growth, yield innovation and reveal customer insights.
Some of the study’s other findings include:
• Barriers are growing to data science adoption – The most commonly cited barriers to data science adoption include lack of skills or training (32%), budget/resources (32%), the wrong organizational structure (14%) and lack of tools/technology (10%).
• Customer insights are lacking – Only 38% of business intelligence analysts and data scientists strongly agree that their company uses data to learn more about customers.
• Data accessibility is lacking – Only 12% of business intelligence professionals and 22% of data scientists strongly believe employees have the access to run experiments on data – undermining a company's ability to rapidly test and validate ideas and thus its approach to innovation.
• A talent shortage looms – 65% of data science professionals believe demand for data science talent will outpace the supply over the next 5 years.
None of those trend lines bode well for businesses, noted Andreas Weigend, head of the social data lab at Stanford University and former chief scientist for Amazon.com, in EMC’s survey.
“We live in a data-driven world [and], increasingly, the efficient operation of organizations across sectors relies on the effective use of vast amounts of data,” he explained.
“Making sense of big data is a combination of organizations having the tools, skills and more importantly, the mindset to see data as the new ‘oil’ fueling a company,” Weigend added. “Unfortunately, the technology has evolved faster than the workforce skills to make sense of it and organizations across sectors must adapt to this new reality or perish.”
“The ‘Big Data’ era has arrived in full force, bringing with it an unprecedented opportunity to transform business and the way we work and live,” Jeremy Burton, EMC’s executive vice president and chief marketing officer, pointed out.
“Through the convergence of massive scale-out storage, next-generation analytics and visualization capability, the technology is in place,” he noted. “What's needed to fully realize its value is a vibrant, interconnected, highly-skilled and empowered data science community to reveal relevant trend patterns and uncover new insights hidden within.”
And it’s that human talent, uncluttered by preconceptions and bias, which can effectively filter and make effective use of all the myriad streams of data flowing in and about our world these days – information that can help generate success for businesses large and small, while staving off disaster in the bargain as well.
