If there is a silver lining to Superstorm Sandy, it is that the issue of climate change and its impact on extreme weather and human health are back on the radar screen. Elected leaders like Governor Cuomo and Mayor Bloomberg, the scientific community and civic leaders are grappling with the big issues, which range from how to manage storm surge to cutting the pollution that causes the problem.
However, there are still those who deny the problem exists. I’ve heard many theories about the cause of this widespread denial, and there’s one in particular that I find especially compelling.
Big data is a wonky term that describes the collection of large, complicated data sets that are extremely challenging to process. More and more, big data is being used to supplant human intuition. The book and film Moneyball show how rigorous statistical analyses were used to displace the collective wisdom of the scouting system in baseball. The use of sabermetrics, while initially an outlier, is now a mainstream part of the sport.
Enter Nate Silver and other analysts who created models that aggregated state and national polls to make projections over the past several election cycles. As Forbes blogger John McQuaid described it, “Silver’s model showed a political, pundit and media class at a classic horse-and-buggy meets internal combustion engine moment.”
Just like the confused and angry scouts of Moneyball, Peggy Noonan’s pre-election column stated, “No one knows what will happen.” In fact, we had some pretty solid projections as to what was to come—it was just that many of the pundits refused to believe the numbers.
Another key area where big data matters is climate models, which use quantitative methods to simulate the interactions of oceans, atmosphere, land, and cryosphere (ice-covered areas). These models help predict temperature changes and the future of the climate. And guess what? The numbers provide solid proof that our climate is changing. There are many other strands of evidence—from temperature change to sea level rise—which all point to the same conclusion.
There’s a lot of skepticism about climate change—especially when intuition or direct observation of a single phenomena (like a snowfall) takes place. These skeptics, like those in the previous examples, fail to realize that the climate models take into account huge amounts of data and find trends that might otherwise be biased by a single experience or local bias.
Although it may feel uncomfortable in its early applications, it is time to embrace the analytical advantages of big data and to be the first generation to use it to solve the world’s most pressing problems—not just boost the performance of our sports teams.