There was an excellent story by Steve Lohr in last Sunday’s New York Times on “The Age of Big Data.” Many of you have probably heard about the term “big data,” which the article describes as ”…a shorthand for advancing trends in technology that open the door to a new approach to understanding the world and making decisions.” According to Lohr, data is growing at 50% per year, more than doubling every 2 years.
The article goes on to describe how the nexus of new technology (e.g. artificial intelligence, data collection/storage and processing speed) and the exploding amount of data available from all imaginable sources, have created an environment whereby business, government, the arts and science can distill new insights from this tsunami of information.
One of the examples offered in the article is the now-familiar story of “Moneyball.” In the book by Michael Lewis, and the recent film starring Brad Pitt, the story described how a professional baseball team general manager used new data sources and statistical analysis to identify undervalued players and built a winning team at a relatively low cost.
This reminded me of the story written last year by Oliver Wyman’s David Hunt, and published in the Fall Newsletter of INFORMS Railway Applications Section, entitled “Big Data and Railroad Analytics.” In his article, David also used Moneyball as an example of the use of big data to analyze new metrics to dramatically improve the performance of an organization. He went on to to describe how the North American Rail industry is just starting to use the treasure trove of data that they dynamically collect, to improve the analysis of their business, lower operating costs and produce greater profitability. David also described the use of the new Traffic Flow Analyzer (TFA), that has been developed to manage the analysis of massive railroad databases. If you missed the article, you can find a copy of it on this blog site.
It looks like big data is here to stay.






