He finds the silver lining in stats
Nate Silver detailed his transformation from ordinary statistician to modern-day election soothsayer in front of a USC audience Sept. 20 as part of the Dennis F. and Brooks Holt Distinguished Lecture Series, offered by the USC Price School of Public Policy’s Bedrosian Center on Governance and the Public Enterprise.
Silver runs the award-winning political website FiveThirtyEight.com — referring to the number of votes in the Electoral College — where he correctly predicted the presidential winner in all 50 states during the 2012 election. After three years housing his blog at The New York Times, Silver recently announced that he will be moving it to ESPN, where he said his statistical analysis will be one-third politics, one-third sports and the remainder on a wide range of topics.
“Nate has distinguished himself as today’s leading political analyst and statistician through the innovative and accurate analysis of political polling,” said Price Dean Jack H. Knott. “The accuracy of his predictions, his straightforward approach to using public data and the understandability of it for the average person won him great acclaim here and abroad.”
The Holt Distinguished Lecture Series was established by longtime USC Price Board of Councilors member Dennis Holt, founding chairman and CEO of U.S. International Media, and his wife, Brooks.
“The Holt series is designed to invite prominent practitioners, influential policymakers and visiting scholars to discuss the key issues around the connections and interactions between communications and policy,” said Professor Raphael Bostic, director of the Bedrosian Center.
“Nate Silver definitely falls into that category of bridging communications and policy, and using data evidence to shape how we think about policy and politics,” he added.
Following his one-hour speech at Bovard Auditorium, titled “Baseball and Politics Are Data Driven,” Silver signed copies of his book, The Signal and The Noise: Why Most Predictions Fail — But Some Don’t.
In his talk, Silver made his success predicting election results — he also nailed 49 of 50 states during the 2008 presidential election — sound so easy. He simply averages public polls, adds up the totals and accounts for margin of error.
“Nothing here is all that conceptually difficult,” Silver said. “Nevertheless … it creates a lot of controversy.”
Silver received push back from traditional media on his projections, their growing popularity and the way they were changing how politics are covered, but his results were undeniable.
His rise in prominence has been encouraging to students, who see an individual who became popular for statistical analysis not that different from their academic research.
“He’s very funny and charming, but he’s someone who’s really making a name for himself on the quality of the work he is doing and the accuracy of his predictions,” said Ryan Cassutt, a second-year public policy master’s student at USC Price. “It was so funny to me watching all these slick pundits up there on the news channels last year getting everything so wrong, and there he is with his relatively simple models running circles around them.”
Silver compared the divide between traditional political reporting and his data-driven journalism to the similar conflict in baseball 10 years ago between the old guard of scouts, who relied on intuition, versus a new batch using statistical methods to find undervalued players, as spotlighted in the 2011 film Moneyball.
Prior to bringing his number crunching to the political world, Silver projected player performance for Baseball Prospectus. Silver said that the two sides get along in baseball these days, but that change could take longer in politics because there’s a presidential election only once every four years.
Silver also explained that we are living in a world with a lot more data, but this substantially increases the complexity and volume of relationships to analyze, leading to a lot of spurious correlations that need to be weeded out.
“You have to apply a filter, you have to have a theory, and to apply some scrutiny,” he said. “There are terrific insights in data … but not without some human intelligence at the console.”
Going forward, Silver suggested that those who follow him into data-driven analysis think probabilistically, know where they’re coming from (in other words, don’t be biased), survey the data landscape to identify quantity and quality under a wide variety of conditions, and use simple trial and error.
“I’m far from the best communicator in the world, and I’m far from the best statistician, but that overlap of skills, I think, is what characterizes FiveThirtyEight,” Silver said.
More stories about: Big Data, FOC, Research, Statistics