Big Data Analysis Tool Predicts Republican Debate Winners, Losers

NEWS ANALYSIS: A big data analysis tool from ICG Solutions trains the big guns of the intelligence community on Twitter as a way to see who the winners and losers were in the Sept. 16 Republican debate.

Download the authoritative guide: Big Data: Mining Data for Revenue

Big Data Debate 2

David Waldrop and I were sitting in the National Press Club's legendary Truman Lounge watching two huge television screens showing CNN's coverage of the second Republican debate.

Sitting on a table in front of us was a laptop computer running Linux and a data analysis tool called LUX2016 and it was getting as much attention as the screens on the walls. Waldrop, who is CEO and founder of ICG Solutions in Chantilly, Va., was pointing out the changes in the constantly moving bar graphs on his laptop.

Those graphs displayed on Waldrop's computer were reporting the readings from Twitter as users everywhere reacted to the debates and to the candidates. The readings tracked the number and nature of the tweets so it could update the results every 10 seconds. On the graphs, we saw segments for each 10-second period along with a color code for statistically important changes.

The color changed according to the variance measured by standard deviations. As a result, if the public on Twitter increased its response to a candidate by 10 standard deviations, the segment was colored red.

Equally important to how much of a response a candidate received was the duration and consistency of the response. For example, we might see a big response for 30 seconds when a candidate said something dramatic, but then a lower number other times.

But not everything worked exactly that way. One candidate, former Hewlett-Packard CEO Carly Fiorina, began picking up steam as the debate began, but instead of dropping when she wasn't talking, her response continued to grow. By 10 p.m., it was obvious that no matter what happened for the next hour of the debate, Fiorina had won.

Later, as we analyzed the numbers and looked at samples of the actual social media activity, it was clear that we were witnessing a turnaround in politics. The numbers didn't appear to come from a social media team, but rather from real people.

Equally surprising was what happened with some of the other candidates. Notably, Republican front-runner Donald Trump's numbers dropped, slowly at first, then quickly.

By the end of the night, Trump's numbers reflecting his social media support were some 900 points down from Fiorina. Afterward, Waldrop and I discussed what this drop might mean. Had we seen the beginning of Trump's decline?

As it turned out, it wasn't the beginning. What we were seeing was perhaps the first public evidence of a greater decline that had begun three weeks earlier. The next day Ben Schrenckinger, reporter for Politico Magazine, published the results of his own study using different data, but showing the same result that we had seen.

Our results also showed data that might give some of the other candidates reason to rethink their campaigns.

Wayne Rash

Wayne Rash

Wayne Rash is a freelance writer and editor with a 35 year history covering technology. He’s a frequent speaker on business, technology issues and enterprise computing. He covers Washington and...