What the U.S. Presidential Election taught us about customer experience surveys

If there is one thing that the recent U.S. presidential results taught us, it is that taking time to get to know your customers is crucial.

Failure to get a true read on the audience you are trying to influence can have devastating consequences. Your success may be based on a shaky foundation of false assumptions. If there is one thing that the recent U.S. presidential results taught us, it is that knowing your customers is crucial.

On Election Day, pollsters were just shy of universally predicting Clinton the victor. The New York Times gave Clinton an 85% chance of victory; the Huffington Post 98%; the Princeton Election Consortium 99%.

By the end of the evening, as we now know, “pollsters flubbed the 2016 presidential election in seismic fashion.”  Polling as a practice and as a profession was dealt a blow from which it will not soon recover.

I found myself very sympathetic to the people who make a living out of gathering and analyzing political data. After all, data-driven planning is a “best” practice.  “Shooting from the hip” and “gut feelings” are so, well, yesterday. Yet, clearly, “best” wasn’t good enough.

On Election Night, before the winner was announced, the featured story seemed to be, “how could the pollsters have gotten it so wrong?” Several theories emerged:

  • Trump mobilized new voters who are harder for pollsters and analysts to gauge.
  • Shy Trump supporters may not have revealed their true thoughts due to social pressure.
  • Pollsters may have unintentionally built a bias into the polls based on their conclusions that Trump could not win.
  • Polls may have under-sampled certain portions of the population.
  • Polls may have over-emphasized the belief that the country’s rising diversity would mean a Clinton victory.
  • A surge in pro-Trump momentum may have been based on the Clinton-damaging FBI announcement so close to the election.

 

As someone involved in a customer experience transformation based on Voice of the Customer feedback and other data, I couldn’t help but wonder: if the Washington pundits could be so wrong, could my “pollsters”–my team that manages the execution of the Net Promoter System (NPS) methodology to measure customer sentiment–be wrong as well?

I ran through each theory with an eye toward uncovering similar vulnerabilities in my company’s own Voice of the Customer program.

  • We certainly run the risk of sampling too few customers and misunderstanding the motives of customers who choose not to respond to an NPS survey. And of customers that do respond, it is often difficult to determine their influence in the decision-making on future product purchases. (Our “win” is a successful customer experience that will eventually lead to additional future purchases.)
  • Customers and partners may be reticent about reporting unfavorable opinions. Alternatively, they may be reticent about reporting favorable opinions.
  • We could have biases built into our survey questions, though we have tried to follow best practices. Look where that got the pollsters!
  • Recent news (“what have you done for me lately”), good or bad, may have a disproportionate impact on results.

 

Yes, there are vulnerabilities in any polling and survey processes. To counter those vulnerabilities, we need to improve both the quantity and quality of the feedback.

We can improve the quantity in the hopes that increased sample sizes representing increased diversity are more likely to normalize around the truth.

We can improve the quality by, wherever possible, establishing trust and conducting face-to-face meetings. With this combination, we are more likely to obtain the purest, most truthful, most valuable feedback.

I think the election pollsters got it wrong because they failed to gauge voter loyalty. The NPS methodology is based on a single question meant to measure customer loyalty over customer satisfaction.  Rather than ask “Who are you voting for,” NPS calls for asking, “How likely are you to recommend this candidate?”  The latter question leads to measuring loyalty and I believe would have flushed out quite a few of those shy Trump supporters.

Do you think that surveys yield honest feedback?

@lynnemahoney


Lynne is Advisor, Global Customer Success, at CA Technologies. As a professional communicator, she writes…

Comments

  • Mark Combs

    Lynne, Hi, hope all is well! One of the analyses I read of the big polling ‘miss’ on the recent election attributed part of it to the fact that many of the people who actually voted for Trump would not admit even to their closest friends and family that they were going to vote for him…coincidentally, I spoke to someone just yesterday who told me that he had voted for Trump, but had never talked about it at all, to anyone, before that conversation. Interesting! mac

    • lynne_mahoney_ca

      Hi Mark! Looks like your trusting relationship flushed out a “shy Trump supporter.” As a company, we need to continue to work on building trusting relationships with our customers. Then our survey results will come closer to reflecting reality. Thanks for your comment!

rewrite

Insights from the app driven world
Subscribe Now >
RECOMMENDED
Are Robots Inheriting Our Prejudices? >Microsoft Targets Password Elimination to Improve Security >APJ: Plug In and Amp Your Business with APIs >