Everyone has an opinion but some are more important than others: researched vs. social

excellent stuff. decision making based the small % of people who use particular social media platforms and are active on it, and who are consciously making comments ( not just hitting a button to retweet, share or pass on headlines without thinking them through, reading body copy or attachments, considering the source of the original etc etc ) is no real reflection of reality.

Thanks Dave!

http://i1.kym-cdn.com/entries/.....04/old.jpg

People vent their opinions on lots of other social sources as well. I not you left out Facebook.

case in point look at the hammering golden boy Mike Baird is getting for his obscene lock out laws comment

Very nice piece, and nice reminder not to assume a relative active minority reflect the mainstream.

So many companies these days focus on social media monitoring as the ultimate way of tracking a brand, but it’s so much more.

And you’re totally right about the “Don’t know/don’t care” issue.
As a quantitative market researcher myself, I often get pushback from clients who want to force a response and remove the middle ground. To a researcher trying to protect the data integrity and actually reflect what the consumer thinks (“I honestly don’t know” “I honestly don’t care” “I honestly couldn’t tell you if I’d buy your product or not, you haven’t convinced me yet”), it’s a difficult challenge when clients say “No, someone MUST have an opinion!”.

Interesting article…The current PM should be well aware of how the ‘don’t knows’ can fatally affect national debate. Looking at the old referendum on the Republic, in which he was in charge of the pro campaign, it was lost even though a majority appeared to want some sort of a republic. They did not want his version. They probably didn’t know they were putting the whole thing off into the never never by voting no. The monarchists still snuffling up their sleeves at how cleverly theymanipulated this into a no vote. Those with long memories can recall past surveys showing a majority of Aussies didn’t know whether Australia had a Constitution. This at a time they were being asked to vote to change it.

Agree with the above, nice piece. The added problem with social is when a comment stream gets hijacked by special interest groups, further skewing the commentary. As the author has pointed out, that ‘don’t know/don’t care’ component could be the iceberg beneath the waterline – Rumsfeld’s unknown unknowns.

It has to also be said, however, that quant/qual polling has its shortcomings. I haven’t found a full explanation yet, but the rise and sustenance of Trump seems to have caught all the professional pundits by surprise. Given the amount of dollars and re-benchmarking in polling methodologies that every US election seems to bring, were the pollsters measuring against legacy understandings of the electorate and missing this emerging (and quite significant) exasperation at the political class? Another (hitherto) unknown unknown?

Yes Georgia, the republican debate and referendum was a classic example of assumed knowledge and not providing the full range of options. Thanks for your comment

It’s a good point Andrew. I focussed on twitter given it seems to get the attention of pollies and media but you’re right, Facebook is increasingly heading that way too. I think Baird uses it well, even if his latest is getting a hammering, it puts him front and centre of the debate

As a film critic, I fully (and a bit self servingly) endorse the view that ‘everyone has an opinion but some are more valuable / informed than others’. Transposing this to social media, I would say the anonymity provides a responsibility-free opinion-magnet that devalues those opinions.

I would have thought that projecting the ‘thoughts’ of the chatterati on most social media platforms as a true reflection of mainstream opinion, or any of a million “measureable brand ‘values’ ” is enough to make most professional researchers faint.

Alan Robertson
Kinesis Media

Lazy journalism 101: if you want to master Journalism 2.0 just comb twitter for opinions

Love this article – think Pete Wilson has nailed it – there is also a trend to reporting opinion polls without the number of respondents, how/when poll was conducted etc – so it ends up in a screaming headline quoting a stat based on a sample size of 87 randoms wandering past in a shopping centre … without the ‘don’t know’ or ‘don’t care’ option.

Isn’t distortion, or more euphemistically, influence of what we think or don’t think exactly the point of so many of our jobs? The Twitterati are at least more reflective of ‘the wisdom of the crowds’ than the inner circle of government managers who decide that day’s ‘talking points’ or the small group of people who make corporate editorial decisions at Fairfax, News Corp and our major broadcasters.

Here’s what Hancock Prospecting said when they bought a big stake in Channel 10 a few years back: “Our company group is interested in making an investment towards the media business given its importance to the nation’s future and has selected Channel 10 for this investment.”

Tweets aren’t science, they’re just another voice in media’s overwhelmingly corporate cacophony.

I have no particular opinion on this matter, but I want to be heard anyway.

Always good to see some common sense discussion in the frothy world of Twitter adoration.

I think this is a great piece with some important points, that I really hope are not lightbulb moments for serious professional researchers. Having said that, there are a few sub-points I’d like to add:
1) The media is also guilty of being even lazier by focusing on the opinion of one (1) person – an expert. A facetious person might see this opinion piece as an example. Sure, their expertise might make their opinion ‘better’, but how much weight should be given to expertise over representativeness? Also, an expert is more likely to have bias or an agenda than a layman. Should differing opinions be also given in the name of balance? These are complicated questions.
2) Sometimes it does make sense to pay attention the noisy minority (i.e. social media streams), precisely because they are noisy, and while they may be the minority of the population they may also be the majority of the population who actually know/care
3) Should we really discount the opinions of people who don’t know or care? Isn’t that a viable option in many cases? Our compulsory voting system would suggest that a disengaged vote is worth as much as an engaged one
4) We must always remember what the average person is capable of telling us. They may be able to tell us what they consciously think and what they did recently, but they are pretty terrible at telling us what they unconsciously think, what they did a while ago or do habitually, absolutely awful at telling us why they think or do anything, and completely hopeless at telling us what other people think and do, and why. And yet these are the questions they are often asked. “Why do you think Donald Trump is so popular with voters?”

Ow! Robbo, how could you tell I had fainted.

Someone mentioned sample size, but to me the big issue is the context and wording of the questions asked. Someone also sagely mentioned the how the Republic debate was lost with exactly this technique.

I once had an assignment at Uni that had to be completed before the end of the lecture. Design a questionnaire that proved that all Australian men are wife-bashers. Pfft we all thought – you can’t as it’s not factual. Good old Eddie Oliver put up a single slide:
Q, Do you still bash you wife – Yes/No.
One word – ‘still’ – changed the question and the outcome.

It was a lesson never forgotten – hence insistence on “Don’t Know/Not Applicable” on every questionnaire I have ever worked on.

Someone also raised the issue of the twitterati being allowed to have and to voice their opinion. Of course they do! But then so do the silent majority – which believe me truly exist (and are BLOODY hard to research). The skill is in what weight or value you attach to social media comment streams.

Cheers.

@ Chiming In

Top level I took away: Do not rely on social for patterns, use researchers. The piece is written by a researcher.

There is an old saying that ‘rings’ true: Don’t put all your eggs in one basket. Another is: (Enter percentage of your choice here) of stats are made up.

‘Surveys’ from research companies can also be skewed to suit the clients agenda.

Nice piece all the same!

Nicely written although in the big theme of things social is an excellent source, given the bias and vested interest in just about all other “research”. So if you want to pump out your dodgy “message” or be told what you want to hear then you continue to use all the usual “research” groups, focus groups, meaningless surveys etc. For most of that better to use social *provided* there is actually enough relevant activity *AND* you actually want to know the reality. Sure, it is an imperfect design, but so are all the others.

For the 20% of serious credible research then you need people with integrity and expertise who know how to design the “experiment” and how to interpret it properly. These people tend to know the difference between correlation and causation 🙂

Thanks all for your generous and insightful comments. Just to pick up on a few of them.

Al, the irony was not lost on me in writing an opinion piece about opinions. And while I may not be an expert I take yours and others point that especially in the commercial world it can be difficult to appear objective and unbiased when dealing with client’s issues. The constant struggle of telling clients what they want to hear versus what they should know. As Bell Ringer and Walter point out traditional research techniques such as surveys and focus groups are by no means pure and are as only as good the people who design them and analyse the data.

And just to clarify, I’m not against using social media analysis as part of opinion measurement (we do it all the time). I think finding the balance and overall context between social media and broader opinion polling is the real challenge. And not always an easy one as many of you have mentioned.

And I would never suggest discounting the opinions of people who don’t know or don’t care. That’s the whole point. They need to be taken into account and understood especially if you do want people to become more engaged with a topic (although the community’s blissful ignorance is often a desired outcome for some clients – “don’t wake the snake!”)

Basing any news story on what “Twitter thinks” is about as useless as those dumb “man in the street” vox pops so beloved of lazy TV journalists.
We know the problem. Question is what, if anything, can we do about it?