Why PR metrics are as much use as a chocolate Karl Stefanovic

I agree with a lot of the above Tim, and getting metrics right is essential.

I do worry about this comment: “When it comes to PR, no wonder many clients take the discipline less seriously than any other agency time.”

What do you base this on?

It worries me that for an industry publication that is supposed to be “Everything under Australia’s media and marketing umbrella” there is a consistent anti-PR tone in a number of articles published on this website.

There are a number of organisations who do some good and often difficult work in this space. Im not talking about the fluffy product based PR with big budgets and flashy stunts that mumbrella loves to poke fun at, but the real public relations around engaging with communities and stakeholders around difficult issues, or the type of PR that works with organisations to help them communicate what they do.

It is no wonder that, as you say, PR is not taken seriously when the only thing that your readers have to base their information on, is the cynical view of a single facet of PR that you choose to publish on here.

Maybe its time for the industry to find a more appropriate publication to represent it.

While we all know that journos barely like to acknowledge the existence of the PR industry let alone the role it plays in generating news content, it’s a bit mean taking a swipe to the entire industry because of one silly firm’s mistake.

Talking down PR relative to advertising also raises an interesting conflict of interest question for a media outlet reliant on advertising revenue.

Hi Tim,

The best kind of communications measurement is undoubtedly content analysis that looks at the favourability, issues and messages of each piece of content and provides real insight into not just the reach but most importantly the quality of the coverage. At Media Monitors we have been doing this kind of analysis for clients for over ten years, and have just recently launched a business dedicated to media analysis and consulting, 360m.

Accurate quantitative measurement is also a highly useful tool and I would respectfully submit that your sample of one does not detract from the fact that there are many communications professionals using very good measurement techniques to analyse their media coverage, both qualitatively and quantitatively.

Patrick Baume
Group Communications Manager
Media Monitors

Whilst agreeing with the move to Reach …. it’s better to understand what Reach is first. Media 101 training anyone?

I have sat in silence, trying to control my jaw from falling, in so many presentations watching pr agencies, social media agencies, etc present these kind of crazy numbers.

If you gently ask about the methodology, you get an immediate reference to some “international standard”.

My favourite is when at the end of the stats, everything gets multiplied by 6, because that’s what some international standard is for word-of-mouth multiplier. Maybe DEC should be praised for not claiming 198m.

Good on you Tim for calling out “The emperor has no clothes”.

The validity of AVE and multiplied “reach” figures are the same issue – it is taking a bunch of assumptions on the audience, and making results of a campaign look more successful than they really are. If you are going to evaluate a campaign properly go for real data, real stats – hits, time spent on page etc – and possibly using a source other than the advertising circulation guides for an agressively marketed, all ecompassing, multi-channel, news website.

‘Well then’ – you give Tim a dig for his apparent anti-PR tone, but then you give a little generalisation backhander to fluffy product based PR. To think, all of these years we haven’t been doing the ‘real’ public relations. I feel duped!

Wouldn’t it be great if PR Agencies had access to everyone’s Google Analytics data?

Unfortunately this isn’t realistic and most often Australian clients don’t have the budget to engage in the expensive but thorough monitoring of companies like Media Monitors.

Perhaps you can suggest a better way Tim?

I agree with @Well Then’s comment and have noticed a lot of negative PR sentiment on this site. It’s a shame that you can’t present more positive ways for PR’s and Media to work together. Disappointing.

Well good on you Tim. Instead of initiating a productive discussion about PR metrics that could have actually achieved something postive you’ve decided to waste the opportunity on an egotistical crack at one agency using measurements currently considered standard.

As for your ridiculous notion that it is startling that an agency would ‘go public’ with a methodology that is already accessible to the public (and has been mentioned on this site previously – https://staging.mumbrella.com.au/ogilvy-launches-new-tool-to-measure-effectiveness-53196).

The AMEC Valid Metrics are a reasonable attempt at measuring media exposure for PR campaigns however as pointed out by AMEC themselves they are not definitive rules for measurement.

As you said there are ways to measure actual page clicks for online media coverage however when it comes to print there are obvious obstacles preventing more specific figures other than circulation being used.

At the end of the day DEC, like all other agencies, will be measured by their client on a great number of metrics other than media reach and it is at this time that any good agency will be able to demonstrate their true value through the ultimate success of business objectives that have been set.

This problem isn’t unique to PR being common everywhere from the MSM to agency land. I very much doubt there is any malice in it – very few people people are sufficiently numerate to get it right.

C’mon … what part of a claim of 33m people reached on day one do you guys not see as an issue that drags down the credibility of the industry.

Pointing it out and a lesson learned IS productive. Letting claims like that pass without challenge is counter-productive in the long run.

Thanks Tim,

Wow. We didn’t expect our results to ruffle so many feathers – especially seeing as the data we referenced is generally audited, and widely accepted by the media buying, creative, digital and other agencies – so why should PR not also reference?

Like many other agencies, we use publishers’ claimed Unique Browsers, Audience, Circulation and/or Readership – data that is actively promoted.

While we agree it would be great for PR agencies and their clients to have direct access to the individual analytics of every article on every site, the reality now is that no PR agency or in-house team can feature this type of evaluation.

For this reason, the ‘opportunity to see’ metric is never employed independently by DEC PR. It is just one element of an overall evaluation program, which is tailored for every client at the outset of a campaign. This goes beyond just potential or actual exposure, but also looks at engagement, influence and action through many interrelated factors including Message Penetration, Qualitative and Quality measures, Sales, Blog comments, Social media discussion and others. All fit within the context of the industry-wide Value Metrics recommendations.

Yes, it’s a complex debate. But most important to DEC PR is ensuring the evaluation and campaign meet the business objectives of the client.

In the case of this campaign, DEC PR was charged with creating awareness about the Save the Bilby Fund and Darrell Lea’s continued and long-term support for helping save this endangered species.

With only limited budget and no ATL support, we – and most importantly our client – are very pleased with the outcomes achieved so far, given the strong coverage now featured nationally on the Today Show, 7PM Project, NOVA FM, News.com.au, News Limited metro newspapers and mX. This exposure will of course be assessed alongside a range of other metrics.

But for now, let’s focus on saving the bilbies: http://www.savethebilbyfund.com/

Lots of “Tim- dont be a hater” comments here- however, claiming to reach 33m people when the population of australia is only 22 is… a little ludicrous.

If that is the accepted metric, then there is something seriously wrong with the system.

I agree with Thomas. This shows a lack of understanding about the fundamentals of media communication planning. The 33m is gross impacts (or opportunit to see). 22m is clearly the ceiling on this number.

There is also the issue of using monthly cumulative data provided from some media instead of average daily (e.g. issue) audience data. A classic example is the use of Monthly UBs from Market Intelligence, where due to the rate of cookie deletion the number of Monthly UBs in Australia is now over 110 million – what used to be a good proxy for audience is now a very poor one.

We could take a few guesses and say that because it was a one-day figure the longitudinal duplication was low but the cross-media duplication was (say) 6 … so the POTENTIAL audience may have been something around 5-6m. Clearly no-one knows the cross-medir duplication but we are starting to get some better data on it. Maybe one day the advertisers will stump up the necessary funding to get some empirical data that we can use in modelling such results.

But the disturbing thing is that a press release was able to go out claiming an impossible figure. Why weren’t alarm bells ringing?

The objectives of the campaign should have been stated in the brief, and how performance against them would be measured. Assuming that were so, the only evaluation that matters is whether those objectives were met.

In fact, it should be in the interests of the PR company to undercount the reach of the campaign. Imagine if traffic at Darrell Lea had increased by 100,000 customers in the lead-up to Father’s day based on activity that coverage reached only 100,000 people!? Now that would be impressive!

For the record, I only learned about this campaign from reading Mumbrella – just 1 of 33 million opportunities to see.

I don’t disagree with Mumbrella or the other comments at all about the ridiculous figure of 33 million. It was not justified, and ill-informed.

Yes, the campaign did reach a large amount of people – and probably warranted touting the result (this is an industry focused publication isn’t it?). DEC should have been more careful with their figures.

What I take offense to is the now Editor in Chief of what is supposed to be an industry publication that quite a number of public relations people read and contribute to regularly, take a swipe at an industry that supports it.

Tim – why don’t you tell us your real thoughts on PR?

The Public Relations Insitiute of Australia has advised all members for several years that Equivalent Advertising Value shoiuld not be used to assess coverage of campaigns. “Nuff said.

I think I saw some PR smoke and mirrors just up there, but let’s forget it to save the Bilby.

@Michael

I think your “result” ruffles so many feathers because it’s no result at all, but rather pure fantasy. If you’ve based your number on audited data (and I have no reason to disbelieve you) then you’ve grossly misunderstood what the data were telling you.

Take heart, you’re hardly Robinson Crusoe.

Lets face it PR is a mostly self created bullshit art that defies measurement and relies for its existence on the ” but what would happen if we didn’t” arguement. In most cases nothing would happen. If it vanished as it should a few car leases would go unpaid, a few restaurants would go broke and a whole lot of wankers would have to find something else to do with their hands. Blogs like this would have less supporters of big tobacco, clubs and pubs, and Tony Abbott and the world would on the whole be a better place.

Tim – my preference would be that you cover it without subjective remarks about the industry.

If you have an issue with metrics, then do some real reporting into it, rather than use one silly press release to label an industry. Surely you have other contacts in the PR industry rather than relying on press releases…..

It’s an issue worth raising, but really, the example Tim gives for news.com.au or his own website could equally be applied to any media – the difference is you can measure online more accurately.

Does anyone believe that the readership figures for newspapers or magazines accurately represent the number of people that actually read a story that, say, appeared on page 12 or buried in a supplement?

Likewise TV figures, which everyone seems to accept despite the extraordinary extrapolation from a handful of set-top boxes.

Hi Tim,

It’s one thing Mumbrella holding a PR agency accountable for it’s actions – and this has been done very well in this case – but I think you owe an apology to the PR industry for the last two paragraphs of your piece.

The fact that one PR agency is happy to go public with a silly measurement figure and methodology doesn’t demonstrate a problem across the whole industry, and many clients don’t take the discipline less serious than any other because of differences in measurement.

Agreed?

Thank you Tim for furnishing me with a link to an article that highlights just one of the many insanities of marketing.

Now I don’t have to rant like a loon about PR numbers being utterly idiotic. I can just forward this link.

Please continue to expose the other 53,749,103 on my current list and I can stop thinking so seriously about trying to get a job in a video shop instead.

Anonymously (due to video shop application reference likely being seen by boss)

I provide conservative estimates for my clients as I consider that to be an ethical business practice. I report on circulation figures for print publications with a note that the readership could potentially be larger than that.

Also Tim is spot on regarding page visits vs site visits. Having worked in-house before, I was amazed at the circulation figures our U.S. PR firm was providing, which were completely disproportionate to the rest of our communications results. I questioned this and it did end up being monthly figures that they were providing. This is ridiculous as we all know online ‘hard’ news stories are viewed by the majority of people for only a day or so after release.

This can also make it difficult for PR agencies – that report accurately – taking on a new account because the client may ask why you are only reaching 800,000 people, for example, when the previous agency was reporting figures in the millions.

Unfortunately it is very difficult to enforce an industry standard and so it is up to each client to scrutinise their agency’s rationale.

Wow! A fair bit of speen venting there from Groucho! Did he get sacked from a PR agency or is he always like this?

No Peter B, Groucho is always like this. I don’t think he’s worked in PR as that would require an occasional shower and the wearing of shoes. The poor sod is deluded into thinking that anyone with an opinion that’s to the right of idiots like Sarah Hanson-Young is some PR hack or stooge funded by Big Tobacco/Big Mining/Big Clubs etc etc..

The reality is it’s regular people like me who hate where these imbeciles are taking our country.

Ha ha ho hum. Over the years I have been a non executive director of 2 or 3 PR companies representing hollding companies on boards and I did shower occasionally – after the board meetings. Sometimes twice. As for shoes I still wear the same ones to clean the stables.

Hey ho hum. I think you’ve inadvertently hit the nail on the head.

I don’t mean your facile assessement of the political spectrum. I don’t mean your denial of the inordinately large influence that big business and its lobbyists have on our country (have you ever met a lobbyists or big business that has faced the ballot ox?x).

I refer to your admission that “regular people” like you are haters.

In the past six or so years this country has become increasingly inflamed by the likes of your reactionary mates and their continues espousal of negativisim and hatred, much to the detriment of our society. May the pendulum of public opinion swing back to a more palatable level soon.

let’s all have a hug Lenny while the country goes to hell, is that more palatable to you? let’s face it, the “government” has done a great job of inflaming almost everyone, don’t hate on me for that

Well done ho hum. You’ve toned it down to vitriol.

If you think this country is going to hell then I think you are deluded. I think restricting tobacco advertising is a good idea. I think stopping the insiduous spread of gambling addiction is a good idea. I think that making mining companies pay at least a rental for consuming the mineral rights that are collectively owned by the people of Australia is a good idea. I think that providing care and assistance for those affected by mental health issues is a good idea. I think that making big polluters pay for the damage they do is a good idea.

I’m not saying that everything is far from rosy (and far from it on immigration issues for example) … but credit where credit is due.

But then again I would say that as I am not hater.

No Lenny, you’re just a misguided Get Up wannabe who has brought the usual grab bag of left issues to an advertising and marketing blog. I’d prefer to live my life without your grubby mits all over it. Thankfully you drips are a minority and expect to be even more minor at the next election.

And funny none of you remember the unbridled hatred dished out to John Howard by the far left while he successfully ran the country for 11 years, and more recently dished out to some innocent chocolate shop owner in Newtown. Hate is apolitical.

ho hum take your silly political trollery elsewhere. this is a comms blog.

@Groucho can i suggest that if you’re not sure whether you a Director of 2 or was it really 3 companies you knock back similar gigs until you see a lawyer and clarify your director’s duties

I knock them all back Bob, the showering was too much, and the shoes hurt. I don’t need a lawyer though, and I need a fucking pedant even less.

Back to the original topic at hand if I may?

While I agree with many comments here that DEC appear to have made a mistake on monthly reach vs short-burst coverage (eg topic only relevant for a day therefore need to factor figures by /30), I do take offense to the somewhat misinformed commentary on PR as a whole.

There are many facets of communications that form ‘PR’, with media relations one spoke of a full strategic wheel. Media relations is indeed the most difficult to measure – but this is not an issue isolated to media relations.

I note the outdoor ad industry also struggles to prove ‘results’. If only media relations pros could work with media to do an eyeball study to demonstrate true readership, sentiment and message recall?

Anyone want to pitch this to Murdoch for me? 😉

My point; metrics based purely on reach are never accurate whether it is media relations or TVCs. Very few communications pros use them in isolation and clients are smart enough to call out the ones who try it. Then agencies who prove business results through holistic comms win these clients and the cycle starts again. It’s no different to ATL or digital.

Karalee, I note the MOVE launched 18 months ago and has greatly assisted outdoor to quantify the audience it delivers.

Also, statistically speaking reach is probably the audience metric that provides the MOST accurate data, simply because of the fact that it is an upper-bound metric – it can’t be more than 100%. (Unless of course you can ‘reach’ 30m of Australians on the first day of a PR campaign.)

Hello me hearties! Let’s just all be honest. Many forms of promotion are hard to truly equate. Therefore just be upfront and state where it is being seen. Online stuff can show views, however that could be a 5 min full read, or a 2 second back button swerve…

I still laugh very loudly at TV stats. I have never met anybody who has a box..?

Shiver me timbers…

Well said Lenny Bryan. I agree totally with your sentiments.

@Groucho I’m not a pedant. If i was I would’ve pointed out the mispelt ‘arguement’ in your first comment. I just called bullshit on your self-important post.

dodgy stats aside, isn’t the key issue just how much more engaged and trusting of the message is an audience reached through PR than any form of advertising?

Does anyone know the stat on this? (ie people read webpages/newspapers/watch TV for the content, not the advertising)

Fuck you PR people.

I am sick to death of my ad sales team getting several calls every day from someone from PR company needing our media kit. Of course you need our media kit. You need to take the editorial space we have given your clients product and multiply it by 3 or 5 in order to impress the client about what a great idea it was to pay you a retainer because you saved so much money on your advertising spend!

Th reality is, 95% of the time my team know what we like and don’t like. It’s rare that a PR organisation makes us aware of something we didn’t know. Even more depressingly, more often than not we know more about the product that is being pitched to us by the PR organisation.

But more importantly than that, the anti advertising argument that is the cornerstone of your business is affecting the bottom line of the media that you are so proud of putting in your client reports.

And don’t give me that bullshit about how media is being democratised and that there is such massive reach with social networks and online etc. You place no value on our websites or social networks. If one of you ring and want to offer something to our readers I am amazed at how quickly that is refused.

But a big glossy write up in a market leading magazine? Yes please!

My sales team will get a call within minutes from some junior in your office claiming to have taken over the media buying duties of that client. And they need a media kit. Stat! Why? We ask. They were already represented by ABC ad agency? No no no you say. This is part of our new special project.

Whatever.

I challenge you to just have the guts and say “we need your media kit because we want to value the publicity that our client has received in advertising dollars.”

Occasionally this is followed up by a client needing to cancel their advertising because they are “getting publicity anyway”. That’s OK. Because I am sure my editor feels overpaid and underworked regardless. Hell, he would probably welcome a pay cut.

Above all else I am in awe of your ability to sleep well at night.

And I think that is because you are nothing more than empty shells of people with a rolodex for hire. Happy to spruik whatever crap someone is willing to pay you to flog at that particular point in time.

PR is for loosers

Was the above comment written in 1980?

today, thanks to the intrawebs, you can download a media kit directly from the website of any publication.

you might also find that the intrawebs, and not the PR industry, has undermined the journalism business model

it’s good to know , though, that the advertising industry won’t just spruik whatever crap someone is willing to pay them to flog at a particular point in time

they really have to believe in things like live-reading betting odds during AFL games and running junk food ads during children’s programming

PR is for people who can spell

No Bob.

PR is for journalists who weren’t good enough to make it.

And you are right. It is the internet. The advent of the internet has helped scale the PR industry back over the last decade.

The point is however valid. Advertising is advertising. PR is a murky world of half truths, misinformation and deception. Your industry is an attempt to disguise advertising as something else. Keeping those waters as murky as you can for as long as you can.

At least I know what Harvey Norman is doing when I see one of their ads. I can turn it off.

All I am asking you, Bob is that you find some other way of measuring your return on investment instead of measuring it in advertising dollars. As an industry, I challenge you all to just grow a set and stop measuring everything in advertising dollars.

This industry has a really thin skin. I can’t help but wonder why that is……..

Not sure why we don’t just settle this tiresome debate the old fashioned way? Dawn at thirty paces?

I mean really, the ‘issues’ here are so outdated and the naivety, even ignorance, from some of the comments here, is getting to be a little silly. I’m willing to leave the ‘aren’t you just threatened PR are stepping on your turf’ argument alone as I’m not sure Andrew Bolt & Gina Reinhart’s Love child could cope from the brain aneurysm.

Tim – suggestion: let’s turn this debate into something constructive. Perhaps a round table or written debate is in order to shed some light on the realities from both sides?

Andrew Bolt & Gina Reinharts Love Child – you are referring to AVE (advertising value equivalent) which is no longer used by the vast majority of PR agencies. Full stop.

However, I know of many ad agencies who use it to prove ‘value’ of their campaign’s media results. You know the ones – they had a great creative idea, executed it, hit MSM and measured its success from the value of the placement received in editorial. Just use the good ‘old internet (perhaps Google or Bing if you prefer) and search for who is touting the AVE of World’s Best Job or NAB’s break-up campaigns. *hint* they’re ad-agencies, buddy.

Well said Bob.

The NAB break-up campaign is bloody awesome though isn’t it.

(and Clems/NAB haven’t claimed AVE they’ve touted cold-hard business results. 350,000 new customers and counting)

@Karalee “Dawn at thirty paces”?

I have heard of pistols at dawn..?

Or are you being very smart and my penny hasn’t dropped..?

H’mmm? Do people still walk the plank..?

So do they still stand by the claim that they reached 33 million????

I love Clems – first Google result for Nab; last para $5m in media coverage in a single day. Journo didn’t get that from thin air, it’s part of your case study.

http://www.news.com.au/busines.....6079138892

Love Child, listen to Karalee. She’s right about your level or naivete and/or ignorance. The vast majority of the PR industry doesn’t use AVE and many people never have. Yet the ad industry will itself refer to AVE whenever they score some free editorial.

Contrary to your assertions the PR industry’s sustained 20 year period of growth has been aided by the internet and the greater reputational risk it poses than abated by it. Contrast this with advertising which is a contracting industry of diminishing relevance – you’re right, people DO just turn ads off these days. Nice own goal.

I had to laugh when i read ‘advertising is advertising’ – oh really? what do you call Brand Power and the like, who run around touting the degree to which consumers are fooled by their format.

Witchery jacket, anyone?

And don’t get me started on the promotion of insecurity, feer, greed and mindless consumerism that form the bread and butter of the advertising industry.

Rather than try to use this forum to launch an ill-advised attack on PR, you might wish to consider using your energies to improve the image of advertising instead.

I do not trust any figures for any media, traditional or new. They remain too easy to manipulate and fake and generally the people providing the figures have an interest in them being as large as possible.

What I trust is sales and long-term behaviour change research. That is all that justifies the expenditure on PR or advertising.

Are the bilby’s saved now?