Does research help or hinder?
Howard Lake | 18 August 2010 | Blogs
We’ve just completed a significant research project and it served as a timely reminder of the pitfalls of reading too much into research findings. I write a guest blog for Ask Charity, the specialist PR and communications service for charities, and in doing some background research was surprised to see just how much research is shoe-horned into communications or used to create eye-catching ‘headlines’.
This isn’t generally what research (particularly qualitative) is for and I’m sure such use led to the adage “lies, damn lies and statistics…”. Consequently, I posted a few thoughts on how to get the best out of such research and how to get carried away with findings; here it is and if you have any thoughts, please let me know.
————————————————————————————————————————————————-
I read a blog post this week that made me think about how we use research to inform communications activity. The post was written by a well-respected fundraising specialist, Jeff Brooks, and was entitled ‘Qualitative research fuels wishful thinking’. I respect Jeff’s thoughts – but suspect there is a bigger question to consider.
He suggests that qualitative research can encourage activity that is highly unlikely to meet its objectives. The example cited is of a study published by The Communication Network, which suggests that “people were more persuaded to act on children’s issues by positive communication than negative”.
This makes sense at face value, and as a sentiment is something we can all probably relate to. Which brings us to the bigger question.
Should we take any qualitative research at face value and act upon it without further questioning? In Jeff’s example he believes professional fundraisers know full well that donors need to see very clearly that there is some negative which can’t be resolved without their help. They also want to see the positive impact their donations will have – but they have to understand the need first.
So was the research totally wrong? It wasn’t wrong, but neither was it 100% fit for purpose. It turns out the subject of the study was issue advocacy, not fundraising, so the answers respondents gave were not in a fundraising context. They weren’t thinking specifically about what motivates them to hand over their hard-earned cash.
A second risk of not questioning research is misinterpreting results. This is much easier to do with qualitative findings and we’ve all seen stories written and decisions made based on a certain interpretation of the facts.
My favourite example is from an ex-colleague (who I can’t name for professional embarrassment reasons). He was planning a party event for Club 18-30 holiday-makers and looking at data from a feedback questionnaire. The survey asked two simple questions about alcoholic drinks: ‘Do you like beer?’ and ‘Do you like spirits?’. Not surprisingly, the results were over 90% for both, so he ordered a similar amount of each for the next event.
You can probably guess what’s coming. The beer ran out after only a few hours and he received lots of complaints. Not until sometime later did he realise that the question around liking beer and spirits actually had nothing to do with people’s preference for one over the other when they were at an event, during the day and in the sunshine.
I don’t think qualitative research fuels wishful thinking. I think we as authors and decision-makers fuel wishful thinking by using the findings out of context or misinterpreting the data. Ultimately, we make decisions inspired by research, so we must make sure that any conclusions we draw and the resultant decisions we make reflect what respondents actually think.
Kevin Baughen is the founder of Bottom Line Ideas, a volunteer for a range of causes and a Trustee
About Howard Lake
Howard Lake is consultant editor of UK Fundraising. He founded the site in 1994 and successfully sold it in 2022. As director of Giving X Ltd he is exploring growing giving on a massive scale.
He is the founder of Fundraising Camp and co-founder of GoodJobs.