Testing, testing 1, 2, 3
Howard Lake | 9 April 2009 | Blogs
If you were about to start a fundraising campaign to your supporters, would it be a good idea to mention the recession to them?
What do you reckon? Tell them or don’t tell them?
Anybody’s guess really. I’d probably have gone with ‘don’t tell them’, but that’s based on my gut feeling.
But that’s the point because that’s what most of the decisions about the recession are based on – gut feeling.
I was talking with a charity fundraiser a couple of months ago about some recently-published advice about how to fundraise during the recession.
“But how do they know that will work?” she said. “Doing the opposite might work just as well or even better.”
She had a fair point because this advice wasn’t based on any evidence or the result of previous experience. It was just somebody telling us what they ‘thought’ would work. But they had no evidence.
There was an item in Third Sector earlier this year that the Prostate Cancer Charity ran a DM split test to its high level donors that made just this differentiation. Some were told about the recession, some weren’t.
The result of this test was that the donors who received a mailing that talked about the recession gave quite a bit less money.
More than the findings from their split test, what I think is important here is that they did the test in the first place.
I don’t reckon fundraisers do enough testing these days. Once upon a time, direct mail fundraisers tested everything, but a lot less of that goes on these days.
I think the problem is that not many people will put aside budget for a test. Yes, they’ll test the new pack against the control pack. But they are reluctant to spend money testing something completely new or on a medium that isn’t direct mail, such as making non-ask phone calls, or trying different styles of donate buttons on the website (which shouldn’t cost much money at all).
And, when someone does do some proper testing, there’s a tendency to dismiss it as being so obvious that what was the point of the test.
A good example of this is the analysis that Adrian Sargeant conducted on the PFRA’s attrition survey from last year.
Adrian found that if you customise at least one donor communication, you’ll get lower attrition rates.
You may reckon this is just common sense and at least one person responding to UK Fundraising’s news story on this research seemed to think the results were blindingly obvious.
Not so though – and Adrian has already responded to say why this is the case.
Why though, do we as a profession tend to look down on testing and research with a sniffy “it’s all so obvious”?
Is it, perhaps that thorough and regular testing brings fundraising uncomfortably close to the ways of commercial marketers? Maybe this is too much associated with the profit motive while we fundraisers prefer to see ourselves telling engaging stories and too much testing would hold us back from our storytelling?
Maybe not testing puts daylight between us and our commercial cousins?
I don’t know if this is what a lot of fundraisers really think, but one thing is for sure. We used to test a lot and we test a lot less now – and there has to be a reason for that.
About Howard Lake
Howard Lake is consultant editor of UK Fundraising. He founded the site in 1994 and successfully sold it in 2022. As director of Giving X Ltd he is exploring growing giving on a massive scale.
He is the founder of Fundraising Camp and co-founder of GoodJobs.