# Testing, testing 1, 2, 3 - Link: https://fundraising.co.uk/2009/04/09/testing-testing-1-2-3/ - Published: 2009-04-09T10:55:39+01:00 - Author: Howard Lake If you were about to start a fundraising campaign to your supporters, would it be a good idea to mention the recession to them? What do you reckon? Tell them or don’t tell them? Anybody’s guess really. I’d probably have gone with ‘don’t tell them’, but that’s based on my gut feeling. ## Advertisement But that’s the point because that’s what most of the decisions about the recession are based on – gut feeling. I was talking with a charity fundraiser a couple of months ago about some recently- published advice about how to fundraise during the recession. “But how do they know that will work?” she said. “Doing the opposite might work just as well or even better.” She had a fair point because this advice wasn’t based on any evidence or the result of previous experience. It was just somebody telling us what they ‘thought’ would work. But they had no evidence. There was an item in Third Sector earlier this year that the Prostate Cancer Charity ran a DM split test to its high level donors that made just this differentiation. Some were told about the recession, some weren’t. The result of this test was that the donors who received a mailing that talked about the recession gave quite a bit less money. More than the findings from their split test, what I think is important here is that they did the test in the first place. I don’t reckon fundraisers do enough testing these days. Once upon a time, direct mail fundraisers tested everything, but a lot less of that goes on these days. I think the problem is that not many people will put aside budget for a test. Yes, they’ll test the new pack against the control pack. But they are reluctant to spend money testing something completely new or on a medium that isn’t direct mail, such as making non-ask phone calls, or trying different styles of donate buttons on the website (which shouldn’t cost much money at all). And, when someone does do some proper testing, there’s a tendency to dismiss it as being so obvious that what was the point of the test. A good example of this is the [analysis](https://www.fundraising.co.uk/news/2009/01/26/regular-customised-communications-reduce-facetoface-donor-attrition) that Adrian Sargeant conducted on the PFRA’s attrition survey from last year. Adrian found that if you customise at least one donor communication, you’ll get lower attrition rates. You may reckon this is just common sense and at least one person responding to UK Fundraising’s news story on this research seemed to think the results were blindingly obvious. Not so though – and Adrian has already responded to say why this is the case. Why though, do we as a profession tend to look down on testing and research with a sniffy “it’s all so obvious”? Is it, perhaps that thorough and regular testing brings fundraising uncomfortably close to the ways of commercial marketers? Maybe this is too much associated with the profit motive while we fundraisers prefer to see ourselves telling engaging stories and too much testing would hold us back from our storytelling? Maybe not testing puts daylight between us and our commercial cousins? I don’t know if this is what a lot of fundraisers really think, but one thing is for sure. We used to test a lot and we test a lot less now – and there has to be a reason for that. * **[Irish fundraising feels the recession](https://fundraising.co.uk/2009/04/20/irish-fundraising-feels-recession/)**( 20 April 2009) * **[Charity workers 'flexible' to help colleagues retain jobs, says survey](https://fundraising.co.uk/2009/04/20/charity-workers-039flexible039-help-colleagues-retain-jobs-says-survey/)**( 20 April 2009) * **[Rich List's donations up 8% despite losing billions](https://fundraising.co.uk/2009/04/26/rich-list039s-donations-8-despite-losing-billions/)**( 26 April 2009) ## Comments ### By: adriansalmon - Link: https://fundraising.co.uk/2009/04/09/testing-testing-1-2-3/#comment-3601 - Published: 2009-04-09T23:46:47+01:00 Thanks for a great post, Gordon. Maybe it’s because I “grew up” in a telephone fundraising agency, and testing had to be what we did – day in day out. If a campaign wasn’t performing the client could pull it – and I’d have to lay off my callers! Maybe it’s partly a matter of timescales – let’s consider a direct mail campaign, where you have your control and your test segments, but you have to wait and wait– maybe 6 weeks? 8 weeks? once you’ve mailed the pack before you can really draw conclusions from your test. If you do 2 or 3 DMs a year then by that point you might be well into your next appeal planning cycle (so let’s stick with the control for now!) and so it isn’t until the one after that that you can start applying your learnings. But with the phone, you get that immediate feedback – you can start comparing your control and your test segments right away, and it’s a buzz when you see something start to work. Because then you can start repositioning your campaign while it’s going along! You can do the same with digital – either with email or Facebook/Twitter, or with the new ‘multivariant testing’ you can do on web page design and copy. The other reason I love my tests is because they’re not cold, clinical, commercial exercises – not in my world. They’re telling me about where my donors’ and prospects’ comfort zones are – and their excitement zones as well. My job is to inspire people and offer them the chance to do something that excites them, but within their financial comfort zone. My tests tell me how well, or poorly, I’m offering that. If something works, it means I suddenly have 200 or 300 people who are more engaged and committed, and I then know that I can generate the same enthusiasm from hundreds more. Perhaps there’s a fear that you need to be a stats whizz to devise and evaluate tests – you don’t. The maths is pretty straightforward and Excel will do it for you – no SPSS required. Perhaps there’s also a fear that if the test doesn’t work the whole campaign might fail – but you don’t need a huge sample to test one difference in approach. You can build it in without jeopardising the income you’re confident of getting. And perhaps there’s a resource problem – because database managers are already overloaded with calls on their time – and for tests you need stats. So if you want the stats as quickly as you can get them, there might be no alternative to learning what reports you need, and how to run them, yourself. But once you get into testing, it’s a fantastic discipline for your fundraising practice. You have to be able to flip between looking at the behaviour of hundreds or thousands of people (which isn’t to say you can’t test in major donor work too) to thinking about the individual motivations that lie behind those mass behaviours– and then working out a way to test your hunches. And you can’t do that without being insatiably interested in people, and finding a way to listen to what your donors– and non donors! – are telling you – visiting your phone agency, doing research, making opportunities to meet donors at events, and giving them channels to talk to you – which will make it a two-way dialogue. Long reply, I’m aware, but I do love this stuff! Adrian Salmon Annual Fund Manager University of Leeds ### By: georgeoverton - Link: https://fundraising.co.uk/2009/04/09/testing-testing-1-2-3/#comment-3602 - Published: 2009-04-16T12:33:57+01:00 The recession is currently on all of our minds, much of the time. So I think it’s open and honest to mention the recession in an appeal letter. While it’s great to test such ideas, I’d question whether deliberately excluding mention of the recession, for the benefit of a probably statistically tiny percentage difference in response, is a good idea. George Overton Prospect Researcher ### By: adriansalmon - Link: https://fundraising.co.uk/2009/04/09/testing-testing-1-2-3/#comment-3603 - Published: 2009-04-16T14:10:52+01:00 In the test Gordon mentions, gift values were up to 45% lower from the pack that explicitly referenced the economic situation. And the group that reduced their gifts by 45% were the high value donors. Next the mid-level (24% reduction) and finally the low-level donors (4% reduction). So it was a very valuable test. And I bet they aren’t mentioning the recession now– I know I’m not! Adrian Salmon Annual Fund Manager University of Leeds