Low response rates to government surveys hinder research and limit the robustness of the policy evaluations and recommendation. To compensate for low rates of response, research teams often need to increase sample sizes, making research more costly.

The U.K. Department for Business, Innovation and Skills (BIS) developed a program called Growth Vouchers in which businesses could apply for funding to help subsidize the cost of obtaining expert advice on topics such as finance, HR, use of IT in business processes, or increasing productivity by advancing the skills of their leadership teams. In order to help evaluate the effectiveness of the intervention, BIS conducted a telephone survey of the businesses in the program. However, motivating businesses to complete surveys was an ongoing challenge.

BIS used behavioral insights to develop and test different opening messages to a 16-minute long telephone survey with 7,000 small businesses from the Growth Vouchers program.[1]

In addition to the original message, they developed four new messages, including:

• Commitment mechanism message: used historic commitment
  • “You agreed to take part in a short telephone interview when you applied to take part in the Growth Voucher Programme in [DATE]. We would now like to interview you about your experience and what has happened in your business since.”
• Reciprocity message: used willingness to “repay a favour”
  • “You received [an online diagnostic assessment/a voucher worth £2,000/a £2,000 subsidy toward business advice] from the Growth Voucher Programme in [DATE]. We would like to hold a short telephone interview with you about your experience and what has happened in your business since.”

Two other messages included a simplified version of the original message, and a social norm message that drew on “herding” effects to try and increase survey completion rates.

A randomized evaluation found that almost 52% of businesses that received the messages reminding them of their commitment to participate in the survey responded by completing the survey, while roughly 46% of those receiving the original message completed it (a 5.5 percentage point increase).

Of businesses that received reciprocity messages reminding them of the benefit they had received from the scheme, 50% participated (a 4 percentage point increase in response rate).

Other messages, such as the one based on the principle of social norms, did not have a statistically significant impact on the number of recipients completing the survey.

BIS estimated the financial benefit of a 1% increase in response rate is in the range of £1,555-£4,720 for an evaluation on the scale of the Growth Vouchers program. These results show that simply by improving the design of introductory messages for surveys, at very little cost, we can significantly increase response rates and deliver better value for money for the taxpayer.

Inspired to implement this design in your own work? Here are some things to think about before you get started:

• Are the behavioral drivers to the problem you are trying to solve similar to the ones described in the challenge section of this project?
• Is it feasible to adapt the design to address your problem?
• Could there be structural barriers at play that might keep the design from having the desired effect?
• Finally, we encourage you to make sure you monitor, test and take steps to iterate on designs often when either adapting them to a new context or scaling up to make sure they’re effective.

Additionally, consider the following insights from the design’s researchers:

• It is advised to test before implementing at scale. Testing such messages is low cost when embedded in existing activities, and the benefits of more effective messages may be large.
• Training interviewers and helping them understand the significance of the messaging is very important to ensure that they repeat messages as expected.