Documented examples of deficient health provider performance include failures to meet coverage targets, absenteeism from post of duty, non-compliance with clinical guidelines, and even malpractice. Provider effort may play an important role in poor performance, when providers already have adequate knowledge and resources. While in-kind pay-for-performance can be a tool to motivate health teams and improve quality of service, their effects are difficult to isolate from the feedback and public recognition often bundled with incentives. Furthermore, there is little evidence of their effect in small teams of members with defined roles and interdependent activities, or their effect versus monetary incentives which are more prone to controversy and expectations of permanency.

While improving the quality of health care is a global challenge, it is particularly salient in the poorest and hardest to reach areas in low and middle-income countries. Starting in 2010, El Salvador implemented a public health system reform through the creation of government funded and managed community health teams, starting with the poorest municipalities. Health teams of seven to ten members cover the primary care needs of the population in pre-defined geographical catchment areas. Financing of the health reform combined public and private funds including the Salud Mesoamerica Initiative (SMI), a public-private partnership managed by the Inter-American Development Bank.

In-kind group incentives were linked to 11 key maternal and child health indicators covering outcomes related to family planning, prenatal, postnatal and child-care. Each indicator was assigned a performance target, based on SMI’s baseline information to account for feasibility. If a community health team met the target, it received points redeemable for in-kind incentives. Points for each indicator were weighted on the expected level of effort to achieve the target.

In a public event in October 2015, 75 community health teams already operating within the organizational structure of the Ministry of Health of El Salvador were randomly assigned to either Phase I of the intervention  – eligible to receive in-kind incentives (treatment) during the 6^th and 12^th month verification cycles, or Phase II – eligible only during the 18^th month cycle (control).

All teams, regardless of their experimental assignment, received the same performance feedback report, supervision and recognition for their achievements in each cycle. The report design was identical, except that teams assigned to the treatment group had a legend in their report explaining the incentive amount obtained during that cycle. Teams assigned to control had only a description of the points obtained overall with no mention of incentives. All teams that achieved 60 or more points received public recognition in front of their peers and a certificate. Reports were delivered at an event held at the end of each cycle with representatives of each of the 75 teams and the Ministry of Health present (See Figure 1). Teams had to earn at least 60 points to receive the minimum incentive of $650 worth of goods and up to $1,000 if they reached 90 or more per cycle.

Figure 1. Team performance reports and certificates

While all teams improved their performance over time, those eligible to receive incentives had a faster rate of improvement at the end of Phase I, 12-months from baseline. Figure 2 shows the density plots of the overall health performance score (aggregated weighted measure of targets met by a health team with weights equal to the points established for each indicator) by time period and treatment. Teams in the treatment group above 60 points received the in-kind incentive. The distribution of performance scores at 12 months is significantly different between treatment and control teams (p<.05).

Figure 2. Distribution of team performance scores by treatment status and time period.

The 11 health indicators used to measure performance are grouped into 4 domains: community outreach, quality of care, timeliness of care and utilization. When analyzed independently per domain, the in-kind incentives significantly increased the average standardized treatment effects – by 0.17 SD for community outreach, by 0.14 SD for quality of care, by 0.10 SD for timeliness of care, and by 0.096 SD for utilization of health services – compared to the control group. The largest effects were in domains that required less team coordination; community outreach relies mostly on community health workers and quality of care is the responsibility of physicians and nurses. The gains were concentrated amongst the lowest and (particularly) the highest performers at baseline. Teams with intermediate performance at baseline that required little effort to meet the performance goals exhibit no response to the incentive scheme.

Inspired to implement this design in your own work? Here are some things to think about before you get started:

• Are the behavioral drivers to the problem you are trying to solve similar to the ones described in the challenge section of this project?
• Is it feasible to adapt the design to address your problem?
• Could there be structural barriers at play that might keep the design from having the desired effect?
• Finally, we encourage you to make sure you monitor, test and take steps to iterate on designs often when either adapting them to a new context or scaling up to make sure they’re effective.

Additionally, consider the following insights from the design’s researcher:

• Random assignment was conducted in a public event with representatives of all community health teams and national authorities. Teams were informed that due to budgetary constraints the incentive scheme would be introduced in phases. Teams in the treatment group were eligible to receive in-kind incentives in the first two verification cycles, and those in the control group in the last one. Since teams in the control group were aware of their eligibility for incentives in the 18-month verification cycle, they could have started working to improve their performance since the start of the experiment, which would attenuate the incentive effects.
• Points redeemable for in-kind incentives were given instead of monetary incentives, as they are expected to: elicit greater effort, be less prone to backfiring by crowding out intrinsic motivation (they can be perceived as a gift-giving act that elicits loyalty and a response independent of the gift value), and be a salient reminder of the team’s performance from which members can derive pride.
• In the catchment areas of each team, household surveys and medical record reviews were collected at baseline and on each 6-month verification cycle, producing a panel of repeated cross-section samples for each area. This data was used to construct the 11 health care performance indicators for each team.