Background: Despite substantial investment in health capacity building in developing countries, evaluations of capacity building effectiveness are scarce. By analysing projects in Africa that had successfully built sustainable capacity, we aimed to identify evidence which could indicate that capacity building was likely to be sustainable. Four projects were selected as case studies using pre-determined criteria, including the apparent achievement of sustainable capacity. By mapping the capacity-building activities in each case study onto a framework previously used for evaluating health research capacity in Ghana, we were able to identify activities that were common to all projects. We used these activities to derive indicators that could then be used in other projects, including to monitor progress towards building sustainable research capacity.

Results: Indicators of sustainable capacity building increased in complexity as projects matured and included: (i) early engagement of stakeholders; explicit plans for scale up; strategies for influencing policies; quality assessments (awareness and experiential stages); (ii) improved resources; institutionalization of activities; innovation (expansion stage); and (iii) funding for core activities secured; management and decision-making led by southern partners (consolidation stage). Projects became sustainable after a median of 66 months. The main challenges to achieving sustainability were high turnover of staff and stakeholders, and difficulties in embedding new activities into existing systems, securing funding and influencing policy development.

Conclusions: It takes many years for capacity building projects to become sustainable therefore indicators : i) should be both generic and context specific; ii) should evolve and increase in sophistication as projects mature; iii) need buy-in from stakeholders and should be revised regularly.