Evaluation of SORT IT finds high impact and cost effectiveness

TDR news item
4 March 2015

An independent evaluation of the Structured Operational Research and Training courses (SORT IT) has found high value in the teaching and course content, calling it innovative and a standard for capacity building. It recommends applying this model to other areas of health research and in closer collaboration with the World Health Organization and TDR.

Operational research (OR) analyzes and identifies strategies and interventions that improve health service delivery. Few professionals in developing countries have been trained in this area, so the funding was designed to increase this capacity.

The evaluation was commissioned by the United Kingdom’s DFID Research and Evidence Division, which provided £1 250 000 (about US$ 1.9 million) to the International Union Against Tuberculosis and Lung Disease (the Union) between 2011-2014 for this new model of training. The courses were developed with Médecins Sans Frontières (MSF) to better identify programme-related constraints and health system challenges.

In 2012, TDR developed a broader programmatic approach and combined efforts with the Union and MSF. The evaluation noted this shift from “the ‘bottom up’ approach with a single applicant into a wider and more collaborative environment where system questions are agreed by ministries of health and national WHO offices, the people who manage those data are then trained and there is agreement about implementing any solutions or recommendations that come out of the year-long training and mentorship.

Support to both individuals and organizations

The evaluation found that trained participants increased not only their knowledge of operational research concepts, but also their skills and engagement in supervising research team members, collecting data, implementing study design and disseminating results. Between 2009 and 2014 the course generated 272 articles published in 84 scientific journals.

“There is potential for further use of this training model in cooperation with the WHO and their wider programme environment, giving access to networks, policy-makers and practitioners, which would widen the model’s ability to influence.”

SORT IT evaluation report authors

One third of participants indicated that their individual efforts made a large difference to their organization. Significant changes included improved policies and guidelines, improved monitoring and screening of patients, more patients receiving antiretroviral therapy, and closer collaboration between tuberculosis and HIV/AIDS programmes at the central level.

Reviewers also noted how the programme has focused on balancing both gender and geographical location, which was felt to influence the longer-term sustainability and impact.

SORT IT builds careers

Dr Ajay Kumar was a participant in the 2010/11 Paris course and it was the first time he had worked on a research project. He investigated HIV and TB co-infection, which resulted in a policy change even before its publication. The impact of Kumar’s work has also aided his own career progression. In 2012 he joined The Union as a fellow, where he now works full time on operational research. Kumar has been listed as a leading or co-author on close to 60 publications and has been promoted as Deputy Director (Research) since the initial SORT IT course.

A large majority of the respondents said that they intended to continue operational research after finishing their training, and around half of them indicated that their organizations gave high or very high priority to this field. The assessment found that one of the positive outcomes was the spill-over effect, where course participants encouraged other colleagues to develop their capacity by sharing knowledge, mentoring and teaching, participating in research projects external to their own department, and using findings in their daily work.

Expansion into more countries and areas

The evaluation identified “potential for further use of this training model in cooperation with the WHO and their wider programme environment, giving access to networks, policy-makers and practitioners, which would widen the model’s ability to influence.” It also supported the plans to build leadership capacity, and the opportunity for faster expansion into new areas of the world.

DFID is providing continued funding for 2015-2019 for regional and national SORT IT courses in Africa and Asia, and TDR is currently looking at additional funding sources. Expansion faces a number of challenges, including increasing the number of workshop facilitators, retention of people in OR and covering more public health issues. A series of recommendations that were made to guide future course development and address potential issues relating to sustainability, scalability and cost include:

  • A strong country-focused approach to the sourcing of participants;
  • Team-based projects rather than individual projects;
  • More transparent process and selection of participants;
  • More involvement from the ministries of health and national TB programmes in the course participants’ home countries to align national and regional priorities and influence uptake of results by policy-makers;
  • Improving dissemination practices (beyond publications), particularly through established networks like WHO and MSF;
  • Introduction of short courses for OR consumers to facilitate the uptake and funding.

For more information, contact Andy Ramsay (ramsaya@who.int).

Related links