Critical appraisal of economic evaluations

Abstract Objective: Various economic evaluations have been conducted on interventions aimed at decreasing obesity. These interventions are changing dietary patterns, behavioural modification, increasing physical activity, surgery or a programme including the previous mentioned interventions. Yet it is hard to assess what the actual quality is of these economic evaluations as most assessment tools are of a qualitative nature. In this thesis the quality of the economic evaluations on obesity is assessed in a quantitative manner. Methods: The economic evaluations (abstracts) on obesity are selected in the NHS Economics Evaluations Database and the quality of these studies is assessed by using the assessment tool of Chiou et al. (2003). This quality assessment tool consists of sixteen criteria which all have their own weight. If a study could fulfil a criterion which was set, the entire weight was appointed. If the criterion could be addressed partially, half of the weight was appointed. None of the weight was appointed if the study could not fulfil the criterion. The sum of the weight of the sixteen criteria reflects the quality of the economic evaluation. To test the association between the characteristics of the economic evaluations and the quality, univariate, bivariate and multivariate analyses were performed. The characteristics are the year of publication, the category of intervention, the type of study, the country in which the study was performed, the location in which the study was performed and the resource of funding. Results: In the NHS EED 34 economic evaluations on interventions aimed at decreasing obesity were selected. These studies were published from 1995 until 2006 and had a mean quality score of 51.9 with a range of 24.1 – 75.8 (SD 13.28). Most studies were not able to fulfil the criteria entirely, because valuable information necessary to draw a conclusion was missing, the choice of the economic model was not discussed, statistical or sensitivity analyses were not performed and the resource of funding was not mentioned. Multivariate analyses showed that the year of publication is positively associated with the quality of the publication, meaning that if time progresses with one year, the quality of the article increases with 1.5 points. The location in which the study was performed was also associated with the quality of the publications. European publications on obesity interventions had a higher quality score, of 14.46 points on average, than non-European countries and thus were of better quality. In the univariate analyses the type of study was associated with the quality of the economic evaluations. The resource of funding was not associated with the quality of the studies, but articles which were funded by pharmaceutical companies did have a higher score than the studies funded by health research organisations. The category of intervention did not have an influence on the quality of the study. Critical appraisal of economic evaluations Quantifying the quality of economic evaluations on obesity May 2007 Master Thesis HEPL Z. Kortram 272598 4 Statistical analyses show a correlation between certain items and the characteristics, but the associations are mostly due to a natural association with the characteristics. A separate item cannot determine the quality of the publication. Conclusion: The main conclusions which can be drawn are the following: • The quality of the economic evaluations on obesity increases as time progresses. • Studies which are based on previous publications have a higher quality than those which are based on a single study. • The location in which an evaluation is performed also affects the quality of the evaluation. • A single criterion of the assessment tool can yet not be used separately to determine the quality of the economic evaluation. The mean quality score of the articles shows that on average the economic evaluations on obesity interventions are not of high quality. But the use of the quality assessment tool can give more insight in the areas in which improvements can be made. The quality assessment tool can be useful before a research is conducted or an article is submitted, but afterwards the tool is also useful to assess the quality of the actual publication. This tool can also affect current policies and improve the way in which interventions aimed at decreasing obesity are implemented in the current interventions programmes. Discussion: There are several recommendations which can be made. More information is needed on the type of scale which is most useful when appointing the weight. It can also be relevant to know what information can be gathered when the assessment tool is used in other disease areas. Perhaps it would be useful to determine if the assessment tool can be adjusted to the disease or risk factor which is examined.