Assessing the Quality of Health Economic Evaluation Research by CHEERS Instrument : A Critical Literature Review in Laos , Cambodia , and Myanmar

Article history: Received on: 26/09/2016 Accepted on: 24/01/2017 Available online: 30/06/2017 Although economic evaluations play vital roles in healthcare setting, omissions of these reports in Laos, Cambodia, and Myanmar make themselves not to be relied or trusted. Therefore, this study appraises deliberately the quality of inside information to decide which report can be used for assisting policy-makers. It was conducted in August 2016, using PubMed, Science Direct and Cochrane databases for searching economic analyses published from 1996 to 2016 with following terms either alone or in combination: economic evaluation, cost-effectiveness analysis, cost-benefit analysis, cost-minimization analysis, cost-utility analysis, Laos, Cambodia, Myanmar. Qualified articles are independently assessed by two reviewers based on Consolidated Health Economic Evaluation Reporting Standards (CHEERS) checklist. 1285 articles citations were identified, but only 17 articles left after the selection. These articles are classified into four quality levels considered as good (n=4), fair (n=6), average (n=4) and below average (n=3). The mean percentage of recommendations fulfilled by papers is 71.3%. In brief, economic evaluations in these countries need to be increased in quantity as well as improved in quality. It is better to standardize by CHEERS guidelines and invest more funding supports for reports.


INTRODUCTION
Myanmar, Laos and Cambodia are developing countries located in Southeast Asia.According to the World Bank website, they are both lower-middle income countries.Myanmar (Than Tun Sein (2014), Laos (Akkhavong, 2014) and Cambodia (Annear, 2015) have been in process of undergoing a healthcare reform.Nevertheless, the healthcare resources are limited.Therefore, it is necessarily needed the adequate, accurate and credible information for making decisions which can lead to economic and health outcome effectiveness in order to guarantee a successful healthcare reform.
Economic evaluations are increasingly being recognized as a fundamental component in the decision-making process for healthcare resource allocation.Economic evaluations have been widely applied in health policy, including the assessment of prevention programs (such as vaccination, screening and health promotion), diagnostics, treatment interventions (such as drugs and surgical procedures), organization of care, and rehabilitation (Husereau et al., 2013).Because of its vital role in healthcare settings, the provided information in economic evaluations must be accurate and reliable.
There are many published guidelines in order to improve and optimize the reporting of health economic evaluations (Drummond and Jefferson, 1996;Nuijten et al., 1998;Ramsey, Willke et al., 2005;Davis et al., 2011), but especially the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) Statement.CHEERS Statement is being simultaneously published across 10 health economics and medical journals in 2013.
It is an attempt to consolidate and update previous health economic evaluation guidelines into one current useful reporting guideline.Previously published checklists or guidance documents related to reporting economic evaluations were identified, and a list of possible items from these efforts was created.A two-round, modified Delphi Panel consisting of representatives from academia, clinical practice, industry, government, and the editorial community was used to identify a minimum set of items important for reporting from larger list (Husereau et al., 2013).With recommendations carefully selected from health economic journal editors and content experts from around the world, CHEERS Statement promises to be an universal tool for health economic evaluation studies.
An uncontrollable increase in the number of such studies in developing countries has rationally led to the quality decline (Gavaza, 2008;Teerawattananon, 2007).Recent systematic reviews conducted in two other Southeast Asian countries including Thailand (Teerawattananon, 2007) and Vietnam (Trung Quang Vo, 2016) reported that the quality of some economic evaluation research studies was poor and serious attention needs to be given to the quality of reporting and analysis.However, little information is known about the quality of health economic studies in Myanmar, Laos and Cambodia.In fact, it has not had any systematic review studies related to health economics conducted in three countries up to now.This study is aim to appraise deliberately the extent of meeting particular methodological and reporting practices based on CHEERS statement of economic evaluation studies conducted in these three countries and performed outcomes clearly to assist policy makers.Besides, we analyze the trend and current situation of the literature on health economic evaluations in these countries.

Study design
The structure is designed as systematic review of articles published from 1996 to 2016.Contents are followed by PRISMA guidelines published in 2009 (Moher et al., 2009).

Search strategy
A literature search was conducted in August 2016 from three reliable databases including PubMed, Cochrane Library and Science Direct for studies published in Laos, Cambodia and Myanmar in the last 20 years (from 1996 to 2016).These following keywords were used to identify health economic evaluation studies: cost-effectiveness analysis, cost-benefit analysis, cost-utility analysis, cost-minimization analysis, economic evaluation, Laos, Myanmar and Cambodia.For PubMed database, keywords were added to advanced search builder and searched in Abstract/Title.Meanwhile, for Cochrane Library and Science Direct database, such terms were found in three sessions Abstract, Title and Keywords.Reference lists of qualified articles were also considered for additional reports.This step will be finished when no more articles could be found.

Exclusion and inclusion criteria
Included articles must be a complete peer-reviewed publication which concentrates on health economy field in Laos, Cambodia and Myanmar.In addition, the chosen one must be related to human beings and be written in English language.On the other hand, exclusion criteria were mentioned to minimize mistakes happening in search process.Studies were excluded if they are editorial, systematic reviews or methodological articles and there were no full text found.The rejection was also applied to publications conducted in both human and other species.Likewise, publications were also excluded if they were implemented in both Laos, Myanmar, Cambodia and other Asian countries.Not conducted in humans, not in health economy and not in Laos, Myanmar and Cambodia.Two blinded reviewers screen the title and keywords of all identified articles in order to determine potential ones.The next stage of filter process is reading abstract and full text to ensure articles that did not match all of exclusion criteria.Finally, qualified studies were obtained for full evaluation.The flow chart of selection process is performed in Figure 1.

Assessment procedure
The assessment procedure is divided into two main tasks.The first one is using recommendations in Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement to assess the quality of studies.The second mission is gathering all characteristics of qualified studies in data collection form.

Quality assessment using the CHEERS instrument
CHEERS checklist which consists of 24 recommendations is a useful, practical tool to improve report's quality and healthcare decisions in turn.Because these recommendations consist of many contents which are hard to be completely gained, and detailed and specific, all authors had to determine which situation is "Yes" and which is "No" at the beginning.In particular, we assumed that studies containing over a half of contents in each item would be assessed as "Yes", or, conversely, the result was "No".After reaching agreement to each other, two reviewers read two randomly selected articles and then examined differences on items' ratings.Disagreements among two primary reviewers were solved by the third researcher.In this session, we did statistic the recommendations fulfilled by each article and the number of articles which had relative content for each recommendation.

Characteristic of included studies
General characteristics and specialized properties of 17 studies are also synthesized in data collection form.General characteristics are information about authors, primary training of first author, country covered, published year and type of journal.Specialized properties consist of study design, time horizon, currency, type of analysis, intervention, outcomes, perspective, and type of data used, cost included and sensitivity analysis.Such information is used to detect the impact of categorical study characteristics on articles' quality.

RESULTS
From the beginning, 1285 articles were obtained from PubMed (n=812), Science Direct (n=289) and Cochrane Library (n=184).In order to avoid the duplication, 1285 articles were imported to Endnote software and arranged to duplicates removal.The result was 189 articles rejected, therefore, the remaining 1096 were moved to the elimination stage by screening abstract, title and keywords.Unfortunately, there were 1012 articles not relating to health economic evaluation or dismissing inclusions such as English language, targeted countries.Next, the full text of 84 potential articles were read carefully to determine which was chosen.Studies were excluded if they were systematic reviews (n=3), out-of-pocket expenditures (n=2), sensitivity analysis of cost (n=1), conducted in Southeast Asia (n=2), only provisional abstract (n=2) and were not original economic evaluations (n=57).Consequently, the study's final sample consisted of 17 qualified articles.

General characteristics of included studies
As we can see from Table 1, Cambodia is the country covered by included studies with the highest rate (n=10), whereas Myanmar had six researches and Laos had only one study.The total number of studies in each country is extremely scarce, moreover, there were almost not any points of time when health economic studies were simultaneously published in both three countries.The period from 2005 to 2014 had a stable increase in number of studies in Cambodia (n=10) with an average number was one per year.After 15 years since 2000, three studies were simultaneously published in 2015 in Myanmar.In 2016, both Laos and Myanmar had one research found.The authors of studies are foreigners, except two researches (Touch et al., 2010;Cho Min and Saul, 2000) conducted by native professors.

Quality of studies assessed by CHEERS instrument
The adherence to items and recommendations of CHEERS of health economic evaluation studies conducted in Laos, Myanmar and Cambodia varied widely.As the statistics shown in Table 2, the number of items and recommendations gained by each study ranges from 11 to 22, the average number is 17 (approximately 71.3% over the total of 24 items to include in CHEERS checklist) with the standard deviation is 3.The highest digital belonged to a study implemented in Myanmar and published in 2015.Meanwhile, the equivalent digital of the lowest article was only 11.The article was conducted in Cambodia and published in 2009.This is because of the lack information of cost data input, health outcomes as well as the shortage in time horizon (less than 7 months).Ten studies (58.8%) met 17 (70.8%)recommendations or even more than.Overall, these results indicate that the quality of health economic evaluations is generally fair, but not good enough.Among twenty-four recommendations, there are some items presented by high percentage of studies whereas some items were almost neglected.In details, the introduction and discussion were presented in 17 (100%) studies, but the characterizing heterogeneity session was shown in two studies only (11.8%).Besides, nine following parts: Title, Abstract, Target population, Time horizon, Study perspective, Comparators, Choice of health outcomes, Estimating resource and cost, Incremental costs and outcomes, Uncertainty analysis were mentioned in more than 14 (80%) studies.On the contrary, two sessions: Conflicts of interest, Measurement and valuation of preference based outcomes were considered in 5 (29.4%) studies and 4 (23.5%)studies, respectively.

Extent and trend of the literature on health economic evaluation
The absolute number of economic evaluation articles published in Laos, Cambodia and Myanmar over the 20-year period  covered in our study was small.It is because of numerous factors such as-lacking of appreciation and expertise in economic evaluations, lacking of collaboration between   researchers, government and decision-makers (Akkhavong 2014;Sein 2014;Annear 2015).Moreover, conducting health economic evaluations is expensive and time consuming.Because of the superiority of these useful tools in health resources allocation, these three countries 2010) need to provide more training opportunities for health economists, train health professionals and policy makers how to manage and use economic data effectively, or promote the application of these analysis in reality.It is also suggested that capacity building local scientists in the conduct of economic evaluations should be a priority area in the future.Summarizing diseases appeared in studies, Figure 2 obviously indicates that infectious diseases dominated the chart with the rate 12 of 17 studies (70.6%).In fact, World Health Organization (Gupta and Guin, 2010) announced that low-income countries lately have a higher share of deaths from: (i) HIV infection, tuberculosis, malaria, (ii) other infectious diseases.Therefore, all three countries are concentrating on costeffectiveness of interventions treating infectious diseases.For example, malaria problems are decreased by using bed-nets (Drake et al., 2015), taking rapid diagnostic test (RDTs) (Chen et al., 2015) or using alternative drug regimens (Cho Min and Saul, 2000).In 17 selected articles, there are four types of interventions applied, consisting of curative interventions, preventive methods, diagnosis programs and service deliveries.Preventive methods (mainly vaccination) and diagnosis programs mainly suggest intensive and extended protection for high-risk population, whereas, service deliveries increase the approaching and afford of all residents population to health care.Although it can't be denied the prevalence of infectious diseases in developing countries, the absence in allocating money for chronic diseases such as cardiovascular, diabetes and cancer can increase death incidence in near future.It is better to invest more in modern contemporary illnesses (cardiovascular diseases, diseases, diabetes, cancer) and balance financial sources for all diseases.Another significant property of these reports is type of economic evaluation.Similar to findings from other developing countries such as Thailand (Teerawattananon 2007), Vietnam (Trung Quang Vo 2016), the study results showed that CEA was the most popular type of economic evaluation used.Of four types of analysis including cost-effectiveness analysis (CEA), cost-utility analysis (CUA), cost-benefit analysis (CBA) and cost-minimization analysis (CME), only two studies (Evans et al., 1996) (Kingkaew et al., 2016) used CUA.This is in contrast with publications in developed countries where CUA has become an important measure used by medical decision makers (Drummond et al., 2008).CUA whose consequences are measured in term of preference-based measures of health, such as quality-adjusted lifeyears (QALY) is often considered as a special case of CEA.This technique is preferred by many economists because it incorporates the utility of the output (Torrance, 1989).CUA attempts to overcome some of the problems of cost-effectiveness analysis but it is more difficult to understand and is more expensive to apply (Lewis, 2004)

Quality of health economic evaluations
The big difference between fulfilling percentage of recommendations can be explained by the importance and the necessity of each factor in economic evaluation studies.Introduction and Discussion were presented in 100% studies because of the indispensable role of two parts.It can be said that Introduction session is the compass for deviating general understandings of the report, whereas Discussion is the place exploiting specific comprehensions about study's findings and providing deep insights for readers.A good report must draw the broad context of the study (based on previous researches and current health policies) and objectives of the study from the beginning as well as explain deliberately results in discussion.Besides two parts above, guidelines for the Economic Evaluation of Health Technologies in Ireland (Ryan, 2010) consider perspectives, comparators, outcome measurements and sensitivity analysis as main elements must be clearly stated.On the contrary, the data heterogeneity and interest conflicts scarcely occurred in economic evaluation studies.Therefore, these parts were frequently neglected in reports, but did not cause any remarkable impacts on researches.
In this study, there is no scoring scale applied.In order to avoid the subjective rating of reviewers, the quality is assessed based on the percentage of recommendations that each study satisfied.According to the statistic in Table 2, the mean percentage of qualified recommendations in each study is 71.3%, especially, only one study in Cambodia published 2009 (Gosselin et al., 2009) has the percentage lower than 50%.Assume that 100% equals to 10 points, we divided the mean percentage by 10 and exchanged it into specific score.By this way we could convert the mean score percentages of economic evaluation studies in Laos, Cambodia and Myanmar into a 10 point scale.As a result, the mean score of seventeen studies is approximately 7.13.Although it is slightly higher than the mean score of economic evaluations in Zimbabwe (5.40) (Gavaza et al., 2008), it is lower than studies in Nigeria (7.29) (Gavaza et al., 2010), South Africa (7.59) (Gavaza et al., 2012) and India (7.80) (Desai PR1 2012).Therefore, health economic researches in Laos, Cambodia and Myanmar need to be enhanced and improved expertise.Moreover, two studies having native primary author are not good in quality (gaining 62.5% and 66.7%).This result asserts that native residents and scientists in three countries are lacking knowledge about health economic analysis.Therefore, educating economic analysis methodology and training cost benefits for medicinal scientists are two essential prerequisites.Moreover, standardization health economic reports and conducting research based on guidelines can minimize unnecessary steps in testing process as well as avoid accident difficulties.Reviewers scored each study based on how these criteria were met.In order to have an overview of quality of health economic evaluations, we divided these studies into four levels depending on the number of items and recommendations each study gained.We categorized researches below average reporting (13 or less recommendations satisfied), good reporting (20 or more recommendations) fair reporting (17 to 19 recommendations), average reporting (14 to 16 recommendations) (see Table 2).It is somewhat surprising that a large percentage (n=13, 76.5%) of the studies we reviewed were fair, average, below average, and only four of the 17 studies (23.5%) were good reporting.Although these estimates and division of quality scale may express our subjectivism, the shortcomings in reporting health economic evaluations in Laos, Cambodia and Myanmar were substantial enough to warrant attention.The low quality of studies might negatively affect the usefulness of health economic evaluation research in policy formulation in these countries.The quality of the studies was significantly associated with the year of publication.Ten studies have highest percentage of qualified recommendations were published in the period from 2014 to 2016.It can be explained that the focus on improving the quality and investing economic evaluation in Laos, Cambodia and Myanmar have been promoted by researchers in the recent year.Furthermore, more standards and guidelines for reporting health economic evaluations have elevated the quality of these researches.

LIMITATIONS
This study encountered some limitations in searching and assessing health economic articles.Firstly, the evaluation and rating each recommendation of two authors was progressed independently and simultaneously.Therefore, we could not prevent all of differences in authors' perspectives, which caused subjective conflicts.Besides, the harshest obstacle in assessment process was lacking of point scale.There has not been an official guideline for determination how to be good, how to be bad and how much to be relied.In this study, we suggested four levels in quality with different requirements of fulfilling percentage.

CONCLUSION
This literature review on economic evaluations is the first study to be implemented in context of Laos, Cambodia and Myanmar.The findings of this review confirmed obviously that the economic evaluation in three countries is still at an early stage.and it has an urgent need for improving the quality of reporting economic evaluation in these countries.The research has also shown that building of local scientists in the conduct of economic evaluations should be a priority area in the future.Therefore, more focuses on teaching, training health professionals and policy makers in utilizing economic data are required.To promote the effectiveness of health economic evaluations in the country condition, these studies should be established by collaboration between researchers across disciplines, and in communication with policy makers as well as having more funding support from health organizations.
Financial support and sponsorship: Nil.

Fig 2 :
Fig 2: Types of diseases and relative interventions.

Table 1 :
Included study distribution by years and countries.

Table 2 :
Number of fulfilling recommendations in each study.

Table 3 :
Economic evaluation assessed by CHEERS statement.