Validation of tools for the evaluation of pharmaceutical services : A systematic review

The objective of this study was to identify and evaluate studies that performed the validation of tools applied to health evaluation with a focus on pharmaceutical services. A systematic literature review was conducted to identify validation studies involving pharmaceutical services. The search for articles strategy was held in databases: EMBASE, PubMed/Medline, Scopus, and CINHAL. The descriptors used were: ("Pharmaceutical services" or "Pharmacy Services") and ("Validation Studies" or "Evaluation Studies"). The results showed that 18 community pharmacies (78%) were leading studies realization scenarios. Regarding the data extraction strategy, the questionnaire was the strategy most commonly used with 20 (86%). With regard to the objectives of the studies, 13 (56%) pointed to the development and validation of tools for verification of patient satisfaction with pharmaceutical services. 20 (86%) studies carried out the analyses of reliability and validity and only one realized (4%) singly the reliability analyses. The reviewed studies did not indicate the potential of their instruments in interventions to improve pharmaceutical services as well as the potential beneficial effects of these interventions. Thus, research is needed to develop and validate instruments directed at pharmaceutical services, especially in hospitals, in order to improve the quality of services provided to users, influence users’ quality of life, and decrease the demand for health services.

patient care services, a practice that takes up more space in the pharmaceutical assignments and requires professional expertise in the management of pharmacotherapy and therapeutic outcomes can be mentioned (Correr et al., 2009).
According to the health ministry pharmacies structuring guidelines, pharmaceutical services are divided into: the technical and managerial services (programming medicines, application process and storage, and activities related to the disposal of waste from health services) and pharmaceutical services technical assistance (dispensation, pharmaceutical care, pharmacotherapeutic follow-up, education actions in health and technical support for health staff) (Brasil, 2009).The purpose of pharmaceutical services is to achieve the best results possible (Health) and improve the quality of life of individuals, families and communities, contribute to the promotion of healthy habits for the population, and promote rationality of medicalization (OPS, 2013).These services are considered a key process as it relates to the direct provision of public service to the end and so contributes to the achievement of health outcomes.
Starting from the premise that the quality of a service must be related to the adequacy of their activities to the needs of the context in which it operates, health quality cannot be evaluated or judged only in technical terms by health professionals.It is necessary to recognize individual preferences and social, seeking to equate them in ensuring equity (Norman, 2012).Therefore, it becomes necessary to conduct ongoing assessments of services and develop tools to ensure the implementation of these services with quality.Therefore, it has become necessary to constantly evaluate health care services and develop tools to ensure that these services are implemented with quality (Vituri et al., 2009).
These sense health service assessment strategies have become an important tool to allow the rationalization of decisions and practices, as the results may be useful to facilitate decision making by managers and support necessary interventions.Currently, there are increasing numbers of questionnaires and scales available in health seeking to ascertain and assess the various research conducted (Alexandre et al., 2011).However, it is imperative that these tools have reliability and validity to minimize the possibility of subjective evaluation (Raymundo, 2009).Thus, the literature has alerted the investigators about the correct evaluation of the quality of the data collection instruments, for the recognition of the quality of the instruments is critical to the legitimacy and credibility of the results of a search (Medeiros et al., 2015).
In this scenery, the validity consists in analyzing whether an assessment instrument measures offered to him is a key determinant factor in the choice and/or application of a measuring instrument and is measured by the extent or degree to which represents the concept the instrument proposes to measure (Bittencourt et al., De Jesus et al. 955 2011).Validity tests ask whether they measure actually the attributes behind it.The fact that there are no a health, the gold standard in which the results can be compared, validation methods normally use the criteria accepted by behavioural sciences (Lima et al., 2013).The use of psychometric analysis performs the assessment of the quality of the instruments by middle of denominated psychometric properties of variables, among which the reliability and validity were highlighted (Pilatti et al., 2010).Therefore, validity is more than a statement of value of a measuring instrument is an extensive process of research.Consider this, Pasquali (2009) didactically divided the validation process into three parts: content validity, construct validity, and criterion validity.
The validation of content is determined by the trial judges also entitled to experts, the extent to which the rated instrument adequately represent all dimensions of the concept to be measured (Moura et al., 2008).Already, the construction and its validity tests are evaluated by the interrelationship of the studies, statistical tests and the correlation of the theory with the variables to be measured (Raymundo, 2009).The criterion validation process describes the relationship between a measure and an objectified criterion is related to a factor that can receive the influence of other factors not integrated into the main variable, which can alter the amplitude of the validity coefficient (Lima et al., 2013).
The other step of the analysis is the psychometric reliability.As regards the extent to which the instrument measures repetitions, evaluated relatively stable are located close to each other and may be verified by testing specific (Vituri et al., 2009).However, the search for quality in research reflects the concern in analyzing the results of different studies conducted with the aim of achieving excellence and quality required for the search of the best results in health (Medeiros et al., 2015).As mentioned earlier, this study aimed to identify and evaluate a systematic review of studies that performed the validation of tools applied to health evaluation with a focus on pharmaceutical services.

METHODOLOGY
A literature review was performed to identify validation studies of measures of various aspects of pharmaceutical services.A search of the following databases was performed: EMBASE, PubMed/ Medline, Scopus, and CINHAL.To identify articles, the following descriptors: ("Pharmaceutical services" OR "Pharmacy services") and ("Validation Studies" OR "Evaluation Studies") were used.Searches were conducted from May to June, 2015.The articles published were included by May 2015, unrestricted by date of publication.
In order to meet the inclusion criteria, studies had to be original works published in English, Spanish, or Portuguese.Articles should address validation of pharmaceutical services assessment tools.Articles that did not have abstracts available were excluded from the review and articles referenced in two or more databases were considered only once.Differences in selection were resolved by discussion with a third reviewer and decisions were made by consensus among the three reviewers.
A seven-step systematic review was performed according to the recommendations of the Cochrane Handbook: (1) formulating the questions (the kinds of services to be evaluated and validation tools, in other words, the psychometric constructs evaluated), (2) location and selection of studies (databases mentioned earlier), (3) critical review of the studies, (4) data collection, (5) analysis and presentation of data, (6) interpretation of results, and (7) refining of conclusions (Higgins et al., 2006).The methodological quality of the validation studies was analyzed by two independent reviewers, based on the Standards for Diagnostic Report Accuracy (Bossuyt et al., 2003); disagreements were resolved by consensus.The degree of agreement in the assessment of titles and abstracts was measured using the kappa coefficient, considering a confidence interval of 95% (Feinstein et al., 1990).The articles that met all inclusion criteria were assessed by the country of production, the scenario in which the study was performed, type of study, participants, duration of the study, key findings, and limitations pointed out by the authors.The type of instrument and psychometric characteristics used for validation in each review article were assessed.

RESULTS
The initial search identified 1830 studies.After exclusion of repeated articles, 1544 eligible titles were considered potentially relevant, and their abstracts were reviewed.After reviewing the abstracts, 105 articles were preselected for evaluation of the full text.Of these, 82 were excluded for the following reasons: the full-text version 45 was not available in the databases, 19 studies only evaluated pharmaceutical services and 18 had not presented data on content, construct, or criterion validity or internal consistency/test-retest reliability.
A substantial agreement was found among raters in the evaluation of abstracts ( = 0.786) and full texts ( = 0.923); this agreement was statistically significant (p < 0.001).Figure 1 shows the selection process and the number of articles in each stage.

DISCUSSION
Ever since the need for the use of valid instruments was identified, many countries are developing and implementing policies for dispensation of medications (Armando et al., 2008;Quispe et al., 2011).It was found in this review that countries in Europe and the U.S.A are working on strategies to improve the quality of pharmaceutical services (Armando et al., 2008;Quispe et al., 2011;Delgado et al., 2009;Allenet et al., 2006;Horvat et al., 2010;Njilele et al., 2012;Sakharkar et al., 2014).Cosendey (2003) states that the process of improving the quality of service still lacks a systematic approach to evaluating drug policies.Thus, there is a need for the development of tools that enable effective monitoring of the implementation of national and international policies of medication dispensation in order to evaluate performance and review priorities.
Regarding the setting of the studies, the community pharmacy was identified as the main venue for pharmaceutical services (Armando et al., 2008;Quispe et al., 2011;Tamargo et al., 2006;Delgado et al., 2009;Martin et al., 2010;Skomo et al., 2009;Ngorsuraches et al., 2008;Fang et al., 2011;Feletto et al., 2011;Armando et al., 2009;Jocic et al., 2014;Sriram et al., 2014).One of the strong pieces of evidence for the confirmation of this result can be linked to the structure of pharmacy.The community pharmacy is easily accessible and serves as an environment that allows good communication between the patient and pharmacist, leading to an increase in the level of trust between the professional and the patient (Quispe et al., 2011).However, this review found that few studies have addressed pharmaceutical services in hospitals (Allenet et al., 2006;Gourley et al., 2001;Njilele et al., 2012;Sakharkar et al., 2014;Al-jumah et al., 2014).There is a need to develop tools to evaluate hospital-based pharmaceutical services, as these tools can serve as a guide to improving the quality of service in daily practice.This recommendation is supported by Allenet (2006) and Gourley (2001), who reveal that the use of these evaluation tools enhances the aspirations of health professionals to improve care, and at the same time provide an idea of how this care is being perceived by the patient.For the development of measures, the studies analyzed in this review used references to previous literature or adapted instruments already in use in other countries.However, the majority of the articles analyzed did not report the type of methodological design used in the study (Armando et al., 2008;Tamargo et al., 2006;Delgado et al., 2009;Allenet et al., 2006;Horvat et al., 2010;Young et al., 2011;Martin et al., 2010;Skomo et al., 2009;Desrochers et al., 2011;Ngorsuraches et al., 2008;Feletto et al. 2011;Njilele et al., 2012;Williams et De Jesus et al. 963 al., 2012).According to Rodrigues (2004) and Polit (2004), study design should be chosen in consideration of the human and material resources available and collected, the duration of the study, the representativeness of the population being studied, ethical issues, selection criteria, and possible biases.In this sense, the articles presented are limited in their reproducibility, as only studies using a cross-sectional design describing their methodologies (Quispe et al., 2011;Fang et al., 2011;Azeredo et al., 2009;Armando et al., 2009).As such, reproduction of these studies in other settings can become difficult to perform or be confusing for the researcher.
As for sample size, there was variation between studies, which may be explained by the diversity of sites where the research was performed, as well as different validation methodologies used by the authors.You can stand out negatively in the study (Armando, 2008) that failure to submit the sample size factor which limits the interpretation and even the majority of studies.According to Fontanella (2008), when it comes to psychosocial issues, the performance of an attribute should reveal functions or representative characteristics that context, perhaps, why the number of individuals appears to secondary way in some studies.Despite this secondary importance, the establishment of a sample size is inevitable as a methodological error in this final number of which the establishment may undermine the credibility of the findings and analyses.
Regarding research participants, patients and pharmacists were the most referenced.The participants indicated in the studies showed various characteristics, such as chronic degenerative diseases (diabetes, hypertension), infection with HIV, and experience of menopause.However, the studies do not clearly report their inclusion criteria.This failure may be related to the question of evaluating the service provided to these individuals.This is done once the patient has taken a starring role in the health system, that position has a direct impact on improving the relationship between him or her and the pharmacist (Ramos, 2003).On the other hand, Jocic (2014) in their study points out how the attitudes and beliefs of health professionals about their work with patients, can affect quality of health care showing improvement in clinical, humanistic and economic results.Thus, it is essential to know how users evaluating the service provided to them, for rethinking professional practices and intervene on the form of organization of services, to its improvement.
In relation to the objectives in the studies analyzed, was the development and validation of questionnaires for verification of patient satisfaction.In this sense, Armando (2008) and Larson (2002) state that user satisfaction is defined by observing the services provided.Azeredo (2009) suggests that it is the quality of the service itself that is responsible for determining satisfaction.However, Panvelkar (2009) showed that the result of the service is not always included as the main criterion for the evaluation of services.Criteria for evaluating the pharmaceutical service should not be restricted to single assessment of how well the services are provided, but must also focus on how well the services meet the needs of patients.Therefore, it is necessary that the tools developed are validated to encourage research on patient satisfaction with pharmacy services, given that the focus of pharmacy services, including within the hospital setting, has expanded beyond dispensing drugs to providing other services (such as assessment of therapeutic regimens, development of monitoring plans, and assessments of physiological parameters).
The type of instrument most commonly used in the validation process of the studies was the questionnaire.With the expansion of services, it has become increasingly difficult for pharmacists to quantify the value of their services, so the use of a simple, practical, and useful questionnaire could easily reflect the level of patient satisfaction with professional services (Armando et al., 2008).The purpose of using questionnaires is to provide a voice to patients in order to assess and improve the services they are receiving.
In this respect, Collins (Rodrigues, 2004) states that questionnaires are useful for measuring patients' reactions to improvements or changes to the service.The reviewed articles did not make clear the duration of studies nor described the methodology used to arrive at consensus on the validation process, only mentioning the constitution of the panel of assessors (judges), changes in the questionnaire, and application to the target audience.According to Wright (2000), Giovinazzo (2001), andCardoso (2005), the usual duration for the validation process using the Delphi technique, which is a systematic method of trial information, is four months to a year, depending on complexity of the subject and the instrument.In this sense, it is necessary to consider all specialties involved in consensus about content validity, because the opinion of experts (evaluators) brings to the researcher constructive feedback on the quality of the measure, as well as solid suggestions for its improvement.
In the reviewed articles, reliability was analyzed in terms of internal consistency; using Cronbach's α.Studies presented alpha values between 0.70 and 0.90, indicating that the items included were appropriate for the measurement instrument (Quispe et al., 2011).The articles did not measure the correlation between alpha and other parameters.Lee (2005) andHair (2002) suggest that this may be linked to the fact that results obtained by the alpha coefficient are broad and based on the internal consistency; therefore, researchers did not doubt the reliability of a measure.However, the existence of redundant or unnecessary items that measure the same aspect of a concept cannot be ruled out, since this aspect is often not disclosed by the researcher in order to obtain high alpha values (Ladeira, 2010).
The articles also addresses the issue of statistical tests by presenting the results of concordance between raters on the items of the measurement instruments, mainly using factor analysis to test construct validity and calculating internal consistency (Cronbach's α) for reliability.According to Pasquali (2009), validity involves whether a test is a legitimate and its results are appropriate representation of the construct of interest.Reliability, in the view of the same author, is related to the test's ability to capture the variations in real measurements in a population through reproducibility of results and internal consistency.This view corroborates Pillati's (2010) opinions on the reliability, stability, and accuracy of the results of a test.In this sense, it appears that the studies conducted jointly psychometric characteristics, increasing its quality and providing strong evidence as to be the most appropriate form of survey data available for this purpose.
The articles studied in this review pointed to the lack of validated instruments in the area of pharmaceutical services.With respect to the testing of measures, the articles bring up the issue of the pilot test, which is conducted before the implementation of the final version of the tools developed (Norman, 2012;Vituri et al., 2009;Raymundo, 2009;Bittencourt et al., 2011;Higgins et al., 2006;Feinstein et al., 1990;Quispe et al., 2011;Young et al., 2011;Njilele et al., 2012;Sakharkar et al., 2014).The process of validation of an instrument defines the ability to measure the same observation in the study, so this observation will affirm if the chosen variables and the theoretical concept being measured were adequate or not (Lima et al., 2013).The implementation of the pilot test can be highlighted positively in the articles included in the review, since this step allowed a prior assessment tool to be applied to a small sample of participants, and permitted evaluation of aspects of its administration and correction of any errors before its final deployment.

Conclusion
This review sought to contribute to addressing a topic relevant to improving the quality of studies related to public health: the development of the process of validation of instruments that evaluate pharmaceutical services.Although, there are different instruments that can be used to evaluate health services in general, there is not yet an instrument that can measure pharmaceutical services in all areas, and that suits all studies so that their results can be compared.
The analysis of variables of the studies demonstrated that the field of pharmaceutical services still needs to be explored, as they are still confounding these services with the actions already implemented in the daily example of the pharmacy dispensing drugs.The reviewed studies did not indicate the potential of their instruments in interventions to improve pharmaceutical services as well as the potential beneficial effects of these interventions.
The limitations of the types of features and study of the various methodologies proposed in the work should guide future studies in this field, helping them to adopt a more comprehensive approach for validating measures.Thus, research is needed to develop and validate instruments directed at pharmaceutical services, especially in hospitals, in order to improve the quality of services provided to users, influence users' quality of life, and decrease the demand for health services.

LIMITATIONS
The present study had some limitations.First, the number of keywords was limited due to the primary purpose of the review, and may restrict the data generated.Moreover, the selection strategy of databases and the restriction of the language of publication may have excluded some very important studies that were not published in the sources and languages used.

Figure 1 .
Figure 1.Selection process and the number of articles in each stage.

Table 1 .
General characteristics of the studies included in the review.São Cristóvão -SE, 2015.
NR: Not reported.

Table 2 .
Methodological description of the articles included in the systematic review.São Cristóvão-SE, 2015.
NR: Not reported.