Preliminary Validation of a Patient Satisfaction Instrument in the Emergency Department

Statement of the purpose: The aim of this study was to develop a valid and reliable satisfaction tool that can be utilised in the emergency departments (EDs) of hospitals throughout Sharjah, United Arab Emirates (UAE). Methods: The study followed a cross-sectional study design. The study was conducted during the period from October 2018 to January 2019. The participants were conveniently sampled. The total number of eligible questionnaires for analysis accounted for 207. The data collection tool was developed following a review of literature which yielded 25 statements. Satisfaction levels were measured using a 3-point Likert scale (satisfied=3, do not know=2, dissatisfied=1). The tool was validated through face validity performed by the research team. Content validity performed by a panel of nine randomly selected specialists. Principal components analysis was done to extract the relevant components to the statements on the tool. Results: Scale content validity index= 0.836. Principal components analysis with oblique rotation extracted three components namely; medical staff performance, duration of the encounter, and general impression about the emergency department. Internal consistency for the tool using the split-half Cronbach’s alpha, part 1=0.80, and part 2=0.82. Conclusion: The findings of the present study support the reliability and validity of the Emergency Services Patient Satisfaction Questionnaire for intended use, following an independent sample of patients at EDs in Sharjah, UAE. The scale is recommended for assessing patient satisfaction with service provision to help hospitals in Sharjah determine to what extent they are meeting the needs of their patients.


Introduction
One of the World Health Organization's (WHO) six building blocks of health systems is the delivery of health services that are effective, safe, and of good quality for those who need them. Emergency care is viewed as an essential contributor to reducing avoidable deaths and disabilities, and thus mandates greater attention. Statistics show that the number of emergency department (ED) patients is steadily increasing. Therefore, it is of utmost importance to plan quality services based on the needs of these patients. Patients' satisfaction is an important quality-of-care indicator for service delivery by ED personnel. Since a patient-centred approach is now widely recognised in the field, focusing on patients' satisfaction is necessary to develop insight into how patients perceive care (Aiken, Sloane, Bruyneel, Van den Heede, & Sermeus, 2013).
Measuring and assessing patient satisfaction nowadays is considered one of the core components of quality assurance and accreditation of hospitals and clinics worldwide. Such activity along with the interpretation of yielded outcomes takes place on regular basis (Worku & Loha, 2017). Patient satisfaction is the level of happiness that clients experience having used a service. It measures the gap between the expected service and the actual experience of the service, from the client's point of view (Assefa, Mosse, & Michael, 2011).
At a hospital level, efforts to provide quality service is usually challenged by a taxing flow of patients; they are further suppressed by the urgent nature of care in the ED. Assessing patients' satisfaction as a quality-of-care indicator is thus required to monitor the non-technical aspects of quality of care in these settings (Abdosh, 2006). Satisfaction is a multidimensional concept that requires measuring collectively in all dimensions to assess the overall satisfaction of an individual (Worku & Loha, 2017, Abdosh, 2006. Among the factors influencing patient satisfaction, the relationship between healthcare providers and patients was reported to be the most influential. Meanwhile, the expectation of patients about the services, including the perceived adequacy of consultation duration, a welcoming approach, and perceived body signalling, are considered to be determinants of satisfaction (Tateke, Woldie, & Ololo, 2012). Other determinants of patient satisfaction in the ED have been identified as waiting times; which is considered a global determinant in patient satisfaction studies, in addition to treatment and nursing care, and the information received from nurses and physicians (Grogan, Conner, Norman, Willits, & Porter, 2000).
As a rule, all studies require valid and reliable tools for measurement, and measuring patient satisfaction is of no exception. Such characteristics of accuracy and consistency of a tool are necessary for providing meaningful and trustworthy results. Validity of a tool is defined as how well the study captures the measure of interest, while reliability means that a measurement is reproducible over time, in different settings, and by different raters which includes agreement among different raters (inter-rater reliability) (Zapf, Castell, Morawietz, & Karch, 2016).
Principal component analysis (PCA) is a validation procedure that extracts a number of probably correlated variables into a smaller number of uncorrelated variables. The main goal is to extract the important information from the data table and to express this information as a set of new orthogonal variables, called principal components (Osborne & Costello, 2004).
The aim of this study was to develop a valid and reliable satisfaction tool that can be utilised in the emergency departments (EDs) of hospitals throughout Sharjah, United Arab Emirates (UAE).

Study Design
This study followed a cross-sectional study design to develop a data collection tool (Emergency Services Patient Satisfaction Questionnaire) for patient satisfaction in the emergency departments of hospitals in Sharjah, UAE.

Setting
Emergency Department at Al-Qassemi Hospital, during the period from October 2018 to January 2019.

Study Sample
A convenient sample was chosen. The total sample size accounted for 207 respondents, who agreed to participate in the study during the assigned period of data collection.

Data Collection Tool
The data collection tool was developed following a review of literature related to patient satisfaction in the ED (Atari & Atari, 2015;Taye, Yassin, & Kebede, 2014;Corbett & McGuigan, 2008;Kristensson & Ekwall, 2008;Bos, Strums, Stellato, Schrijvers, & van Stel, 2013;Bos, Strums, Stellato, Schrijvers, & van Stel, 2012;Soleimanpour et al., 2011;Marco, Davis, & Olson, 2015). The final version of the tool included 25 statements to measure patient satisfaction, namely: waiting period during the registration process, i.e. time spent filling out forms after arriving at the ED before seeing a nurse; waiting period before being seen by a doctor or nurse; total time spent in the ED; availability of someone who spoke your language in the ED; time spent with medical staff in the ED; courtesy and respect of ED physicians; courtesy and respect of ED nursing staff; treatment offered to you by ED medical staff; explanation provided to you by ED medical staff about your condition, medication, and treatment; involvement in making decisions about your own care and treatment; explanation provided to you by ED medical staff regarding your lab test results; services provided by the laboratory department; services provided by the radiology department; services provided by the hospital pharmacy; time taken for staff to provide pain control; pain control measures and procedures received; performance of medical staff in examining and treating you; general atmosphere in the waiting room, such as dimensions, crowdedness, and availability of chairs; cleanliness of the ED; privacy during examination or treatment; security/safety procedures in the ED; ease of locating ED via signs and directions; information provided to you before leaving the ED regarding diagnosis, medications, and treatment; information provided to you before leaving the ED regarding warning signs and symptoms for your illness or treatment; information provided to you regarding follow-up care. Data collection on the satisfaction scale took place using the interview method for patients in the emergency room after receiving treatment and before exiting.
Satisfaction levels were measured using a 3-point Likert scale (satisfied=3, do not know=2, dissatisfied=1). Data were entered, organised, tabulated, and analysed using IBM SPSS Statistics for Windows, Version 21.0. Principal Components Analysis (PCA) was used for the extraction of the anticipated factors with oblique rotation. All statistical tests of significance were adjusted at the 5% level of significance.

Tool Validation:
The tool was validated through the following steps: Step 1: Face validity, which was conducted through the research team's literature review on patient satisfaction scales in healthcare. Satisfaction statements were formulated and compiled.
Step 2: Content validity, which included a panel of nine randomly selected specialists, working as directors of the ED at different hospitals in Abu Dhabi, Dubai, and Sharjah. Three directors were holders of Masters in Healthcare Management, while the rest were American Board certified specialists in Emergency Medicine. Round one content validation included consensus regarding relevance, as well as the inclusion or exclusion of the statements present on the questionnaire (inclusion code=1, inclusion after rephrase=2, exclusion code=0) regarding concordance and validity of the statements for measuring patient satisfaction in the ED. Round two included the agreement on the final version of the satisfaction tool (agree=1, disagree=0).
The content validity index (CVI), a proportion agreement method, estimates the content validity quantitatively. Previously, the method has been described for determining and quantifying the validity of different items in questionnaires (Polit, Beck, & Owen, 2007). In this study, the content validity was calculated through cross validation of agreement on the binary variable, which was determined by calculating a content validity ratio (CVR) for each item = [(E -(N / 2)) / (N / 2)], where: N = the total number of experts E = the number who rated the object as essential S-CVI/Average (scale content validity index) = computed as the average Item Content Validity Index (I-CVIs) for all the items in the questionnaire = 20.9/25 = .836 Total agreement = the number of items that achieved the I-CVI of 1.00 divided by the total number of items to be validated in the tool = 11/25 = .44 According to the Lawshe Table, the numeric value of CVR used to include items in the tool is determined by critical values of CVR, based on the number of experts. The numeric value that should be used is .78 (based on 9 experts). The yielded index (.44), indicates a level of acceptable content validity, although it does not ensure that the level of agreement is above that which may have occurred by chance. The final version of the patient satisfaction tool included items with a CVR > .5.
Back-to-back translation from English to Arabic was performed by two independent English literature faculty staff to reach consistency between the English and Arabic versions of the tool for both English and Arabic speakers. Back translation is important when translating an instrument to make the two versions equivalent.
Step 3: PCA took place on the 207 questionnaires eligible for statistical analysis. PCA was performed with three extracted components after examining the scree plot, and used the direct oblimin rotation for possible collinearity between the extracted components. The following statements (services provided by the radiology department and services provided by the hospital pharmacy) were not included in the analysis, due to the high loading for the option (4=inapplicable) by most of the respondents in the sample under study. Thus, the total number of statements eligible for PCA analysis were a total of 23 statements.

Ethical Considerations
The study proposal was granted the ethical approval by the Research Ethics Committee, Sharjah Medial Disrtict, Ministry of Health and Prevention, Sharjah -United Arab Emirates in May 2017. Participants were informed about the aim of the data collection as declared on the cover page of the data collection tool. Participants signed an approval to participate in the study. Anonymity was assured throughout the data collection phase by absence of any personal identification variables or indications on the questionnaire.
Bartlett's Test of sphericity was used to test the null hypothesis, that variables in the population correlation matrix were uncorrelated (threshold: p < .05). With Bartlet's Test = 3262.7, p < .00001, this supported the factorability of the scale statement correlation matrix.  Vol. 12, No. 10; doctor/nurse period, total time, courtesy and respect of nurses, services provided by the laboratory department, services provided by the radiology department, cleanliness of the ED, privacy during examination or treatment, security/safety procedures in the ED, instructions about warning signs, and instructions about follow-up care. Moderate agreement (content validity score= .77) accounted for ten (10) statements on the scale. These statements were: courtesy and respect of physicians, treatment by medical staff, explanation provided by medical staff, patient involvement in treatment decisions, explanation of laboratory results, time taken to offer pain control, pain control measures and procedures received, general atmosphere in the waiting room, signs and directions to the ED, and instructions given before leaving the ED. This left only four (4) statements with weak agreement (content validity score= .55), namely: native language availability, time spent with medical staff, services provided by the hospital pharmacy, and performance of medical staff. Based on the scree plot, PCA was performed with three extracted components. Table 2 presents the correlation loadings per statement on the extracted components, after using the oblimin rotation. The analysis extracted three components, which accounted for 51.

Discussion
The aim of this study was to develop and validate a universal tool for patient satisfaction with emergency departments in Sharjah, UAE. Construct validity was performed using PCA, which yielded three domains out of the 23 statements eligible for analysis. The components analysis compiled statistically related statements, which are congruent with the domains generally utilised on patient satisfaction scales. The three domains were labelled: medical staff performance, duration of the encounter in the ED, and general impression of the ED. The nomenclature of the domains is consistent with the universal domain titles in the majority of validated satisfaction questionnaires about healthcare, in general, and emergency departments, specifically. Medical staff performance included 6 statements consistent with similar validation studies in emergency departments, (Bos et al., 2013;Bos et al., 2012;Soleimanpour et al., 2011;Muntlin, Gunningberg, & Carlsson, 2006) duration of the encounter had 13 statements in concordance with other studies, (Bos et al., 2013;Bos et al., 2012;Soleimanpour et al., 2011;Muntlin, Gunningberg, & Carlsson, 2006;Marco et al., 2015;Muntlin et al., 2006;Ekwall & Davis, 2010) and general impression about the ED entailed 13 statements (Bos et al., 2013;Bos et al., 2012;Ekwall & Davis, 2010;Lee & Yom, 2007;Messina, Vencia, Mecheroni, Dionisi, Baragatti, & Nante, 2014). Backup analysis of the internal reliability of the scale as a whole, as well as subscale analysis, revealed high reliability coefficients. While this probably indicates a highly consistent scale, such reliability requires further testing upon a number of samples and under variant situations.
The number of individuals included in the sample (n=207) was consistent with the acceptable least sample size needed for factor analysis, which is around 150 (Yaghmaei, 2003). With values above .6, the Kaiser-Meyer-Olkin index also confirmed the adequacy of the sample size for factor solution quality (Yaghmaei, 2003;Zaghloul, Hayajneh, & AlMarzouqi, 2010).
In order to achieve content relevance pertaining to a qualitative instrument, it is necessary to quantify the degree of agreement among the panel of experts. Several methods within the context of the degree of agreement have been proposed by researchers. Such methods included averaging experts' ratings on item relevance and using a pre-established criterion of acceptability (Beck & Gable, 2001); another method by calculating the coefficient alpha to quantify agreement of item relevance by three or more experts; or computing a multi-rater kappa coefficient (Polit & Beck, 2006).
Despite the criticisms shared among researchers, CVI is the most widely reported measure for content relevance of an instrument (Polit & Beck, 2006). In a study by Wynd et al., 2003 addressed the deficiency of the CVI formula to adjust for chance agreement, in comparison to the inter-rater kappa statistic (Wynd, Schmidt, & Schaefer, 2003).
In instrument construction, researchers strive to receive universally congruent expert ratings to yield an acceptable agreement on the items of the instrument, with a content index in this study of .44 for the scale items. The index indicates a level of some content validity, although it does not ensure that the level of agreement is above that which may have occurred by chance. However, as has been argued by Polit and Beck (Polit & Beck, 2006), as the number of experts increases, the likelihood of achieving total agreement decreases, a conclusion in favour of the validating the instrument under study.

Limitations
The scale item to sample size ratio for this study was 1:10. Such ration could have affected the component extraction results, as it was recommended in a study on questionnaire validation to be within the limits of 1:15 (Polit, Beck, & Owen, 2007).

Conclusion
The findings of the present study support the reliability and validity of the Emergency Services Patient Satisfaction Questionnaire (23 statements) for intended use, following an independent sample of patients at the ED of one of Sharjah hospitals, UAE. The scale can thus be a useful tool for assessing patient satisfaction with service provision to help hospitals in Sharjah determine to what extent they are meeting the needs of their patients. Yet, a confirmatory PCA as well as reliability analysis using a second sample would significantly validate the instrument for general utilization.