THE INDONESIAN VERSION OF EVIDENCE- BASED PRACTICE QUESTIONNAIRE (EBPQ): AN EVALUATION OF RELIABILITY

Introduction: The use of evidence-based practice (EBP) has been demonstrated to increase the quality of patient care. However, the extent to which it is practiced in Indonesia is yet to be determined. It was necessary to develop a means of measuring EBP in order to acquire a profile of the use of EBP in Indonesia. The EBP questionnaire (EBPQ) developed by Upton and Upton was selected in the current study to evaluate the perceptions of healthcare workers regarding their knowledge of, attitudes toward, and practice of EBP. This questionnaire is widely recognized; however, an Indonesian version has not yet been developed. This study aimed to translate the EBPQ developed by Upton and Upton into Indonesian and to evaluate its reliability. Method: WHO framework on how to translate and adapt an instrument was applied. On completion of the forward translation and discussion process, backward translation of the EBPQ was performed, after which it was pretested and finalized. Reliability was tested by testing the questionnaire on 42 nurses at five hospitals in Depok and Jakarta. Result: Four words were changed. All items are valid. The reliability analysis resulted Cronbach’s  of 0.96 ( = 0.92, 0.80, and 0.96 for practice, attitude, and knowledge, respectively. Thus, 24 translated statements determined to be valid and reliable, were included in the final version. Conclusion: The Indonesian translated version of the EBPQ proposed by Upton and Upton was demonstrated to be valid and reliable. Further studies on the perceptions of healthcare workers are warranted.


INTRODUCTION
The adoption of evidence-based practice (EBP) to assist the clinical decision-making of healthcare workers is important. EBP combines the best available evidence with clinical expertise and client preferences (Sackett, Rosenberg, Gray, Haynes, & Richardson, 1996;Sigma Theta Tau International, 2005).
Nevertheless, the benefits of practicing EBP were undervalued (McDonald et al., 2010). It was demonstrated that patients' deaths in South-East Asian region, including Indonesia, were due to poor clinical practices (McDonald et al., 2010). Insufficient access to the best available evidence and skills needed to produce and interpret the evidence was cited as root causes. The use of EPB is sometimes misunderstood in clinical practice (Martis, Ho, and Crowther (2008). Thus, EBP-based research is vital for its implementation in south-east Asia.
The implementation of EBP in Indonesia has been encouraged in the nursing area through the legislation issued in 2005 (Indonesian Health Ministry, 2005) and then restated in 2014 (Indonesian Republic, 2014) Nevertheless, the Indonesian nurses' EBP remains inadequate (Stagner, 2017) and its studies remain scant. The extent to which it is practiced based on the Indonesian nurses' perspective is yet to be determined. A reliable instrument is required to examine the EBP perception of nurses in Indonesia. However, it is presently not available.
The EBP questionnaire (EBPQ) developed by Upton and Upton (2006) was selected in current study as a self-report measure to enable an evaluation of healthcare workers' perceptions of their own EBPs. The questionnaire covers the relevant and practical aspects of EBPs used on a daily basis. It contains three subscales which are knowledge attitudes, and practice. The knowledge subscale comprises 14 statements, the attitudes subscale consists of four pairs of statements, and the practice subscale contains six statements. The 24 items are scored from 1-7. Positive responses are equated with a high score.
This questionnaire has been demonstrated to be a valid and reliable measurement. A systematic review reported 24 EBP instruments were used to measure the knowledge, skills, and attitudes of nurses. The Upton and Upton's EBPQ was included in the study (Leung, Trevena, & Waters, 2014).

METHOD Translation of the evidence-based practice questionnaire
The World Health Organization's guideline for instrument translation and adaptation was applied to the translation process (World Health Organization, 2017). It involves four steps, (1') forward translation, (2') backward translation, (3') pretests and cognitive interviews, and (4') compilation of the last version. At the first step, the EBPQ was translated by health and education professionals for whom Bahasa Indonesia was their first language and with adequate proficiency in English. At the second step, the translated EBPQ was evaluated by a nurse manager, a medical researcher in a government office, and a medical lecturer to confirm its content validity. Consequently, the questionnaire underwent back translation to English language by an independent translator who did not know the questionnaire. Thereafter, the EBP authors were consulted regarding the back-translated EBPQ. It was then pretested with two nurses at the third stage.
The content for the questionnaire was then finalized. Validity and reliability of the evidence-based practice questionnaire The EBPQ was evaluated using several processes. Its content was validated and its reliability was measured against internal consistency using SPSS ®. . The EBPQ validation was determined by correlating the single items in each subscale with the total subscale score and by comparing the individual item scores with the total subscale scores. The reliability score was measured by the consistency of each item in the subscale and in total. Reliability testing was performed on 42 bachelor degree nurses and diploma three nurses at five hospitals. This test was carried out in Depok and Jakarta. Inter-rater reliability examinations were performed by researchers and enumerators to measure the questionnaire's consistency. The tests were conducted to nurses and doctors at two hospitals and a primary healthcare center. Table 1 presents the final Indonesian version of EBPQ. Several changes to words were effected in the second step of the translation process. Three items were impacted by cultural differences. "Evidence" was changed to "scientific evidence" and "IT" skills were changed to "information technology" skills. One modified item to reach better exemplify Indonesian culture: "I welcome questions about my practice" was changed to "I welcome criticism of or questions about my practice." Consensus was attained by three healthcare workers and medical lecturers over the word usage.

Translation of the evidence-based practice questionnaire
The EBPQ-Indonesia was then translated back into English by an English translator and then compared to the original version. Following the backward translation process, discrepancies were identified in relation to five items, namely "evaluate the outcomes of your practices," "My practice has changed because of evidence I have found," "awareness of major information types and sources," "knowledge of how to retrieve evidence," "the ability to critically analyze the evidence against set standards," and the "ability to apply information to individual cases." Pre-testing and carrying out cognitive interviews to confirm the subjects' understanding of the statements in the questionnaire and identify any difficulties in interpreting the statements comprised the third step. Pre-testing was conducted by two nurses with a tertiary education (i.e., a Bachelor's degree) and one nurse with a diploma. They suggested the word literature to be added to the word scientific evidence. There is no other difficulty in understanding the questionnaire.
Further steps were taken to finalize the questionnaire content. Item number 2 in the practice subscale was changed from "track down the relevant evidence once you have formulated the question" to "search for the relevant literature or scientific evidence after formulating the question." Item 3 was revised from "critically appraise, against set criteria, any literature you have discovered" to "critically assess any literature or scientific evidence that you have found in accordance with certain criteria." Item 4 was revised from "integrate the evidence you have found with your expertise" to "integrate the literature/scientific evidence that you've found with your expertise." Item 5 was revised from "evaluate the outcomes of your practice" to "evaluate the results of the health intervention or services that you provided." Item number 1 in the attitudesbased subscale was revised from "My workload is too great for me to keep up to date with all the new evidence" to "My workload is too heavy for me to know all the new literature." Item 2 was revised from "My practice changed because of evidence I found" to "My clinical practice changed because of the new science I read or learned." Item 6 was revised from "I welcome questions about my practice" to "I accept criticism of or questions about my clinical practice." Item 2 in the knowledge subscale was revised from "IT" skills to "information technology" skills. Item 5 was revised from "awareness of major information types and sources" to "knowledge about types and sources of information." Item 7 was revised from "knowledge of how to retrieve evidence" to "knowledge of how to identify scientific articles." Item 8 was revised from "ability to critically analyze the evidence against set standards" to the "ability to critically compare the scientific literature or evidence against existing standards." Item number 11 was changed from the "ability to apply information to individual cases" to the "ability to apply the knowledge in each case."

Validity
All 24 items are valid with Pearson correlation coefficient ranged from 0.451 and 0.875.

Reliability Internal consistency
The average inter-correlation among the items was shown to be  = 0.956, with scores of  = 0.92,  = 0.80, and  = 0.96 for practice, attitudes, and knowledge, respectively.

Inter-rater reliability
This questionnaire was conducted to five nurses and a doctor.
They were asked about their comprehension of the instrument, and they were able to understand it.

DISCUSSION
Following testing, the internal reliability of the Indonesian version of the EBPQ was found to be congruent with that described in other studies. High reliability (Cronbach's  = 0.956) was reported for all the items, higher than that obtained using the original EBPQ proposed by Upton and Upton (2006) (i.e.,  = 0.87) . It was also slightly higher than that reported in other studies by Ammouri et al. (2014) Shafiei, Baratimarnani, Goharinezhad, Kalhor, and Azmal (2014).
Although there was a slight difference between the EBPQ results obtained in the study by Shafiei et al. (2014), the internal consistency reliability of the overall scores was demonstrated in all the studies, as were similarities between the subscales scores. In particular, the attitude subscale scores obtained using the Indonesian version and others in the aforementioned studies were consistently lower than those obtained for the practice and knowledge subscales. This implies that the all statements listed in the Bahasa Indonesia were similarly reliable to those utilized in previous studies.

CONCLUSION
The EBPQ proposed by Upton and Upton (2006) was translated into Bahasa Indonesia and subsequently evaluated for validity and reliability. The findings suggest that the Indonesian version of EBPQ is very reliable. It is recommended to be used to measure EBP knowledge, attitudes, and practices of healthcare workers in Indonesia.

ACKNOWLEDMENT
This project was established by the World Class University/University of Indonesia project in the and funded by the Japan International Cooperation Agency. generation, synthesis and implementation to improve the care of mothers and babies in South East Asia: methods and design of the SEA-ORCHID Project using a logical framework approach.