|Year : 2019 | Volume
| Issue : 1 | Page : 30-35
Determination of visual portfolio for surgeons overseas assessment of surgical needs Nigeria study: Consensus generation through an e-Delphi process
Felix Makinde Alakaloko1, Etienne St-Louis2, Adesoji O Ademuyiwa3, Dan Poenaru2, Christopher Bode3
1 Department of Surgery, Pediatric Surgery Unit, Lagos University Teaching Hospital, Lagos, Nigeria
2 Division of Pediatric General and Thoracic Surgery, Montreal Children's Hospital, Montreal, Canada
3 Department of Surgery, College of Medicine, University of Lagos, Lagos, Nigeria
|Date of Web Publication||11-Mar-2019|
Dr. Felix Makinde Alakaloko
Department of Surgery, Pediatric Surgery Unit, Lagos University Teaching Hospital, Lagos
Source of Support: None, Conflict of Interest: None
Background: Surgery as a public health priority has received little attention until recently. There is a significant unmeasured and unmet burden of surgical illness in low- and middle-income countries (LMICs). Our aim was to generate a consensus among expert pediatric surgeons practicing in LMICs regarding the spectrum of pediatric surgical conditions that we should look out for in a community-based survey for Surgeons OverSeas Assessment of Surgical Needs Nigeria study. Materials and Methods: The Delphi methodology was utilized to identify sets of variables from among a panel of experts. Each variable was scored on a 5-point Likert scale. The experts were provided with an anonymous summary of the results after the first round. A consensus was achieved after two rounds, defined by an improvement in the standard deviation (SD) of scores for a particular variable over that of the previous round. We invited 76 pediatric surgeons through e-mail across Africa but predominantly from Nigeria. Results: Twenty-one pediatric surgeons gave consent to participate through return of mail. Thirteen (62%) answered the first round statements and 8 (38%) the second round. In general, the strength of agreement to all statements of the questionnaire improved between the first and second rounds. Overall consensus, as expressed by the decrease in the mean SD from 0.84 in the first round to 0.68 in the second round, also improved over time. The strength of consensus improved for 23 (74%) of the statements. The strength of consensus decreased for the remaining 8 (26%) of statements. Out of the 31 consensus-generating statements, 16 (51%) scored high agreement, 13 (42%) scored low agreement, and 2 (15%) scored perfect disagreement. Conclusion: We have successfully identified the pediatric surgical conditions to be included in any community survey of pediatric surgical need in an LMIC setting.
Keywords: Delphi, global surgery, low- and middle-income countries, Surgeons OverSeas Assessment of Surgical Needs
|How to cite this article:|
Alakaloko FM, St-Louis E, Ademuyiwa AO, Poenaru D, Bode C. Determination of visual portfolio for surgeons overseas assessment of surgical needs Nigeria study: Consensus generation through an e-Delphi process. Niger J Surg 2019;25:30-5
|How to cite this URL:|
Alakaloko FM, St-Louis E, Ademuyiwa AO, Poenaru D, Bode C. Determination of visual portfolio for surgeons overseas assessment of surgical needs Nigeria study: Consensus generation through an e-Delphi process. Niger J Surg [serial online] 2019 [cited 2019 May 20];25:30-5. Available from: http://www.nigerianjsurg.com/text.asp?2019/25/1/30/253785
| Introduction|| |
Surgical care is often erroneously perceived as a highly specialized and cost-ineffective area of medicine. On the contrary, it is essential at all levels of health-care delivery, including primary care, with provision of curative care for conditions and injuries. This role of surgery as a public health priority has received little attention until recently, when the Lancet Commission on Global Surgery sheds light on the significant unmet burden of surgical illness in low- and middle-income countries (LMICs) and highlighted the significant health and economic benefits that would result from better access to safe surgical, anesthesia, and obstetric care. Indeed, many illnesses such as cancers, trauma/road traffic injuries, and cardiovascular and metabolic diseases require surgery to restore health. These illnesses affect far more people than several other nonsurgical diseases combined. It is estimated that about 5 billion people do not have access to safe, affordable surgery, and anesthesia when needed. Among the approximately 300 million surgical procedures that are performed annually around the world, most occur in high-income countries (HICs), and it is largely unknown what the burden of surgical conditions are in LMICs. Furthermore, the significant strides made in promoting the role of surgery in matters of global health have largely ignored the significant burden of childhood surgical conditions, which are often distinct and whose treatment mandates a particular set of skills, equipment, and infrastructure.
The Surgeons OverSeas Assessment of Surgical Needs (SOSAS) is a survey tool designed using the Demographic and Health Surveys, the WHO guidelines for conducting community surveys for injuries and violence, and a survey tool for road traffic injuries. It has been used to estimate the unmet surgical needs in the communities in a few Asian and African countries.
Those studies have, however, not included physical examination and were not tailored toward children. The SOSAS Nigeria study survey tool aims to include these modifications in addition to the use of a visual portfolio for congenital conditions in children.
The aim of this study was to determine the spectrum of conditions to be included in the visual portfolio using an e-Delphi consensus-generating technique among pediatric surgeons.
| Materials and Methods|| |
Our study utilized the Delphi method, which seeks to attain a consensus among group members through a series of questionnaires. The chain of questionnaires is sent to a predetermined group of experts by mail or online.
This approach ensures confidentiality for respondents, as questionnaires are answered anonymously and individually by each member of the group. Furthermore, the e-Delphi method prevents undue influence by any member of the group on the responses of the other participants. The answers are reviewed and collated, and then, an anonymous summary of responses is sent back to the group members along with the next iteration of the questionnaire. This process is repeated until a group consensus is reached. This usually only takes two iterations but can sometimes take as many as six rounds before a consensus is reached. A consensus is defined by an improvement in the standard deviation (SD) of the scores of a particular question.
The study protocol was approved by the Lagos University Teaching Hospital Ethics and Research Board and conforms with the Declaration of Helsinki of ethical consent.
The experts answered the questionnaire in two rounds. After the first round, the participants were provided with an anonymous summary of the results with additional congenital anomalies nominated for inclusion. During the second round, experts were encouraged to review their earlier responses while considering the overall group's responses in the first iteration. This avoids direct influence from colleaguesin the setting of a debate or live discussion.
The findings were summarized in a series of statements. Participants were invited to rate their agreement with these statements using a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree).
Each round of the survey was administered online through a form designed on Microsoft Word® version 10 (Microsoft. Seattle, WA, USA).
In general, a sample size of about 10–15 is adequate for homogeneous panel of experts undergoing a Delphi survey. Seventy-six pediatric surgeons from across Africa but predominantly from Nigeria were invited to answer the questionnaire. Individuals were solicited for participation in the survey through e-mail invitation. They were given the opportunity to ask questions regarding the study and were asked to sign an electronic consent form. The first set of questions was only sent to those that consented to participation through an e-mail response to the lead investigator. The survey remained open for 3 months, and reminder e-mails were sent every 2 weeks to the respondents.
The responses collected for each statement were used to measure the strength of agreement, based on the mean score, and the strength of consensus, based on the SD of the recorded scores.
Perfect agreement with a statement was defined as a second-round mean score of 5 and perfect disagreement as a second-round mean score of 0. Perfect consensus was defined as a second-round SD of 0.00. High and low scores were defined as second-round scores larger or smaller than the mean of all recorded second-round scores, respectively. Consensus agreement was defined by decreasing SD in subsequent rounds.
| Results|| |
Of the 76 pediatric surgeons invited, 21 consented to participate through return of e-mail. Thirteen (62%) answered the first round statements and 8 (38%) responded in the second round [Table 1]. Among respondents, there were ten pediatric general surgeons, two pediatric urologists, and one pediatric orthopedic surgeon. The male-to-female ratio of the respondents was 3.3:1. About 85% of them work in the Federal Teaching Hospital, while 15% work with the state-owned teaching hospital as shown in [Table 2].
|Table 1: Respondent's participation and mean standard distribution of responses at each round of the Delphi survey|
Click here to view
Agreement and consensus
In general, the degree of consensus to all statements of the questionnaire improved between the first and second rounds, expressed by the decrease in the mean SD from 0.84 in the first round to 0.68 in the second round. This occurred despite the smaller number of respondents in the second round. [Table 3] shows the summary of the mean scores and the SDs for each statement in both rounds. In general, consensus opinion was achieved for the following question items that are to be retained in the SOSAS Nigeria study: hydrocephalus, cleft lip, cleft palate, angular dermoid cyst, cystic hygroma, umbilical hernia, patent urachus, hypospadias, hydrocele, undescended testis, inguinoscrotal hernia, congenital clubfoot, and anorectal malformation with vestibular fistula.
|Table 3: Mean score and standard deviation for each statement in the first and second rounds|
Click here to view
The statements were also categorized based on the strength of the agreement by responders and the strength of the consensus. Some statements resulted in perfect agreement, but two of the statements resulted in perfect disagreement: the survey items for osteogenesis imperfecta and congenital arthrogryposis multiplex.
The strength of consensus improved for 23 (74%) of the statements and decreased for the remaining 8 (26%) of statements. Out of the 31 consensus-generating statements, 16 (51%) had a high score of agreement and 13 (42%) had a low score of agreement, while 2 (15%) had perfect disagreement as depicted in [Table 4].
|Table 4: Classification of statements by strength of agreement and quality of consensus|
Click here to view
| Discussion|| |
The study has allowed us to survey a panel of experts about the need to carry out a community survey, as well as the congenital surgical conditions that should be included in the survey tool. The Delphi method used was initially developed for forecasting purposes but is now gaining wide popularity in qualitative medical research for generating consensus by limiting potential biases resulting from the traditional individual or group interview methods.
The panelist are pediatric surgeons that practice in major tertiary hospitals and hence have a huge burden of referrals from the rural centers which may have influenced their responses on certain disease conditions included for the survey. The spectrum of conditions that were selected by Delphi is hydrocephalus, cleft lip, cleft palate, angular dermoid cyst, and cystic hygroma. The potential cosmetic challenges these conditions pose among other things may have contributed to their inclusion. Other conditions selected are umbilical hernia, patent urachus, hypospadias, hydrocele, undescended testis, inguinoscrotal hernia, congenital clubfoot, and anorectal malformation with vestibular fistula.
There are set criteria for selecting surgically treatable conditions. These criteria are that these conditions must be (I) identifiable by the patient, (II) objectively identifiable by examination, and (III) acceptable both ethically and culturally. These criteria were set by the original developer of the SOSAS instrument and were based on the fact that the surgical condition should be identifiable by the patient (hence, these exclude many internal conditions) and should be able to be confirmed by examination (objective assessment) in a culturally acceptable manner.
Furthermore, it is worthy of note that not all surgically treatable conditions will be addressed in the survey, and therefore, the measured prevalence of surgically treatable conditions will be underestimating the real surgical burden of disease. The developers of the instrument realize that ethical and cultural acceptability may be different for countries, and therefore, the SOSAS tool may need to be adapted to suit local customs wherever it is going to be used. It is meant to be open source, for anyone to use to determine the prevalence of surgically treatable conditions in LMICs or specific regions within them. It has been used in West Africa, and a pioneer study in Sierra Leone has been replicated in Nepal, Rwanda, and Uganda.
This work is a modification of the SOSAS tool called the pediatric SOSAS to evaluate the need for surgery in the pediatric subpopulation. This modification includes three aspects which are to identify congenital anomalies in children and to do physical examination of children and use of picture portfolio which was not used in the previous study.
Nonetheless, our study has its limitations: the low response rate of 17% of the total invited panelists in the first round which further decreased to about 10% in the second round has a tendency of biasing the SD away from the null. This implies that the consensus achieved after the second round might have been more significant if the response rate was higher. However, there was still consensus achieved for a majority of the statements which is evidenced by a decreasing mean SD in the second round.
The reasons for the low response rate of the e-Delphi methodology may be due to the unpopularity of the methodology for analysis among medical experts in our environment as evidenced by the paucity of literature utilizing it. The use of a wireless audience participation system at any gathering has made it possible to complete the various rounds in 1 day, thereby maximizing the response rate with advantages for both participants and the researcher.
| Conclusion|| |
Based on the expert-generated consensus, we can conclude that the set of variables we achieved consensus on are necessary to be included in any community survey of pediatric surgical need in a LMIC. We believe that the findings of this Delphi survey could serve as a guideline to assist in setting out for a nation-wide community-based survey in Nigeria.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Gani F, Hundt J, Daniel M, Efron JE, Makary MA, Pawlik TM, et al.
Variations in hospitals costs for surgical procedures: Inefficient care or sick patients? Am J Surg 2017;213:1-9.
Meara JG, Leather AJ, Lars H. Global Surgery 2030: Evidence and solutions for achieving health, welfare and economic development. Lancet Comm 2015;386:9993:569-624.
Farmer PE, Kim JY. Surgery and global health: A view from beyond the OR. World J Surg 2008;32:533-6.
Fuller AT, Butler EK, Tran TM, Makumbi F, Luboga S, Muhumza C, et al.
Surgeons OverSeas assessment of surgical need (SOSAS) Uganda: Update for household survey. World J Surg 2015;39:2900-7.
Brown B, Cochran S, Dalkey N. The Delphi Method II, Structure of Experiments, Rand Corporation, RM-5957-PR; June, 1969.
Adler M, Ziglio E, editors. Gazing into the Oracle: The Delphi Method and its Application to Social Policy and Public Health. London: Jessica Kingsley Publishers; 1996.
Dalkey N. Analyses from a group opinion study. Futures 1969;1:541-51.
Custer RL, Scarcella JA. The modified Delphi technique. A rational modification. J Vocat Tech Educ 1999;15:50-8.
Rowe G, Wright G. Expert opinions in forecasting; the role of the Delphi technique. Princ Forecast 2001;12:6-144.
Tar-Ching AW, Loney T, Elias A, Ali S. Use of an audience response system to maximize response rates and expedite a modified Delphi process for consensus on occupational health. J Occup Med Toxicol 2016;11:9.
Elissa KB, Tran TM, Anthony TF, Brammel A, Joao RV, Luciano DA, et al
. Quantifying the pediatric surgical need in Uganda: Results of a nationwide cross-sectional, household survey. Pediatr Surg Int 2016;32:1075-85.
Groen RS, Samai M, Stewart KA, Cassidy LD, Kamara TB, Yambasu SE, et al.
Untreated surgical conditions in Sierra Leone: A cluster randomised, cross-sectional, countrywide survey. Lancet 2012;380:1082-7.
Gupta S, Ranjit A, Shrestha R, Wong EG, Robinson WC, Shrestha S, et al.
Surgical needs of Nepal: Pilot study of population based survey in Pokhara, Nepal. World J Surg 2014;38:3041-6.
Petroze RT, Groen RS, Niyonkuru F, Mallory M, Ntaganda E, Joharifard S, et al.
Estimating operative disease prevalence in a low-income country: Results of a nationwide population survey in Rwanda. Surgery 2013;153:457-64.
[Table 1], [Table 2], [Table 3], [Table 4]