The English dictionary defines consensus as a “general agreement about something.” It is seen as the middle ground in decision-making; between total assent and total disagreement. The process of consensus depends on the participants having shared values and common goals. Consensus leads to the generation of agreement on specific issues that provide overall directions for the future. The Second International Consensus Meeting (ICM) on orthopaedic infections had the above objectives in mind. The second meeting built on the success of the first ICM meeting that was held in 2013 and implemented additional steps, based on the input of the delegates from the prior meeting, with the intention of improving the outcome. The second ICM meeting was different in three aspects:

  1. It included delegates from all subspecialties of orthopaedics including: Hip and Knee Arthroplasty; Foot and Ankle; Oncology; Pediatrics; Shoulder and Elbow; Spine; Sports; and Trauma.
  2. The consensus was conducted according to the Delphi method again (see below). However, this time instead of having a central group conducting the research and writing out the recommendations and rationale for each question, individual delegates were engaged. For each question, the delegates evaluated the available literature, extracted the evidence for current practices, and identified the areas in need of further research. The level of evidence related to each “recommendation” was also identified. To the best of our knowledge, no published work related to orthopaedic infections was missed.
  3. The meeting allowed for the participation of representatives from governmental organizations, payers, and business administrations. Although these participants were not allowed to vote in the process, their presence was deemed to be important for developing the roadmap for funding, supporting, and approving technologies related to orthopaedic infections in the future.

The name Delphi derives from the Oracle of Delphi and was developed in the beginning of the Cold war to forecast the impact of technology on warfare. General Henry H. Arnold had ordered the creation of a report for the United States Army Air Corps on technological capabilities that could be used in future warfare. Very soon it became apparent that forecasting methods, technological approaches, and quantitative models could not be used, as little “scientific evidence” had been published in this field. To overcome these limitations, the Delphi method was developed by Project RAND during the 1950 and 1960s [1]. The Delphi method continues to be used by the military today and has found its way into the scientific and medical communities [2].

The exact description of the Delphi method that was utilized in the first ICM meeting has been previously published [3] and the document or executive summaries has also been published in various venues [4–6]. The second ICM also followed similar steps with the entire process being supervised by Dr. W. Cats Baril. The seed for the second consensus meeting was set in soil in June 2016, when at the request of many experts from around the world, we decided to proceed with the second meeting. Thirteen specific steps were followed:

Step 1 (June 2016): Selection of Delegates. This step aimed to gather the experts from around the globe, with no country overlooked, who could lend their expertise to the consensus process. The delegates were identified based on their publication track record in the field (at least five publications within the last five years), specialty society nominations, or their clinical expertise (high volume) in taking care of patients with orthopaedic infections. The search identified 953 delegates who were sent invitations. Some of the delegates did not respond to the invitation (63) or declined to participate (21), leaving 869 potential delegates to participate.

Step 2 (Dec 2016 to April 2017): Identification of Issues. The delegates were then asked to send in between 5 and 10 questions (issues) in the field of orthopaedic infections that they felt needed to be explored. A total of 3210 questions were received.

Step 3 (April 2017 to August 2017): Ranking of Questions. The collected questions were then sent to the delegates again and they were asked to prioritize them. In this process, we did not deliberately remove duplicate questions and did not make any changes to the “writing” of the questions. We believed that “duplications” perhaps represented the higher priority of a question.

Step 4 (August 2017): Evaluation of Ranked Questions. Once the ranking had been received, the duplicate questions were removed, and the stem of each question was rewritten according to the Delphi method. This step was necessary to remove “suggestive” words such as “what is the role of…?” as opposed to “Is there a role…?” This left us with 652 questions that comprised the final set of questions to be explored.

Step 5 (August 2017 to November 2017): Assignment of Questions. The final set of questions were then assigned to at least two delegates per question based on the publication track record of the delegate or the desire of a delegate to research a specific question. The delegates were given specific instructions on how to conduct research on the topics presented in each question and how to write up the responses.

Step 6 (December 2017- February 2018): Systematic Review. During this time period the delegates were actively engaged in researching a specific question and preparing the preliminary document related to each question. The two delegates assigned to each question were working independently for all workgroups except for the Shoulder group, who decided to work together. No published work in the English langauge were meant to be missed during this process.

Step 7 (February 2018 to April 2018): Inter-delegate Discussions. The document received from one delegate was then sent to the other and both delegates were made aware of each other’s write up and research. The activity was coordinated centrally to create one document that was acceptable by both delegates. Over 6,000 emails were exchanged during this process alone.

Step 8 (April 2018 to June 2018): Document Merging/Editing. All received documents were reviewed, written up, checked to remove plagiarism, references updated, and the English language edited.

Step 9 (June 2018 to July 2018): Document Evaluation by All Delegates. Although the documents generated were posted on the website (www.ICMPhilly.com) for many months and available for view by EVERYONE (including the public), the final document was sent to the delegates and they were asked to review any and all questions that were posted live on the website. We received numerous comments from delegates during this period and implemented any and all appropriate changes to the document prior to the meeting.

Step 10 (July 2018): Final Pre-Meeting Review/Editing. The entire document was reviewed by the internal editorial team and some additional changes were made. The latest publications, up until June 30, 2018, were also checked and added to relevant sections.

Step 11 (July 25 to 26, 2018): Pre-Vote Discussion. All delegates who had traveled to Philadelphia met in their workgroup and discussed some of the questions in their field. The questions were divided into four categories: 1) Highly clinically relevant with little evidence supporting the recommendation; 2) Highly controversial and clinically relevant; 3) Highly relevant and with great supportive evidence for the recommendation; and 4) Not clinically highly relevant with or without supportive evidence. During the meeting, questions from categories 1 and 2 were discussed.

Step 12 (July 27, 2018). Voting. All questions were presented on a screen and the delegates were allowed to vote in real time. The results of voting appeared on the screen shortly after the vote. There were three possible responses to each recommendation: agree, disagree, or abstain. The process of voting was clearly explained by Dr. Willy Cats-Baril clearly to the delegates prior to voting.

Step 13 (August 2018 onwards): Dissemination of Consensus Document. Following the meeting the voting results were implemented into the document. The document was additionally reviewed by outside editors of Journals, in particular Dr. Michael A. Mont and his fellow Dr. Nipun Sodhi, Dr. Thomas Bauer, and Dr. Chick Yates. The delegates were given the opportunity to review the final document over a four week period and to provide any additional feedback. All suggested and appropriate changes were implemented into the document. The final document was then sent to various journals for publication as well as for publication in a consolidated book form. The final document is also being translated into different languages.

As can be seen from the above, the delegates were very engaged at every step of the way in generating the consensus document. It is clear, however, that a complex process like the one above, may fall victim to some shortcomings and errors. We made every effort to minimize those as much as possible. We also attempted to be inclusive of all experts from around the world. We are certain that we may have missed some very deserving experts, who should have been part of this process. We apologize in advance to those experts whom we missed, to the readers who will have to endure some errors in the document, to the authors of reports that may have been missed unintentionally, and to anyone else who may feel perturbed because of our shortcomings. We hope that the document that is generated will serve the orthopaedic community for years to come and improve the care of our patients.

We wish to thank Orthopedic Research and Education Foundation (OREF) and the American Association of Hip and Knee Surgeons for their generous donation to support the publication of the consensus document.


[1] Dalkey N, Helmer O. An Experimental Application of the Delphi Method to the Use of Experts n.d.
[2] Adler M, Ziglio E. Gazing into the Oracle The Delphi Method and its Application to Social Policy and Public Health. Jessica Kingsley Publishers; n.d.
[3] Cats-Baril W, Gehrke T, Huff K, Kendoff D, Maltenfort M, Parvizi J. International consensus on periprosthetic joint infection: description of the consensus process. Clin Orthop Relat Res 2013;471:4065–75. doi:10.1007/s11999-013-3329-4.
[4] Parvizi J, Gehrke T. Executive summary. J Arthroplasty 2014;29:5. doi:10.1016/j.arth.2013.09.023.
[5] Parvizi J, Gehrke T. International consensus on periprosthetic joint infection: let cumulative wisdom be a guide. J Bone Joint Surg Am 2014;96:441. doi:10.2106/JBJS.N.00023.
[6] Parvizi J, Gehrke T, Chen AF. Proceedings of the International Consensus on Periprosthetic Joint Infection. Bone Joint J 2013;95-B:1450–2. doi:10.1302/0301-620X.95B11.33135.

%d bloggers like this: