Advertisement

Report from the International Dermatology Outcome Measures Initiative

      The International Dermatology Outcome Measures is a nonprofit organization dedicated to developing evidence-based, patient-centered outcome measures for dermatologic conditions. At the 2018 Alopecia Areata Research Summit, Dr Gottlieb, President of the International Dermatology Outcome Measures, presented an overview of their work in psoriasis, hidradenitis suppurativa, acne, and eczema and discussed the potential areas of mutual interest with the National Alopecia Areata Foundation. Herein, we present a summary of the topics discussed.

      Abbreviations:

      AAD (American Academy of Dermatology), IDEOM (International Dermatology Outcome Measures)

      International Dermatology OUTCOME MEASURES INITIATIVE

      The International Dermatology Outcome Measures (IDEOM) initiative was established in 2013 to address the need for standardized, validated, and patient-centered outcome measures. With the mission statement: “[To] establish patient-centered measurements to enhance research and treatment for those with dermatologic disease,” IDEOM has involved patients, physicians, payers, health economists, nonprofit organizations, and regulatory agencies to develop and/or identify meaningful outcome measures for clinical research and clinical practice (
      • Gottlieb A.B.
      • Levin A.A.
      • Armstrong A.W.
      • Abernethy A.
      • Duffin K.C.
      • Bhushan R.
      • et al.
      The International Dermatology Outcome Measures Group: formation of patient-centered outcome measures in dermatology.
      ).
      To address the need for outcome measures in multiple areas of dermatologic disease, IDEOM has established several workgroups and collaborations. At the first IDEOM meeting back in 2013, stakeholders agreed that psoriasis would be the disease prototype (
      • Gottlieb A.B.
      • Armstrong A.W.
      • Christensen R.
      • Garg A.
      • Duffin K.C.
      • Boehncke W.H.
      • et al.
      The International Dermatology Outcome Measures initiative as applied to psoriatic disease outcomes: a report from the GRAPPA 2013 meeting.
      ). The Psoriasis Working Group was then established with significant help from the US National Psoriasis Foundation. Next, IDEOM collaborated with the Cochrane Skin Group– Core Outcome Set Initiative and Zealand University Hospital, Roskilde, Denmark, to found the Hidradenitis Suppurativa Core Outcomes Set International Collaboration (
      • Thorlacius L.
      • Ingram J.R.
      • Garg A.
      • Villumsen B.
      • Esmann S.
      • Kirby J.S.
      • et al.
      Protocol for the development of a core domain set for hidradenitis suppurativa trial outcomes.
      ). In addition, IDEOM allied with the Acne Core Outcomes Research Network. Finally, to advance outcome measurement in clinical practice, IDEOM is collaborating with the American Academy of Dermatology (AAD) (
      • Gottlieb A.
      • Salame N.
      • Armstrong A.W.
      • Merola J.F.
      • Parra S.
      • Takeshita J.
      • et al.
      A provider global assessment quality measure for clinical practice for inflammatory skin disorders.
      ). Herein, we present an overview of selected deliverables to date.

      Psoriasis working group

      Following the Outcome Measures in Rheumatology model (
      • Boers M.
      • Brooks P.
      • Strand C.V.
      • Tugwell P.
      The OMERACT filter for outcome measures in rheumatology.
      ), the Psoriasis Working Group is developing a Core Outcome Set (i.e., the minimum set of outcomes that should be measured in all clinical trials for a given disease-state) for psoriasis clinical trials (
      • Williamson P.R.
      • Altman D.G.
      • Bagley H.
      • Barnes K.L.
      • Blazeby J.M.
      • Brookes S.T.
      • et al.
      The COMET handbook: version 1.0.
      ). Accordingly, IDEOM first defined what to measure by establishing a Core Domain Set for psoriasis clinical trials (i.e., the minimum set of relevant domains and subdomains to be measured). This effort consisted of a two-round Delphi process that spanned from 2013 to 2018. All relevant stakeholders, including patients, physicians, scientists, pharmaceutical industry representatives, advocacy organizations regulators, and payers, were involved in the process. The Core Domain Set for Psoriasis Clinical Trials consists of skin manifestations (primary: body surface area, erythema, induration, and/or scale), location of skin lesions (palmar-plantar and scalp psoriasis), investigator global, psoriasis, and psoriatic arthritis symptoms, patient global, treatment satisfaction, and health-related QOL (
      • Callis Duffin K.
      • Merola J.F.
      • Christensen R.
      • Latella J.
      • Garg A.
      • Gottlieb A.B.
      • et al.
      Identifying a core domain set to assess psoriasis in clinical trials.
      ).
      Now, IDEOM is focused on defining how to measure these domains. Within the Psoriasis Working Group, two parallel workgroups were formed to initiate this process: the Psoriatic Arthritis Working Group and the Treatment Satisfaction Working Group.

      Psoriatic Arthritis Working Group

      Psoriatic arthritis is a severe and underdiagnosed comorbidity of psoriasis that occurs in one of three patients with psoriasis (
      • Ogdie A.
      • Weiss P.
      The epidemiology of psoriatic arthritis.
      ). Despite this, current clinical trials for psoriasis do not screen for psoriatic arthritis, and the impact that psoriatic arthritis may have on the outcomes measured in a psoriasis clinical trial is unknown. Furthermore, there is paucity of data regarding the effect of psoriasis treatment on psoriatic arthritis. Although ideally both “signs” (enthesitis, spondylitis, arthritis, dactylitis) and “symptoms” (pain, patient global, and physical function) of psoriatic arthritis should be measured in every patient, only psoriatic arthritis symptoms met consensus criteria for the Core Domain Set, as dermatologists tend to feel uncomfortable conducting rheumatologic physical examination (
      • Helliwell P.
      • Coates L.
      • Chandran V.
      • Gladman D.
      • de Wit M.
      • FitzGerald O.
      • et al.
      Qualifying unmet needs and improving standards of care in psoriatic arthritis.
      ,
      • Kavanaugh A.
      • Helliwell P.
      • Ritchlin C.T.
      Psoriatic arthritis and burden of disease: patient perspectives from the population-based multinational assessment of psoriasis and psoriatic arthritis (MAPP) survey.
      ).
      The goals of the workgroup were then to (i) standardize screening for psoriatic arthritis in psoriasis clinical trials, and (ii) identify what is the most appropriate instrument to measure symptoms of psoriatic arthritis in this context. Following the recommendations from the COnsensus-based Standards for the selection of health Measurement INstruments, Core Outcome Measures in Effectiveness Trials initiative (
      • Prinsen C.A.
      • Vohra S.
      • Rose M.R.
      • Boers M.
      • Tugwell P.
      • Clarke M.
      • et al.
      How to select outcome measurement instruments for outcomes included in a “Core Outcome Set” - a practical guideline.
      ), and the Outcome Measures in Rheumatology filter 2.1 (
      • Boers M.
      • Kirwan J.R.
      • Wells G.
      • Beaton D.
      • Gossec L.
      • d’Agostino M.A.
      • et al.
      Developing core outcome measurement sets for clinical trials: OMERACT filter 2.0.
      ), the workgroup conducted an international online Delphi survey that was informed by a literature review and a pre-Delphi exercise (
      • Perez-Chada L.M.
      • Cohen J.M.
      • Gottlieb A.B.
      • Duffin K.C.
      • Garg A.
      • Latella J.
      • et al.
      Achieving international consensus on the assessment of psoriatic arthritis in psoriasis clinical trials: an International Dermatology Outcome Measures (IDEOM) initiative.
      ).
      In all, 1,222 international stakeholders were invited to participate, and 297 completed the survey. Stakeholders included rheumatologists (45.1%), dermatologists (25.9%), patients (7.4%), industry partners (8.8%), dermatologist–rheumatologists (5.1%), patient association representatives (3.4%), and others (4.3%). Regarding screening for psoriatic arthritis, stakeholders agreed that all patients enrolling in a psoriasis clinical trial should be screened for psoriatic arthritis. As for the selection of a patient-reported outcome measure for “psoriatic arthritis symptoms,” the Psoriatic Arthritis Impact of Disease-9 was voted as the instrument of highest quality. These results were, then, endorsed at the 2018 IDEOM annual meeting by 40 relevant stakeholders.
      Therefore, the IDEOM Psoriatic Arthritis Working Group recommends that the screening and measurement of psoriatic arthritis symptoms should occur in psoriasis clinical trials and registries. The next steps involve the dissemination and implementation of results in the psoriasis clinical trials and registries.

      Treatment satisfaction

      Treatment satisfaction is defined as the patient’s perception of the process of taking medication and the outcomes of taking that medication (
      • Shikiar R.
      • Rentz A.M.
      Satisfaction with medication: an overview of conceptual, methodologic, and regulatory issues.
      ). Therefore, the measurement of treatment satisfaction should address both treatment processes (i.e., treatment location, frequency, duration, route of administration, formulation, and cost) and treatment outcomes (i.e., treatment benefit and side effects) (
      • Shikiar R.
      • Rentz A.M.
      Satisfaction with medication: an overview of conceptual, methodologic, and regulatory issues.
      ).
      Following the COnsensus-based Standards for the selection of health Measurement INstruments guidelines on how to select outcome measures for the Core Outcome Set, we conducted a systematic literature review to identify all treatment satisfaction instruments that are used in dermatology and studies on the measurement properties of identified instruments. Next, we evaluated the methodological quality of identified studies using the COnsensus-based Standards for the selection of health Measurement INstruments checklist and the quality of the measurement properties of the instruments. Finally, we performed a qualitative synthesis of the evidence to define whether a Treatment Satisfaction tool was appropriate for the Core Outcome Set (
      • Salame N.
      • Perez-Chada L.M.
      • Singh S.
      • Callis Duffin K.
      • Garg A.
      • Gottlieb A.B.
      • et al.
      Are your patients satisfied a systematic review of treatment satisfaction measures in psoriasis.
      ).
      In all, we identified 11 treatment satisfaction instruments of which two were generic (the Beliefs about Medicines Questionnaire and the Treatment Satisfaction Questionnaire for Medication), three were dermatology-specific (Patient Benefit Index, Vehicle-Preference Score, Topical Therapy Adherence Questionnaire, and Patient Preference Questionnaire), and six were psoriasis-specific (Reflective evaluation of psoriasis Efficacy of Treatment and Severity, Spanish Satisfaction with Treatment of Psoriasis Questionnaire, Nail Assessment in Psoriasis and Psoriasis Satisfaction Questionnaire–Patient Benefit Index, and Desired Improvement Tool). Among these instruments, most lacked information on several measurement properties and presented substantial variability in validity, reliability, and responsiveness. Results were discussed at the 2018 IDEOM annual meeting, where it was agreed that a new treatment satisfaction instrument is required. IDEOM is currently working on the development of such an instrument.

      Hidradenitis suppurativa

      In following guidance from the Core Outcome Measures in Effectiveness Trials and Outcome Measures in Rheumatology initiatives, Hidradenitis Suppurativa Core Outcomes Set, International Collaboration conducted a Delphi consensus process to develop a Core Outcome Set suitable for interventional trials (
      • Thorlacius L.
      • Ingram J.R.
      • Garg A.
      • Villumsen B.
      • Esmann S.
      • Kirby J.S.
      • et al.
      Protocol for the development of a core domain set for hidradenitis suppurativa trial outcomes.
      ). Almost 100 participants from 19 countries spread over four continents comprised the stakeholder group, within which patients and experts had roughly equal representation. A comprehensive literature review, as well as a nominal process, yielded approximately 60 items deemed potentially important for measurement in trials by stakeholders. Consensus on items and development of the Core Domain Set was achieved through five anonymous e-Delphi rounds and four face-to-face consensus meetings. The core domains achieved by consensus include physical signs, hidradenitis suppurativa-specific QOL, global assessment, pain, and progression of disease course. In addition, the “symptoms of hidradenitis suppurativa” domain, which was endorsed by patients but not by healthcare professionals, was voted for inclusion to the Core Domain Set but is not considered mandatory for measurement (
      • Thorlacius L.
      • Ingram J.R.
      • Villumsen B.
      • Esmann S.
      • Kirby J.S.
      • Gottlieb A.B.
      • et al.
      A core domain set for hidradenitis suppurativa trial outcomes: an international Delphi process.
      ).
      Hidradenitis Suppurativa Core Outcomes Set International Collaboration workgroups are evaluating existing measures that adequately capture domain items. When measures do not exist for the specified items, or when measures may not be sufficiently validated, workgroups have been charged with developing new measures. Included in this initiative is the development and validation of a disease-specific QOL instrument known as the Hidradenitis Suppurativa QOL instrument (
      • Riis P.T.
      • Vinding G.R.
      • Ring H.C.
      • Jemec G.B.
      Disutility in patients with hidradenitis suppurativa: a cross-sectional study using EuroQoL-5D.
      ,
      • Sisic M.
      • Kirby J.S.
      • Boyal S.
      • Plant L.
      • McLellan C.
      • Tan J.
      Development of a quality-of-life measure for hidradenitis suppurativa.
      ).

      IDEOM–AAD collaboration for clinical practice

      As the healthcare systems continue to shift from fee-for-service to value-based payments models, dermatologists need to identify feasible, valid, and clinically meaningful outcome measures for clinical practice (
      • Porter M.E.
      What is value in health care?.
      ). Indeed, outcome measures are considered the gold standard in measuring the quality of care, and together with costs of care, they are used to determine value (
      Agency for Healthcare Research and Quality
      Types of health care quality measures.
      ).
      To address this gap, IDEOM and AAD formed an alliance to achieve consensus on which are the most appropriate outcome measures for inflammatory skin diseases in clinical practice.
      First, the IDEOM–AAD conducted a two-round Delphi survey in New York (February 2018) to agree on a provider-assessed global disease severity metric. A total of 36 stakeholders participated in this meeting, including dermatologists, rheumatologists–dermatologists, pediatric dermatologists, quality measures experts, AAD members, one patient, and a research fellow. Participants voted on (i) which of 23 inflammatory dermatoses discussed should be assessed by a global measure and (ii) which scale type would be more appropriate to quantify global disease severity (i.e., a 5-point Likert scale with or without descriptors, a numerical rating scale, or a dichotomous clear and/or almost clear versus not outcome) (
      • Gottlieb A.
      • Salame N.
      • Armstrong A.W.
      • Merola J.F.
      • Parra S.
      • Takeshita J.
      • et al.
      A provider global assessment quality measure for clinical practice for inflammatory skin disorders.
      ). The consensus was defined by prespecified endpoints for each question.
      Psoriasis, atopic dermatitis, and acne were the inflammatory dermatoses selected to be initially assessed by a global measure. As for the scale type, participants favored a 5-point ordinal scale (clear = 0, almost clear = 1, mild = 2, moderate = 3, severe = 4) with disease-specific descriptors provided through referenced electronic platforms (e.g., AAD website, phone applications) (
      • Gottlieb A.
      • Salame N.
      • Armstrong A.W.
      • Merola J.F.
      • Parra S.
      • Takeshita J.
      • et al.
      A provider global assessment quality measure for clinical practice for inflammatory skin disorders.
      ). These results were then discussed at the 2018 IDEOM annual meeting in Washington, DC. Among 86 stakeholders, 94% agreed that treatment response should also be evaluated by patient-reported outcome measures, and 88% voted this should occur in every patient visit.
      Next, the IDEOM–AAD collaboration held a second consensus meeting in Chicago, Illinois (October 2018) to reach consensus on a patient-reported instrument for acne, psoriasis, and atopic dermatitis in clinical practice. Several patient global assessment instruments that were retrieved from a literature review and Skindex instruments were discussed. Voting focused on identifying (i) what is the minimal set of outcomes that should be measured in clinical practice, and (ii) which is the most appropriate patient-reported instrument for quality measurement.
      Among 53 stakeholders, >70% agreed that the minimal set of outcomes that should be measured consists of the identification of patient goals, assessment of treatment harm, and assessment of the adequacy of treatment response. Regarding which is the most appropriate patient-reported instrument, the audience did not agree on a unique instrument reflecting that no instrument captures all relevant domains affected in patients with chronic inflammatory dermatoses. While Skindex instruments were the most preferred instrument, the audience also endorsed the use of a 5-point patient global assessment (0 = clear to 4 = severe) with an optional check-box: “worst ever.” In addition, a new instrument targeted at measuring change since treatment initiation (called trajectory measure) was proposed.
      The IDEOM–AAD collaboration is currently working on demonstrating the validity of these instruments and establishing them as process measures (i.e., measures that indicate what a provider does to improve or maintain health) for now (
      • Porter M.E.
      What is value in health care?.
      ,
      • Porter M.E.
      • Larsson S.
      • Lee T.H.
      Standardizing patient outcomes measurement.
      ). The standardized use of these instruments in clinical practice will be a critical first step toward optimal quality measurement.

      Summary

      Drawing on IDEOM’s experience on multiple disease states, IDEOM may serve as an advisor for the National Alopecia Areata Foundation to advance outcome measurement in alopecia areata. Furthermore, IDEOM may facilitate fundraising and annual meeting at which all relevant stakeholders, including Food and Drug Administration representatives, industry, patients, disease-state experts, and members from the AAD, can work together to overcome current research challenges in alopecia areata.

      Conflict of Interest

      ABG is an advisor and/or consultant for AbbVie, Beiersdorf, Bristol-Myers Squibb, Celgene, Incyte, Janssen Ortho, Eli Lilly ICOS LLC, Novartis, Sun Pharmaceutical Industries, UCB, Valeant Pharmaceutical International, Valeant Pharmaceutical North America LLC, XBiotech, Leo, Avotres Therapeutics, and Boeringer Ingelheim. She received research and/or educational grants from Janssen Ortho, Incyte, XBiotech, Novartis, Boeringer Ingelheim, and UCB. ABG is also a speaker for AbbVie, Eli Lilly, Janssen Biotech; and is an investigator for Incyte, Janssen Ortho, Lilly ICOS X Biotech, and UCB. In addition, she participated in the development of the National Psoriasis Foundation Psoriasis Score, an instrument discussed in this study. JFM has served as a consultant and/or investigator for Biogen, AbbVie, Dermavant, Amgen, Eli Lilly, Novartis, Pfizer, Janssen, UCB, Samumed, Science37, Celgene, Sanofi Regeneron, Arena, Sun Pharma, EMD Sorono, Avotres, Leo Pharma, Merck, and GlaxoSmithKline. AWA has served as investigator, advisor and/or consultant to Leo Pharma, Novartis, Dermira, UCB Pharma, AbbVie, Janssen, Eli Lilly, Regeneron & Sanofi, Science 37, Modernizing Medicine, Merck, Parexel, Celgene, Ortho Dermatologics, and Pfizer. AG served as a consultant for AbbVie, Amgen, Boehringer Ingelheim, Incyte, Janssen, Pfizer, UCB, Viela Bio, and Asana Biosciences. In addition, he received research and/or educational grants from UCB, AbbVie, and National Psoriasis Foundation and served as an investigator for AbbVie. LMP-C and JL state no conflict of interest.

      Acknowledgments

      This article is published as part of a supplement sponsored by the National Alopecia Areata Foundation .
      Funding for the Summit and publication of this supplement was provided by the National Alopecia Areata Foundation. This Summit was supported (in part) by the National Institute of Arthritis and Musculoskeletal and Skin Diseases under award number R13AR074890 . The opinions or views expressed in this professional supplement are those of the authors and do not necessarily reflect the official views, opinions, or recommendations of the National Institutes of Health or the National Alopecia Areata Foundation. City, state, and country in which the work was done: New York, New York, USA.

      Author Contributions

      Conceptualization: ABG; Supervision: JFM ABG; Visualization: LMP-C; Writing - Original Draft Preparation: LMP-C; Writing - Review and Editing: LMP-C, JFM, AWA, AG, JL, ABG

      References

        • Agency for Healthcare Research and Quality
        Types of health care quality measures.
        https://www.ahrq.gov/talkingquality/measures/types.html
        Date: 2011
        Date accessed: March 31, 2019
        • Boers M.
        • Brooks P.
        • Strand C.V.
        • Tugwell P.
        The OMERACT filter for outcome measures in rheumatology.
        J Rheumatol. 1998; 25: 198-199
        • Boers M.
        • Kirwan J.R.
        • Wells G.
        • Beaton D.
        • Gossec L.
        • d’Agostino M.A.
        • et al.
        Developing core outcome measurement sets for clinical trials: OMERACT filter 2.0.
        J Clin Epidemiol. 2014; 67: 745-753
        • Callis Duffin K.
        • Merola J.F.
        • Christensen R.
        • Latella J.
        • Garg A.
        • Gottlieb A.B.
        • et al.
        Identifying a core domain set to assess psoriasis in clinical trials.
        JAMA Dermatol. 2018; 154: 1137-1144
        • Gottlieb A.
        • Salame N.
        • Armstrong A.W.
        • Merola J.F.
        • Parra S.
        • Takeshita J.
        • et al.
        A provider global assessment quality measure for clinical practice for inflammatory skin disorders.
        J Am Acad Dermatol. 2019; 80: 823-828
        • Gottlieb A.B.
        • Armstrong A.W.
        • Christensen R.
        • Garg A.
        • Duffin K.C.
        • Boehncke W.H.
        • et al.
        The International Dermatology Outcome Measures initiative as applied to psoriatic disease outcomes: a report from the GRAPPA 2013 meeting.
        J Rheumatol. 2014; 41: 1227-1229
        • Gottlieb A.B.
        • Levin A.A.
        • Armstrong A.W.
        • Abernethy A.
        • Duffin K.C.
        • Bhushan R.
        • et al.
        The International Dermatology Outcome Measures Group: formation of patient-centered outcome measures in dermatology.
        J Am Acad Dermatol. 2015; 72: 345-348
        • Helliwell P.
        • Coates L.
        • Chandran V.
        • Gladman D.
        • de Wit M.
        • FitzGerald O.
        • et al.
        Qualifying unmet needs and improving standards of care in psoriatic arthritis.
        Arthritis Care Res (Hoboken). 2014; 66: 1759-1766
        • Kavanaugh A.
        • Helliwell P.
        • Ritchlin C.T.
        Psoriatic arthritis and burden of disease: patient perspectives from the population-based multinational assessment of psoriasis and psoriatic arthritis (MAPP) survey.
        Rheumatol Ther. 2016; 3: 91-102
        • Ogdie A.
        • Weiss P.
        The epidemiology of psoriatic arthritis.
        Rheum Dis Clin North Am. 2015; 41: 545-568
        • Perez-Chada L.M.
        • Cohen J.M.
        • Gottlieb A.B.
        • Duffin K.C.
        • Garg A.
        • Latella J.
        • et al.
        Achieving international consensus on the assessment of psoriatic arthritis in psoriasis clinical trials: an International Dermatology Outcome Measures (IDEOM) initiative.
        Arch Dermatol Res. 2018; 310: 701-710
        • Porter M.E.
        What is value in health care?.
        N Engl J Med. 2010; 363: 2477-2481
        • Porter M.E.
        • Larsson S.
        • Lee T.H.
        Standardizing patient outcomes measurement.
        N Engl J Med. 2016; 374: 504-506
        • Prinsen C.A.
        • Vohra S.
        • Rose M.R.
        • Boers M.
        • Tugwell P.
        • Clarke M.
        • et al.
        How to select outcome measurement instruments for outcomes included in a “Core Outcome Set” - a practical guideline.
        Trials. 2016; 17: 449
        • Riis P.T.
        • Vinding G.R.
        • Ring H.C.
        • Jemec G.B.
        Disutility in patients with hidradenitis suppurativa: a cross-sectional study using EuroQoL-5D.
        Acta Derm Venereol. 2016; 96: 222-226
        • Salame N.
        • Perez-Chada L.M.
        • Singh S.
        • Callis Duffin K.
        • Garg A.
        • Gottlieb A.B.
        • et al.
        Are your patients satisfied a systematic review of treatment satisfaction measures in psoriasis.
        Dermatology. 2018; 234: 157-165
        • Shikiar R.
        • Rentz A.M.
        Satisfaction with medication: an overview of conceptual, methodologic, and regulatory issues.
        Value Health. 2004; 7: 204-215
        • Sisic M.
        • Kirby J.S.
        • Boyal S.
        • Plant L.
        • McLellan C.
        • Tan J.
        Development of a quality-of-life measure for hidradenitis suppurativa.
        J Cutan Med Surg. 2017; 21: 152-155
        • Thorlacius L.
        • Ingram J.R.
        • Garg A.
        • Villumsen B.
        • Esmann S.
        • Kirby J.S.
        • et al.
        Protocol for the development of a core domain set for hidradenitis suppurativa trial outcomes.
        BMJ, Open. 2017; 7e014733
        • Thorlacius L.
        • Ingram J.R.
        • Villumsen B.
        • Esmann S.
        • Kirby J.S.
        • Gottlieb A.B.
        • et al.
        A core domain set for hidradenitis suppurativa trial outcomes: an international Delphi process.
        Br J Dermatol. 2018; 179: 642-650
        • Williamson P.R.
        • Altman D.G.
        • Bagley H.
        • Barnes K.L.
        • Blazeby J.M.
        • Brookes S.T.
        • et al.
        The COMET handbook: version 1.0.
        Trials. 2017; 18: 280