Reliability and Agreement of Ultrasonographic Measures of the Ovarian Stroma: Impact of Methodology

Overview

Journal J Ultrasound Med

Date 2021 Dec 16

PMID 34913507

Authors

Heidi Vanden Brink

Avigdor Nosrati

Andres E Ortiz-Flores

Ashley Tam

Brittany Y Jarrett

Lynn Johnson

Andrew K Peppin

Marla E Lujan

Affiliations

Soon will be listed here.

Abstract

Objectives: Increased ovarian stromal area (SA), stromal-to-ovarian area ratio (S/A), and echogenicity (SEcho) on ultrasonography have been proposed as diagnostic markers for polycystic ovary syndrome. Although several methods to evaluate the stroma exist, their reproducibility has not been defined which limits clinical utility. This study aimed to determine the interrater reliability and agreement of methods to evaluate SA, S/A, and SEcho.

Methods: Five raters tested 3 methods to obtain SA and S/A, and one to obtain SEcho on 30 ovarian cineloops under two imaging conditions, simulating real-time (free-choice) or offline (fixed-frame) imaging. For SA, Method 1 subtracted follicular area from the ovarian area, Method 2 involved outlining the periphery of the stroma, and Method 3 represented a hybrid approach in which central follicles were subtracted from the outlined stroma. SEcho was scored on a subjective 3-tiered scale. Intraclass correlation coefficients (ICCs) and the coefficient of variation were determined for SA and S/A, and Fleiss' kappa agreement statistics (κ) were determined for SEcho.

Results: Interrater reliability of SA was superior using Method 1 (ICC = 0.558 and ICC = 0.705) versus Method 2 (ICC = 0.522 and ICC = 0.230) or Method 3 (ICC = 0.429 and ICC = 0.305) under free-choice and fixed-frame imaging conditions, respectively. Interrater reliability of S/A was also moderate to poor across methods. SEcho was also not reliably assessed across raters (κ = <0.500).

Conclusions: Ultrasonographic assessments of the ovarian stroma were associated with moderate to poor reproducibility. Indirect estimates of the ovarian stroma (Method 1) could be optimized to yield a reproducible approach, clarifying the clinical relevance of the stroma.

References

Yoo R, Sirlin C, Gottschalk M, Chang R . Ovarian imaging by magnetic resonance in obese adolescent girls with polycystic ovary syndrome: a pilot study. Fertil Steril. 2005; 84(4):985-95. DOI: 10.1016/j.fertnstert.2005.04.039. View

Zaidi J, Campbell S, Pittrof R, Kyei-Mensah A, Shaker A, Jacobs H . Ovarian stromal blood flow in women with polycystic ovaries--a possible new marker for diagnosis?. Hum Reprod. 1995; 10(8):1992-6. DOI: 10.1093/oxfordjournals.humrep.a136222. View

Barbieri R, Makris A, Ryan K . Insulin stimulates androgen accumulation in incubations of human ovarian stroma and theca. Obstet Gynecol. 1984; 64(3 Suppl):73S-80S. DOI: 10.1097/00006250-198409001-00019. View

Koo T, Li M . A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. J Chiropr Med. 2016; 15(2):155-63. PMC: 4913118. DOI: 10.1016/j.jcm.2016.02.012. View

Kelley K . Sample size planning for the coefficient of variation from the accuracy in parameter estimation approach. Behav Res Methods. 2008; 39(4):755-66. DOI: 10.3758/bf03192966. View

Pache T, Wladimiroff J, Hop W, Fauser B . How to discriminate between normal and polycystic ovaries: transvaginal US study. Radiology. 1992; 183(2):421-3. DOI: 10.1148/radiology.183.2.1561343. View

Romualdi D, Giuliani M, Cristello F, Fulghesu A, Selvaggi L, Lanzone A . Metformin effects on ovarian ultrasound appearance and steroidogenic function in normal-weight normoinsulinemic women with polycystic ovary syndrome: a randomized double-blind placebo-controlled clinical trial. Fertil Steril. 2009; 93(7):2303-10. DOI: 10.1016/j.fertnstert.2009.01.114. View

Barbieri R, Makris A, RANDALL R, Daniels G, KISTNER R, Ryan K . Insulin stimulates androgen accumulation in incubations of ovarian stroma obtained from women with hyperandrogenism. J Clin Endocrinol Metab. 1986; 62(5):904-10. DOI: 10.1210/jcem-62-5-904. View

Atiomo W, Pearson S, Shaw S, Prentice A, Dubbins P . Ultrasound criteria in the diagnosis of polycystic ovary syndrome (PCOS). Ultrasound Med Biol. 2000; 26(6):977-80. DOI: 10.1016/s0301-5629(00)00219-2. View

10.

Cyr L, Francis K . Measures of clinical agreement for nominal and categorical data: the kappa coefficient. Comput Biol Med. 1992; 22(4):239-46. DOI: 10.1016/0010-4825(92)90063-s. View

11.

Pache T, Hop W, Wladimiroff J, Schipper J, Fauser B . Transvaginal sonography and abnormal ovarian appearance in menstrual cycle disturbances. Ultrasound Med Biol. 1991; 17(6):589-93. DOI: 10.1016/0301-5629(91)90029-v. View

12.

Dewailly D, Robert Y, Helin I, Ardaens Y, Lemaitre L, Fossati P . Ovarian stromal hypertrophy in hyperandrogenic women. Clin Endocrinol (Oxf). 1994; 41(5):557-62. DOI: 10.1111/j.1365-2265.1994.tb01818.x. View

13.

Belosi C, Selvaggi L, Apa R, Guido M, Romualdi D, Fulghesu A . Is the PCOS diagnosis solved by ESHRE/ASRM 2003 consensus or could it include ultrasound examination of the ovarian stroma?. Hum Reprod. 2006; 21(12):3108-15. DOI: 10.1093/humrep/del306. View

14.

de Vet H, Terwee C, Knol D, Bouter L . When to use agreement versus reliability measures. J Clin Epidemiol. 2006; 59(10):1033-9. DOI: 10.1016/j.jclinepi.2005.10.015. View

15.

Kyei-Mensah A, Maconochie N, Zaidi J, Pittrof R, Campbell S, Tan S . Transvaginal three-dimensional ultrasound: reproducibility of ovarian and endometrial volume measurements. Fertil Steril. 1996; 66(5):718-22. DOI: 10.1016/s0015-0282(16)58624-5. View

16.

Kaleli S, Erel C, Oral E, Elter K, Akman C, Colgar U . Ovarian stromal hypertrophy in polycystic ovary syndrome. J Reprod Med. 1998; 43(10):893-7. View

17.

Broekmans F, de Ziegler D, Howles C, Gougeon A, Trew G, Olivennes F . The antral follicle count: practical recommendations for better standardization. Fertil Steril. 2009; 94(3):1044-51. DOI: 10.1016/j.fertnstert.2009.04.040. View

18.

Christ J, Willis A, Brooks E, Vanden Brink H, Jarrett B, Pierson R . Follicle number, not assessments of the ovarian stroma, represents the best ultrasonographic marker of polycystic ovary syndrome. Fertil Steril. 2013; 101(1):280-287.e1. PMC: 4033910. DOI: 10.1016/j.fertnstert.2013.10.001. View

19.

Franks S, Hardy K . Androgen Action in the Ovary. Front Endocrinol (Lausanne). 2018; 9:452. PMC: 6097027. DOI: 10.3389/fendo.2018.00452. View

20.

Robert Y, Dubrulle F, Gaillandre L, Ardaens Y, Lemaitre L, Dewailly D . Ultrasound assessment of ovarian stroma hypertrophy in hyperandrogenism and ovulation disorders: visual analysis versus computerized quantification. Fertil Steril. 1995; 64(2):307-12. View