Validity and Reliability Analysis of the PlotDigitizer Software Program for Data Extraction from Single-Case Graphs

Overview

Journal Perspect Behav Sci

Publisher Springer Nature

Specialty Psychiatry

Date 2022 Mar 28

PMID 35342869

Authors

Orhan Aydin

Muhammed Yasin Yassikaya

Affiliations

Soon will be listed here.

Abstract

Access to raw data of graphs presented in original articles to calculate the effect size of single-case research is a challenge for researchers conducting studies such as meta-analysis. Researchers typically use data extraction software programs to extract raw data from the graphs in articles. In this study, we aimed to analyze the validity and reliability of the PlotDigitizer software program, which is widely used in literature and an alternative to other data extraction programs, on computers with different operating systems. We performed the digitization of 6.846 data points on three different computers using 15 hypothetical graphs with 20 data series and 186 graphs with 242 data series from 29 published articles to accomplish the goal. Besides, using the values we digitized, we recalculated the 23 effect sizes presented in the original articles for validity analysis. Based on our sampling, we calculated intercoder and intracoder Pearson correlation coefficients. The results showed that PlotDigitizer could be an alternative to other programs as it is free and can run on many current and outdated systems, and it is valid and reliable as it is nearly perfect. Based on the obtained results and considering the data extraction process, we presented various recommendations for the researchers that will use the PlotDigitizer program for the quantitative analysis of single-case graphs.

Citing Articles

Associations of the gut, cervical, and vaginal microbiota with cervical cancer: a systematic review and meta-analysis.

Wen Q, Wang S, Min Y, Liu X, Fang J, Lang J BMC Womens Health. 2025; 25(1):65.

PMID: 39955550 PMC: 11829412. DOI: 10.1186/s12905-025-03599-1.

Sex Differences in Grip Strength From Birth to Age 16: A Meta-Analysis.

Nuzzo J Eur J Sport Sci. 2025; 25(3):e12268.

PMID: 39954272 PMC: 11829709. DOI: 10.1002/ejsc.12268.

Effects of continuous positive airway pressure on renal function in adults with obstructive sleep apnea: a systematic review and meta-analysis.

Peng H, Hu C, Wu Y, Liu W, Tsai C, Kuan Y Sleep Breath. 2025; 29(1):102.

PMID: 39945962 DOI: 10.1007/s11325-025-03256-9.

Quercetin as a therapeutic agent for skin problems: a systematic review and meta-analysis on antioxidant effects, oxidative stress, inflammation, wound healing, hyperpigmentation, aging, and skin cancer.

Okselni T, Septama A, Juliadmi D, Dewi R, Angelina M, Yuliani T Naunyn Schmiedebergs Arch Pharmacol. 2024; .

PMID: 39738831 DOI: 10.1007/s00210-024-03722-3.

Effects of Valproic Acid Embryonic Exposure on Zebrafish: A Systematic Review and Meta-Analysis.

Flores-Prieto B, Manzo-Denes J, Hernandez-Aguilar M, Coria-Avila G, Herrera-Covarrubias D, Aranda-Abreu G NeuroSci. 2024; 5(4):650-665.

PMID: 39728678 PMC: 11678182. DOI: 10.3390/neurosci5040046.

References

Moeyaert M, Maggin D, Verkuilen J . Reliability, Validity, and Usability of Data Extraction Programs for Single-Case Research Designs. Behav Modif. 2016; 40(6):874-900. DOI: 10.1177/0145445516645763. View

Smith J . Single-case experimental designs: a systematic review of published research and current standards. Psychol Methods. 2012; 17(4):510-50. PMC: 3652808. DOI: 10.1037/a0029312. View

Shadish W, Sullivan K . Characteristics of single-case designs used to assess intervention effects in 2008. Behav Res Methods. 2011; 43(4):971-80. DOI: 10.3758/s13428-011-0111-y. View

Zimmerman K, Pustejovsky J, Ledford J, Barton E, Severini K, Lloyd B . Single-case synthesis tools II: Comparing quantitative outcome measures. Res Dev Disabil. 2018; 79:65-76. DOI: 10.1016/j.ridd.2018.02.001. View

Parker R, Vannest K, Davis J, Sauber S . Combining nonoverlap and trend for single-case research: Tau-U. Behav Ther. 2011; 42(2):284-99. DOI: 10.1016/j.beth.2010.08.006. View

Rochat L, Manolov R, Billieux J . Efficacy of metacognitive therapy in improving mental health: A meta-analysis of single-case studies. J Clin Psychol. 2017; 74(6):896-915. DOI: 10.1002/jclp.22567. View

Jelicic Kadic A, Vucic K, Dosenovic S, Sapunar D, Puljak L . Extracting data from figures with software was faster, with higher interrater reliability than manual extraction. J Clin Epidemiol. 2016; 74:119-23. DOI: 10.1016/j.jclinepi.2016.01.002. View

Parker R, Vannest K . An improved effect size for single-case research: nonoverlap of all pairs. Behav Ther. 2009; 40(4):357-67. DOI: 10.1016/j.beth.2008.10.006. View

Flower A, McKenna J, Upreti G . Validity and Reliability of GraphClick and DataThief III for Data Extraction. Behav Modif. 2015; 40(3):396-413. DOI: 10.1177/0145445515616105. View

10.

Odom S, Barton E, Reichow B, Swaminathan H, Pustejovsky J . Between-case standardized effect size analysis of single case designs: Examination of the two methods. Res Dev Disabil. 2018; 79:88-96. DOI: 10.1016/j.ridd.2018.05.009. View

11.

Manolov R, Solanas A, Sierra V . Extrapolating baseline trend in single-case data: Problems and tentative solutions. Behav Res Methods. 2018; 51(6):2847-2869. DOI: 10.3758/s13428-018-1165-x. View

12.

Shadish W, Brasil I, Illingworth D, White K, Galindo R, Nagler E . Using UnGraph to extract data from image files: verification of reliability and validity. Behav Res Methods. 2009; 41(1):177-183. DOI: 10.3758/BRM.41.1.177. View

13.

Manolov R, Vannest K . A Visual Aid and Objective Rule Encompassing the Data Features of Visual Analysis. Behav Modif. 2019; 47(6):1345-1376. DOI: 10.1177/0145445519854323. View

14.

Manolov R, Rochat L . Further developments in summarising and meta-analysing single-case data: An illustration with neurobehavioural interventions in acquired brain injury. Neuropsychol Rehabil. 2015; 25(5):637-62. DOI: 10.1080/09602011.2015.1064452. View

15.

Drevon D, Fursa S, Malcolm A . Intercoder Reliability and Validity of WebPlotDigitizer in Extracting Graphed Data. Behav Modif. 2016; 41(2):323-339. DOI: 10.1177/0145445516673998. View

16.

Wichnick-Gillis A, Vener S, Poulson C . Script fading for children with autism: Generalization of social initiation skills from school to home. J Appl Behav Anal. 2018; 52(2):451-466. DOI: 10.1002/jaba.534. View

17.

Maggin D, Odom S . Evaluating single-case research data for systematic review: a commentary for the special issue. J Sch Psychol. 2014; 52(2):237-41. DOI: 10.1016/j.jsp.2014.01.002. View