Publication Integrity in Orthopaedic Journals: the Self-citation in Orthopaedic Research (SCOR) Threshold

Overview

Journal Eur J Orthop Surg Traumatol

Specialty Orthopedics

Date 2019 Dec 21

PMID 31858259

Citations 9

Authors

Kavin Sundaram

Jared Warren

Hiba K Anis

Alison K Klika

Nicolas S Piuzzi

Affiliations

Soon will be listed here.

Abstract

Background: The impact factor (IF) is the most commonly used bibliometric method for rating academic journals. However, the practice of journals' self-citation may artificially elevate the IF. Additional bibliometric methods including Eigenfactor scale, SCImago Journal Ranking (SJR), and corrected IF (cIF) have been created. Comparing general-interest and specialized orthopaedic journals, the aims of this study were to assess: (1) the effect of journal´s self-citation on IF; (2) differences in bibliometric analysis; and (3) to determine thresholds for monitoring self-citation practices by defining the self-citation in orthopaedic research (SCOR) Threshold.

Methods: The journal citation reports and SCImago Journal and Country Rank databases were queried for orthopaedic journals from 1997 to 2017. The following bibliometrics were compared between general-interest and specialized journals: IF, cIF, Eigenfactor, self-citation rates, and SJR. A novel metric, the cIF ratio, was proposed to represent the relationship between a journal's IF and cIF. Thresholds for cIF were based on statistical outliers of cIF ratio within general-interest and specialized journals were calculated. Outliers were defined as data points that were greater than the third quartile by 1.5 times the interquartile range using the last 10 years studied (2007-2017).

Results: Specialized orthopaedic journals had a higher median self-citation rates compared to general-interest journals (11.85% vs. 6.36%, p < 0.001). Overall, cIF ratio declined over study period, and general-interest journals had a lower cIF ratio than specialized journals (8.77% vs. 19.54%, p < 0.001). Overall, general-interest journals had more favourable values for the bibliometric indices studied compared to specialized journals The SCOR threshold for cIF ratio was determined as 25.4% for general-interest journals and 53.3% for specialized journals.

Conclusion: Overall, self-citation occurs at a higher rate in specialized versus general-interest orthopaedic journals. We propose the use of a cIF ratio along with the SCOR threshold as a tool to evaluate and monitor journal self-citation practices in orthopaedic research.

Citing Articles

Comparison of journal and top publisher self-citation rates in COVID-19 research.

Quincho-Lopez A PLoS One. 2024; 19(12):e0314976.

PMID: 39636930 PMC: 11620575. DOI: 10.1371/journal.pone.0314976.

Novel approach for quantitative and qualitative authors research profiling using feature fusion and tree-based learning approach.

Umer M, Aljrees T, Ullah S, Bashir A PeerJ Comput Sci. 2024; 9:e1752.

PMID: 38192451 PMC: 10773922. DOI: 10.7717/peerj-cs.1752.

The Altmetric Attention Score Is Associated With Citation Rates and May Reflect Academic Impact in the Total Joint Arthroplasty Literature.

Kunze K, Vadhera A, Polce E, Higuera C, Siddiqi A, Chahla J HSS J. 2023; 19(1):37-43.

PMID: 36776509 PMC: 9837400. DOI: 10.1177/15563316221115723.

Author self-citations in the urology literature.

Aggarwal V Arab J Urol. 2022; 20(3):168-173.

PMID: 35935906 PMC: 9354642. DOI: 10.1080/2090598X.2022.2056976.

Brazilian authors don't cite Brazilian authors: Nothing has changed since 1994.

Giordano V, Lyra J, Bonadiman J, Lech O Rev Bras Ortop (Sao Paulo). 2021; 56(2):154-160.

PMID: 33935310 PMC: 8075637. DOI: 10.1055/s-0041-1728702.

References

Kurmis A . Understanding the limitations of the journal impact factor. J Bone Joint Surg Am. 2003; 85(12):2449-54. DOI: 10.2106/00004623-200312000-00028. View

van der Pol C, McInnes M, Petrcich W, Tunis A, Hanna R . Is quality and completeness of reporting of systematic reviews and meta-analyses published in high impact radiology journals associated with citation rates?. PLoS One. 2015; 10(3):e0119892. PMC: 4361663. DOI: 10.1371/journal.pone.0119892. View

Mimouni M, Segal O . Self-citation rate and impact factor in ophthalmology. Ophthalmic Res. 2014; 52(3):136-40. DOI: 10.1159/000366284. View

Rizkallah J, Sin D . Integrative approach to quality assessment of medical journals using impact factor, eigenfactor, and article influence scores. PLoS One. 2010; 5(4):e10204. PMC: 2855371. DOI: 10.1371/journal.pone.0010204. View

Henderson A, Upile T, Pilavakis Y, Patel N . Reporting guidelines and journal quality in otolaryngology. Clin Otolaryngol. 2015; 41(5):461-6. DOI: 10.1111/coa.12546. View

Miyamoto S . Self-citation rate and impact factor in the field of plastic and reconstructive surgery. J Plast Surg Hand Surg. 2017; 52(1):40-46. DOI: 10.1080/2000656X.2017.1319847. View

Yuen J . Comparison of Impact Factor, Eigenfactor Metrics, and SCImago Journal Rank Indicator and h-index for Neurosurgical and Spinal Surgical Journals. World Neurosurg. 2018; 119:e328-e337. DOI: 10.1016/j.wneu.2018.07.144. View

Fassoulaki A, Paraskeva A, Papilas K, Karabinis G . Self-citations in six anaesthesia journals and their significance in determining the impact factor. Br J Anaesth. 2000; 84(2):266-9. DOI: 10.1093/oxfordjournals.bja.a013418. View

Siebelt M, Siebelt T, Pilot P, Bloem R, Bhandari M, Poolman R . Citation analysis of orthopaedic literature; 18 major orthopaedic journals compared for Impact Factor and SCImago. BMC Musculoskelet Disord. 2010; 11:4. PMC: 2821374. DOI: 10.1186/1471-2474-11-4. View

10.

Butler J, Sebastian A, Kaye I, Wagner S, Morrissey P, Schroeder G . Understanding Traditional Research Impact Metrics. Clin Spine Surg. 2017; 30(4):164-166. DOI: 10.1097/BSD.0000000000000530. View

11.

Miro O, Brown A, Graham C, Ducharme J, Martin-Sanchez F, Cone D . Relationship between category size and journals' impact factor: implications for emergency medicine journals and researchers. Eur J Emerg Med. 2014; 22(5):355-62. DOI: 10.1097/MEJ.0000000000000184. View

12.

Garfield E . Citation analysis as a tool in journal evaluation. Science. 1972; 178(4060):471-9. DOI: 10.1126/science.178.4060.471. View

13.

Slobogean G, Bhandari M . Introducing levels of evidence to the Journal of Orthopaedic Trauma: implementation and future directions. J Orthop Trauma. 2012; 26(3):127-8. DOI: 10.1097/BOT.0b013e318247c931. View

14.

Mohamed N, Gwam C, Etcheson J, George N, Piuzzi N, Rosas S . Impact factors of orthopaedic journals between 2010 and 2016: trends and comparisons with other surgical specialties. Ann Transl Med. 2018; 6(7):114. PMC: 6015939. DOI: 10.21037/atm.2018.03.02. View

15.

Hawkinson M, Krueger C, Carroll J . Self-Citation Does Not Appear to Artificially Inflate Orthopaedic Journal Ranking. J Surg Orthop Adv. 2018; 27(2):131-135. View

16.

Garfield E . The history and meaning of the journal impact factor. JAMA. 2006; 295(1):90-3. DOI: 10.1001/jama.295.1.90. View

17.

Brown T . Journal quality metrics: options to consider other than impact factors. Am J Occup Ther. 2011; 65(3):346-50. DOI: 10.5014/ajot.2011.001396. View

18.

Moverley R, Rankin K, McNamara I, Davidson D, Reed M, Sprowson A . Impact factors of orthopaedic journals between 2000 and 2010: trends and comparisons with other surgical specialties. Int Orthop. 2013; 37(4):561-7. PMC: 3609972. DOI: 10.1007/s00264-012-1769-1. View

19.

Garfield E . How can impact factors be improved?. BMJ. 1996; 313(7054):411-3. PMC: 2351785. DOI: 10.1136/bmj.313.7054.411. View

20.

Garner R, Hirsch J, Albuquerque F, Fargen K . Bibliometric indices: defining academic productivity and citation rates of researchers, departments and journals. J Neurointerv Surg. 2017; 10(2):102-106. DOI: 10.1136/neurintsurg-2017-013265. View