A Modified Sequential Probability Ratio Test

Overview

Journal J Math Psychol

Date 2022 May 2

PMID 35496657

Authors

Sandipan Pramanik

Valen E Johnson

Anirban Bhattacharya

Affiliations

Soon will be listed here.

Abstract

We describe a modified sequential probability ratio test that can be used to reduce the average sample size required to perform statistical hypothesis tests at specified levels of significance and power. Examples are provided for tests, tests, and tests of binomial success probabilities. A description of a software package to implement the test designs is provided. We compare the sample sizes required in fixed design tests conducted at 5% significance levels to the average sample sizes required in sequential tests conducted at 0.5% significance levels, and we find that the two sample sizes are approximately equal.

Citing Articles

Accommodating Serial Correlation and Sequential Design Elements in Personalized Studies and Aggregated Personalized Studies.

Schork N Harv Data Sci Rev. 2023; 2022(SI3).

PMID: 37032736 PMC: 10081537. DOI: 10.1162/99608f92.f1eef6f4.

Efficient alternatives for Bayesian hypothesis tests in psychology.

Pramanik S, Johnson V Psychol Methods. 2022; 29(2):243-261.

PMID: 35420854 PMC: 9561355. DOI: 10.1037/met0000482.

References

Johnson V . UNIFORMLY MOST POWERFUL BAYESIAN TESTS. Ann Stat. 2014; 41(4):1716-1741. PMC: 3960084. DOI: 10.1214/13-AOS1123. View

DeMets D, Lan K . Interim analysis: the alpha spending function approach. Stat Med. 1994; 13(13-14):1341-52; discussion 1353-6. DOI: 10.1002/sim.4780131308. View

Kelley K, Bilson Darku F, Chattopadhyay B . Accuracy in parameter estimation for a general class of effect sizes: A sequential approach. Psychol Methods. 2017; 23(2):226-243. DOI: 10.1037/met0000127. View

Schonbrodt F, Wagenmakers E, Zehetleitner M, Perugini M . Sequential hypothesis testing with Bayes factors: Efficiently testing mean differences. Psychol Methods. 2015; 22(2):322-339. DOI: 10.1037/met0000061. View

Chattopadhyay B, Kelley K . Estimating the standardized mean difference with minimum risk: Maximizing accuracy and minimizing cost with sequential estimation. Psychol Methods. 2016; 22(1):94-113. DOI: 10.1037/met0000089. View

Nikooienejad A, Johnson V . On the Existence of Uniformly Most Powerful Bayesian Tests With Application to Non-Central Chi-Squared Tests. Bayesian Anal. 2021; 16(1):93-109. PMC: 8189570. DOI: 10.1214/19-ba1194. View

Dreber A, Pfeiffer T, Almenberg J, Isaksson S, Wilson B, Chen Y . Using prediction markets to estimate the reproducibility of scientific research. Proc Natl Acad Sci U S A. 2015; 112(50):15343-7. PMC: 4687569. DOI: 10.1073/pnas.1516179112. View

Kelley K, Bilson Darku F, Chattopadhyay B . Sequential accuracy in parameter estimation for population correlation coefficients. Psychol Methods. 2019; 24(4):492-515. DOI: 10.1037/met0000203. View

OBrien P, Fleming T . A multiple testing procedure for clinical trials. Biometrics. 1979; 35(3):549-56. View

10.

Benjamin D, Berger J, Johannesson M, Nosek B, Wagenmakers E, Berk R . Redefine statistical significance. Nat Hum Behav. 2019; 2(1):6-10. DOI: 10.1038/s41562-017-0189-z. View

11.

. PSYCHOLOGY. Estimating the reproducibility of psychological science. Science. 2015; 349(6251):aac4716. DOI: 10.1126/science.aac4716. View

12.

Amrhein V, Greenland S, McShane B . Scientists rise up against statistical significance. Nature. 2019; 567(7748):305-307. DOI: 10.1038/d41586-019-00857-9. View

13.

Johnson V, Payne R, Wang T, Asher A, Mandal S . On the Reproducibility of Psychological Science. J Am Stat Assoc. 2018; 112(517):1-10. PMC: 5976261. DOI: 10.1080/01621459.2016.1240079. View

14.

Johnson V . Revised standards for statistical evidence. Proc Natl Acad Sci U S A. 2013; 110(48):19313-7. PMC: 3845140. DOI: 10.1073/pnas.1313476110. View

15.

Pike H . Statistical significance should be abandoned, say scientists. BMJ. 2019; 364:l1374. DOI: 10.1136/bmj.l1374. View

16.

Haybittle J . Repeated assessment of results in clinical trials of cancer treatment. Br J Radiol. 1971; 44(526):793-7. DOI: 10.1259/0007-1285-44-526-793. View

17.

Shih M, Lai T, Heyse J, Chen J . Sequential generalized likelihood ratio tests for vaccine safety evaluation. Stat Med. 2010; 29(26):2698-708. PMC: 2975406. DOI: 10.1002/sim.4036. View

18.

Savalei V, Dunn E . Is the call to abandon p-values the red herring of the replicability crisis?. Front Psychol. 2015; 6:245. PMC: 4351564. DOI: 10.3389/fpsyg.2015.00245. View