Power and Sample Size Calculation Tools

2022-12-13

To install and load pwrss:

install.packages("pwrss")
library(pwrss)

pwrss R package allows statistical power and minimum required sample size calculations for

• (1) testing a proportion (one-sample) against a constant,
• (2) testing a mean (one-sample) against a constant,
  
• (3) testing difference between two proportions (independent samples),
• (4) testing difference between two means (independent and paired samples),
• (5) testing a correlation (one-sample) against a constant,
  
• (6) testing difference between two correlations (independent samples),
• (7) testing a coefficient (standardized or unstandardized) against a constant in multiple linear regression,
• (8) testing an indirect effect (standardized or unstandardized) in the mediation analysis (Sobel, Joint, and Monte Carlo),
• (9) testing an R-squared against zero in linear regression
• (10) testing an R-squared difference against zero in hierarchical regression
• (11) testing an eta-squared or f-squared (for main and interaction effects) against zero in analysis of variance (ANOVA) (could be one-way, two-way, and three-way),
• (12) testing an eta-squared or f-squared (for main and interaction effects) against zero in analysis of covariance (ANCOVA) (could be one-way, two-way, and three-way),
• (13) testing an eta-squared or f-squared (for between, within, and interaction effects) against zero in one-way repeated measures analysis of variance (RM-ANOVA) (with non-sphericity correction and repeated measures correlation).

Alternative hypothesis can be formulated as "not equal", "less", "greater", "non-inferior", "superior", or "equivalent" in (1), (2), (3), and (4); as "not equal", "less", or "greater" in (5), (6), (7) and (8); but always as "greater" in (9), (10), (11), (12), and (13). If you find the package and related material useful, along with other references, please cite as:

• Bulus, M. (2022). pwrss: Power and Sample Size Calculation Tools. R package version 0.1.0. https://CRAN.R-project.org/package=pwrss

• Bulus, M., & Polat, C. (2022, November 7). Istatistiksel Guc Analizine Giris [Introduction to Statistical Power Analysis]. https://doi.org/10.35542/osf.io/tfyxq

A Few Snapshots

1. Generic Functions for z, t, and F tests

These generic functions calculate and return statistical power along with optional plots for Type I and Type II error rates (as long as the test statistic and degrees of freedom is known).

power.t.test(ncp = 1.96, df = 99, alpha = 0.05, alternative = "equivalent")

power.z.test(ncp = 1.96, alpha = 0.05, alternative = "equivalent")

power.f.test(ncp = 2, df1 = 2, df2 = 98, alpha = 0.05)

plot() function (S3 method) is a wrapper around the generic functions above.

# comparing two means
design1 <- pwrss.t.2means(mu1 = 0.20, margin = -0.05, paired = TRUE,
                         power = 0.80, alternative = "non-inferior")
plot(design1)

# ANCOVA design
design2 <- pwrss.f.ancova(eta2 = 0.10, n.levels = c(2,3),
                         power = .80)
plot(design2)

# indirect effect in mediation analysis
design3 <- pwrss.z.med(a = 0.10, b = 0.20, cp = 0.10,
                         power = .80)
plot(design3)

2. Standardizd versus Unstandardized Input for t Test

# it is sufficient to provide standardized difference between two groups for 'mu1'
# e.g. Cohen's d = 0.50 for mu1, because mu2 = 0 by default
pwrss.t.2means(mu1 = 0.50, power = .80)

##  Difference between two means (independent samples t test) 
##  H0: mu1 = mu2 
##  HA: mu1 != mu2 
##  ------------------------------ 
##   Statistical power = 0.8 
##   n1 = 64 
##   n2 = 64 
##  ------------------------------ 
##  Alternative = "not equal" 
##  Degrees of freedom = 125.54 
##  Non-centrality parameter = 2.823 
##  Type I error rate = 0.05 
##  Type II error rate = 0.2

pwrss.t.2means(mu1 = 0.50, power = .80, paired  = TRUE)

##  Difference between two means (paired samples t test) 
##  H0: mu1 = mu2 
##  HA: mu1 != mu2 
##  ------------------------------ 
##   Statistical power = 0.8 
##   n = 34 
##  ------------------------------ 
##  Alternative = "not equal" 
##  Degrees of freedom = 32.37 
##  Non-centrality parameter = 2.889 
##  Type I error rate = 0.05 
##  Type II error rate = 0.2

# it is sufficient to provide pooled standard deviation for sd1
# because sd2 = sd1 by default
pwrss.t.2means(mu1 = 10, mu2 = 5, sd1 = 10, power = .80)

##  Difference between two means (independent samples t test) 
##  H0: mu1 = mu2 
##  HA: mu1 != mu2 
##  ------------------------------ 
##   Statistical power = 0.8 
##   n1 = 64 
##   n2 = 64 
##  ------------------------------ 
##  Alternative = "not equal" 
##  Degrees of freedom = 125.54 
##  Non-centrality parameter = 2.823 
##  Type I error rate = 0.05 
##  Type II error rate = 0.2

3. One-way, Two-way, and Three-way Analysis of Variance (ANOVA) or Analysis of Covariance (ANCOVA)

# one-way ANOVA
pwrss.f.ancova(n.levels = 2, n.covariates = 0,
               power = .80, eta2 = 0.10)

##  One-way Analysis of Variance (ANOVA) 
##   H0: 'eta2' or 'f2' = 0 
##   HA: 'eta2' or 'f2' > 0 
##  ------------------------------------
##  Factor A: 2 levels 
##  ------------------------------------
##  Given eta2 = 0.1 or f2 = 0.111 
##   Total n = 73 
##  ------------------------------------
##  Numerator degrees of freedom =  1 
##  Denominator degrees of freedom = 70.61 
##  Non-centrality parameter = 8.07

# one-way ANCOVA
pwrss.f.ancova(n.levels = 2, n.covariates = 1,
               power = .80, eta2 = 0.08)

##  One-way Analysis of Covariance (ANCOVA) 
##   H0: 'eta2' or 'f2' = 0 
##   HA: 'eta2' or 'f2' > 0 
##  ------------------------------------
##  Factor A: 2 levels 
##  ------------------------------------
##  Given eta2 = 0.08 or f2 = 0.087 
##   Total n = 93 
##  ------------------------------------
##  Numerator degrees of freedom =  1 
##  Denominator degrees of freedom = 89.24 
##  Non-centrality parameter = 8.02

# two-way ANOVA
pwrss.f.ancova(n.levels = c(2,3), n.covariates = 0,
               power = .80, eta2 = 0.10)

##  Two-way Analysis of Variance (ANOVA) 
##   H0: 'eta2' or 'f2' = 0 
##   HA: 'eta2' or 'f2' > 0 
##  ------------------------------------
##  Factor A: 2 levels 
##  Factor B: 3 levels 
##  ------------------------------------
##  Given eta2 = 0.1 or f2 = 0.111 
##   Total n = 73 (for A) 
##   Total n = 90 (for B) 
##   Total n = 90 (for A x B) 
##  ------------------------------------
##  Numerator degrees of freedom =  1 2 2 
##  Denominator degrees of freedom = 66.73 83.89 83.89 
##  Non-centrality parameter = 8.08 9.99 9.99

# two-way ANCOVA
pwrss.f.ancova(n.levels = c(2,3), n.covariates = 1,
               power = .80, eta2 = 0.08)

##  Two-way Analysis of Covariance (ANCOVA) 
##   H0: 'eta2' or 'f2' = 0 
##   HA: 'eta2' or 'f2' > 0 
##  ------------------------------------
##  Factor A: 2 levels 
##  Factor B: 3 levels 
##  ------------------------------------
##  Given eta2 = 0.08 or f2 = 0.087 
##   Total n = 93 (for A) 
##   Total n = 114 (for B) 
##   Total n = 114 (for A x B) 
##  ------------------------------------
##  Numerator degrees of freedom =  1 2 2 
##  Denominator degrees of freedom = 85.34 106.96 106.96 
##  Non-centrality parameter = 8.03 9.91 9.91

# three-way ANOVA
pwrss.f.ancova(n.levels = c(2,3,2), n.covariates = 0,
               power = .80, eta2 = 0.10)

##  Three-way Analysis of Variance (ANOVA) 
##   H0: 'eta2' or 'f2' = 0 
##   HA: 'eta2' or 'f2' > 0 
##  ------------------------------------
##  Factor A: 2 levels 
##  Factor B: 3 levels 
##  Factor C: 2 levels 
##  ------------------------------------
##  Given eta2 = 0.1 or f2 = 0.111 
##   Total n = 73 (for A) 
##   Total n = 91 (for B) 
##   Total n = 73 (for C) 
##   Total n = 91 (for A x B) 
##   Total n = 73 (for A x C) 
##   Total n = 91 (for B x C) 
##   Total n = 91 (for A x B x C) 
##  ------------------------------------
##  Numerator degrees of freedom =  1 2 1 2 1 2 2 
##  Denominator degrees of freedom = 60.93 78.13 60.93 78.13 60.93 78.13 78.13 
##  Non-centrality parameter = 8.1 10.01 8.1 10.01 8.1 10.01 10.01

# three-way ANCOVA
pwrss.f.ancova(n.levels = c(2,3,2), n.covariates = 1,
               power = .80, eta2 = 0.08)

##  Three-way Analysis of Covariance (ANCOVA) 
##   H0: 'eta2' or 'f2' = 0 
##   HA: 'eta2' or 'f2' > 0 
##  ------------------------------------
##  Factor A: 2 levels 
##  Factor B: 3 levels 
##  Factor C: 2 levels 
##  ------------------------------------
##  Given eta2 = 0.08 or f2 = 0.087 
##   Total n = 93 (for A) 
##   Total n = 115 (for B) 
##   Total n = 93 (for C) 
##   Total n = 115 (for A x B) 
##   Total n = 93 (for A x C) 
##   Total n = 115 (for B x C) 
##   Total n = 115 (for A x B x C) 
##  ------------------------------------
##  Numerator degrees of freedom =  1 2 1 2 1 2 2 
##  Denominator degrees of freedom = 79.49 101.15 79.49 101.15 79.49 101.15 101.15 
##  Non-centrality parameter = 8.04 9.93 8.04 9.93 8.04 9.93 9.93

4. One-way Repeated Measures Analysis of Variance (RM-ANOVA)

pwrss.f.rmanova(eta2 = 0.10,  n.levels = 2, n.measurements = 3,
                 repmeasures.r = 0.50, type = "between", power = 0.80)

##  One-way repeated measures analysis of variance (F test) 
##  H0: eta2 = 0 (or f2 = 0) 
##  HA: eta2 > 0 (or f2 > 0) 
##  ------------------------------ 
##  Number of levels (groups) = 2 
##  Number of measurement time points = 3 
##  ------------------------------ 
##   Statistical power = 0.8 
##   Total n = 50 
##  ------------------------------ 
##  Type of the effect = "between" 
##  Numerator degrees of freedom = 1 
##  Denominator degrees of freedom = 47.09 
##  Non-centrality parameter = 8.182 
##  Type I error rate = 0.05 
##  Type II error rate = 0.2

pwrss.f.rmanova(eta2 = 0.10,  n.levels = 2, n.measurements = 3,
                repmeasures.r = 0.50, epsilon = 1,
                type = "within", power = 0.80)

##  One-way repeated measures analysis of variance (F test) 
##  H0: eta2 = 0 (or f2 = 0) 
##  HA: eta2 > 0 (or f2 > 0) 
##  ------------------------------ 
##  Number of levels (groups) = 2 
##  Number of measurement time points = 3 
##  ------------------------------ 
##   Statistical power = 0.8 
##   Total n = 17 
##  ------------------------------ 
##  Type of the effect = "within" 
##  Numerator degrees of freedom = 2 
##  Denominator degrees of freedom = 28.207 
##  Non-centrality parameter = 10.736 
##  Type I error rate = 0.05 
##  Type II error rate = 0.2

pwrss.f.rmanova(eta2 = 0.10,  n.levels = 2, n.measurements = 3,
                repmeasures.r = 0.50, epsilon = 1,
                type = "interaction", power = 0.80)

##  One-way repeated measures analysis of variance (F test) 
##  H0: eta2 = 0 (or f2 = 0) 
##  HA: eta2 > 0 (or f2 > 0) 
##  ------------------------------ 
##  Number of levels (groups) = 2 
##  Number of measurement time points = 3 
##  ------------------------------ 
##   Statistical power = 0.8 
##   Total n = 17 
##  ------------------------------ 
##  Type of the effect = "interaction" 
##  Numerator degrees of freedom = 2 
##  Denominator degrees of freedom = 28.207 
##  Non-centrality parameter = 10.736 
##  Type I error rate = 0.05 
##  Type II error rate = 0.2

5. Multiple Linear Regression

# regression coefficient for a continuous predictor
pwrss.t.reg(beta = 0.20, r2 = 0.041, k = 1,
            power = 0.80)

##  Linear regression coefficient (one sample t test) 
##  H0: beta = beta0 
##  HA: beta != beta0 
##  ------------------------------ 
##   Statistical power = 0.8 
##   n = 191 
##  ------------------------------ 
##  Degrees of freedom = 188.116 
##  Non-centrality parameter = 2.816 
##  Type I error rate = 0.05 
##  Type II error rate = 0.2

pwrss.z.reg(beta = 0.20, r2 = 0.041, 
            power = 0.80)

##  Linear regression coefficient (one sample z test) 
##  H0: beta = beta0 
##  HA: beta != beta0 
##  ------------------------------ 
##   Statistical power = 0.8 
##   n = 189 
##  ------------------------------ 
##  Non-centrality parameter = 2.802 
##  Type I error rate = 0.05 
##  Type II error rate = 0.2

# regression coefficient for a binary predictor
p <- 0.50 # proportion of subjects in one group
pwrss.t.reg(beta = 0.20, r2 = 0.041, k = 1,
            sdx = sqrt(p*(1-p)), power = 0.80)

##  Linear regression coefficient (one sample t test) 
##  H0: beta = beta0 
##  HA: beta != beta0 
##  ------------------------------ 
##   Statistical power = 0.8 
##   n = 755 
##  ------------------------------ 
##  Degrees of freedom = 752.631 
##  Non-centrality parameter = 2.805 
##  Type I error rate = 0.05 
##  Type II error rate = 0.2

# R-squared against zero
pwrss.f.reg(r2 = 0.041, k = 1,
            power = 0.80)

##  R-squared compared to 0 in linear regression (F test) 
##  H0: r2 = 0 
##  HA: r2 > 0 
##  ------------------------------ 
##   Statistical power = 0.8 
##   n = 186 
##  ------------------------------ 
##  Numerator degrees of freedom = 1 
##  Denominator degrees of freedom = 183.527 
##  Non-centrality parameter = 7.932 
##  Type I error rate = 0.05 
##  Type II error rate = 0.2

# R-squared difference against zero in hierarchical regression
pwrss.f.reg(r2 = 0.041, k = 5, m = 3,
            power = 0.80)

##  R-squared change in hierarchical linear regression (F test) 
##  H0: r2 = 0 
##  HA: r2 > 0 
##  ------------------------------ 
##   Statistical power = 0.8 
##   n = 259 
##  ------------------------------ 
##  Numerator degrees of freedom = 3 
##  Denominator degrees of freedom = 252.985 
##  Non-centrality parameter = 11.072 
##  Type I error rate = 0.05 
##  Type II error rate = 0.2

6. Indirect Effect in Mediation Analysis

# indirect effect in mediation analysis
# X (cont.), M (cont.) , Z (cont.)
pwrss.z.med(a = 0.25, b = 0.25, cp = 0.10,
            n = 200)

##  Indirect effect in the mediation model
##  H0: a * b = 0 
##  HA: a * b != 0 
##  ------------------------------ 
##  Statistical power for 
##        Sobel's z test = 0.721 
##       Aroian's z test = 0.705 
##      Goodman's z test = 0.738 
##            Joint test = 0.902 
##      Monte Carlo test = 0.911 
##  ------------------------------ 
##  Sample size for 
##        Sobel's z test = 200 
##       Aroian's z test = 200 
##      Goodman's z test = 200 
##  ------------------------------ 
##  Non-centrality parameter for 
##        Sobel's z test = 2.547 
##       Aroian's z test = 2.499 
##      Goodman's z test = 2.598 
##  ------------------------------ 
##  Type I error rate = 0.05

# X (binary), M (cont.) , Z (cont.)
p <- 0.50 # proportion of subjects in one group
pwrss.z.med(a = 0.25, b = 0.25, cp = 0.10,
            sdx = sqrt(p*(1-p)), n = 200)

##  Indirect effect in the mediation model
##  H0: a * b = 0 
##  HA: a * b != 0 
##  ------------------------------ 
##  Statistical power for 
##        Sobel's z test = 0.359 
##       Aroian's z test = 0.342 
##      Goodman's z test = 0.379 
##            Joint test = 0.409 
##      Monte Carlo test = 0.39 
##  ------------------------------ 
##  Sample size for 
##        Sobel's z test = 200 
##       Aroian's z test = 200 
##      Goodman's z test = 200 
##  ------------------------------ 
##  Non-centrality parameter for 
##        Sobel's z test = 1.599 
##       Aroian's z test = 1.552 
##      Goodman's z test = 1.651 
##  ------------------------------ 
##  Type I error rate = 0.05

# covariate adjusted mediator and outcome model
# X (cont.), M (cont.) , Z (cont.)
pwrss.z.med(a = 0.25, b = 0.25, cp = 0.10,
            r2m.x = 0.50, r2y.mx = 0.50, n = 200)

##  Indirect effect in the mediation model
##  H0: a * b = 0 
##  HA: a * b != 0 
##  ------------------------------ 
##  Statistical power for 
##        Sobel's z test = 0.823 
##       Aroian's z test = 0.813 
##      Goodman's z test = 0.833 
##            Joint test = 0.941 
##      Monte Carlo test = 0.935 
##  ------------------------------ 
##  Sample size for 
##        Sobel's z test = 200 
##       Aroian's z test = 200 
##      Goodman's z test = 200 
##  ------------------------------ 
##  Non-centrality parameter for 
##        Sobel's z test = 2.887 
##       Aroian's z test = 2.849 
##      Goodman's z test = 2.926 
##  ------------------------------ 
##  Type I error rate = 0.05

# covariate adjusted mediator and outcome model
# X (binary), M (cont.) , Z (cont.)
p <- 0.50 # proportion of subjects in one group
pwrss.z.med(a = 0.25, b = 0.25, cp = 0.10,
            sdx = sqrt(p*(1-p)), 
            r2m.x = 0.50, r2y.mx = 0.50, n = 200)

##  Indirect effect in the mediation model
##  H0: a * b = 0 
##  HA: a * b != 0 
##  ------------------------------ 
##  Statistical power for 
##        Sobel's z test = 0.532 
##       Aroian's z test = 0.512 
##      Goodman's z test = 0.555 
##            Joint test = 0.665 
##      Monte Carlo test = 0.661 
##  ------------------------------ 
##  Sample size for 
##        Sobel's z test = 200 
##       Aroian's z test = 200 
##      Goodman's z test = 200 
##  ------------------------------ 
##  Non-centrality parameter for 
##        Sobel's z test = 2.041 
##       Aroian's z test = 1.989 
##      Goodman's z test = 2.098 
##  ------------------------------ 
##  Type I error rate = 0.05

--o--