Monday, 5 June 2017

80% Power for Pearson's r .01 to .98 According to R Package pwr

Statistical power and power analysis have been topics of discussion in psychological research since the 1960s (see Cohen, 1992 for a brief overview). Statistical power was defined by Cohen as "the long-term probability, given the population ES [e.g. r,  d ], alpha [e.g. .05, .01], and N [sample size], of rejecting the H0 [the null hypothesis]" (1992, p. 156). The bits in square brackets that ruin that quotation are mine. Sorry.

The idea of statistical power makes most (perfect) sense from the classic NHST statistical framework, where the researcher is thought of as drawing random samples from some larger population. Power analysis provides an answer to the following question: if (a) a population effect size of some magnitude exists, (b) I have a significance criterion of x for my statistical tests, and (c) I draw samples from a population, what sample size do I need if, in the long term, I am to correctly reject the null hypothesis with a certain probability?

The theoretically required sample size depends on the effect size anticipated, the significance criterion employed, and the desired long-term probability of rejecting the null hypothesis. In practice, the smaller the effect size anticipated, the smaller the significance criterion employed, and the greater the desired long term probability of rejecting the null hypothesis, the larger the sample size the researcher will require. Statistical power thus depends on the nature of your phenomena (strong versus weak), the nature of your statistical tests (conservative versus liberal), and your approach to uncertainty in the long term (risk-averse versus risk-taking).

According to Cohen (1992, p. 156) power depends on one other thing too: whether you are doing a one-sided or two-sided test. In the context of a Pearson's r correlation a two-sided test is when you don't have an a priori hypothesis as to the sign (+ versus -) of the correlation. A one-sided test is when you do. I have a feeling that in social and personality psychology most tests of correlations will be one-sided (e.g. that self-esteem is positively associated with life satisfaction, but negatively associated with depression). So if you use R keep in mind that the pwr package computes power for a two-sided test as default and the values in the table of Cohen (1992) are also for two-sided tests. Sample sizes for .10 should be used for one-side tests (p. 156). In my Masters I planned a sample size for a research project involving only one-side tests without knowing that. The practical implications of ignoring the one-side/two sided distinction will depend on effect size: at the low end of the effect size spectrum you might need a few hundred more participants for a two-sided test; as soon as your correlation reaches medium effect size territory, however, we're only looking at a dozen more.

To help get a feel for the theoretically required sample size for 80% power for cross-sectional work with Pearson's rs and to concretely describe power analysis, I have made the table below. It contains the sample size required for all whole number Pearson's rs from .01 through .98, for two-sided and one-sided tests provided by the R package pwr (which is based on Cohen, 1988). It also presents one-sided N as a proportion of two-sided N, as well as the difference in N  between the twoThe sample sizes required are exactly the same for negative correlations.

For one-sided tests the following code is required:

pwr.r.test(n = , r = .10 , sig.level = .05, power = .80, alternative = "greater")

For two-sided tests the following code is required:

pwr.r.test(n = , r = .005 , sig.level = .05, power = .80)

r
Two-Sided
One-Sided
One-Sided as % of Two-Sided N
Two-Sided –
One-Sided
.01
78485.47
61823.42
78.77%
-16662.05
.02
19619.02
15454.24
78.77%
-4164.78
.03
8717.822
6867.354
78.77%
-1850.468
.04
4902.403
3861.944
78.78%
-1040.459
.05
3136.41
2470.869
78.78%
-665.541
.06
2177.105
1715.223
78.78%
-461.882
.07
1598.674
1259.593
78.79%
-339.081
.08
1223.25
963.8714
78.80%
-259.3786
.09
965.8606
761.1258
78.80%
-204.7348
.10
781.7516
616.1032
78.81%
-165.6484
.11
645.5317
508.8029
78.82%
-136.7288
.12
541.9252
427.1924
78.83%
-114.7328
.13
461.2951
363.6803
78.84%
-97.6148
.14
397.3175
313.2856
78.85%
-84.0319
.15
345.7037
272.6297
78.86%
-73.074
.16
303.4615
239.356
78.88%
-64.1055
.17
268.4522
211.7797
78.89%
-56.6725
.18
239.1141
188.6706
78.90%
-50.4435
.19
214.2852
169.1135
78.92%
-45.1717
.20
193.0867
152.416
78.94%
-40.6707
.21
174.8439
138.0468
78.95%
-36.7971
.22
159.0316
125.5922
78.97%
-33.4394
.23
145.2367
114.7266
78.99%
-30.5101
.24
133.1299
105.1909
79.01%
-27.939
.25
122.4466
96.77644
79.04%
-25.67016
.26
112.9721
89.31424
79.06%
-23.65786
.27
104.5307
82.66581
79.08%
-21.86489
.28
96.97745
76.71702
79.11%
-20.26043
.29
90.19199
71.37305
79.13%
-18.81894
.30
84.07364
66.55463
79.16%
-17.51901
.31
78.53776
62.19504
79.19%
-16.34272
.32
73.51267
58.23783
79.22%
-15.27484
.33
68.93741
54.63501
79.25%
-14.3024
.34
64.75983
51.34549
79.29%
-13.41434
.35
60.93514
48.33393
79.32%
-12.60121
.36
57.42466
45.56996
79.36%
-11.8547
.37
54.19491
43.02712
79.39%
-11.16779
.38
51.21669
40.68248
79.43%
-10.53421
.39
48.46455
38.51594
79.47%
-9.94861
.40
45.91614
36.50995
79.51%
-9.40619
.41
43.55186
34.64901
79.56%
-8.90285
.42
41.35434
32.91948
79.60%
-8.43486
.43
39.30827
31.30931
79.65%
-7.99896
.44
37.40001
29.80769
79.70%
-7.59232
.45
35.61745
28.40513
79.75%
-7.21232
.46
33.94976
27.09311
79.80%
-6.85665
.47
32.38727
25.86401
79.86%
-6.52326
.48
30.92127
24.71096
79.92%
-6.21031
.49
29.54402
23.6278
79.97%
-5.91622
.50
28.24841
22.60907
80.04%
-5.63934
.51
27.0281
21.64969
80.10%
-5.37841
.52
25.87738
20.74515
80.17%
-5.13223
.53
24.79101
19.89129
80.24%
-4.89972
.54
23.76423
19.08448
80.31%
-4.67975
.55
22.79277
18.32132
80.38%
-4.47145
.56
21.87264
17.59861
80.46%
-4.27403
.57
21.00035
16.91359
80.54%
-4.08676
.58
20.17253
16.26368
80.62%
-3.90885
.59
19.3862
15.64652
80.71%
-3.73968
.60
18.63858
15.05985
80.80%
-3.57873
.61
17.92718
14.50177
80.89%
-3.42541
.62
17.24958
13.97038
80.99%
-3.2792
.63
16.60367
13.46401
81.09%
-3.13966
.64
15.98746
12.98108
81.20%
-3.00638
.65
15.3991
12.52014
81.30%
-2.87896
.66
14.83688
12.07985
81.42%
-2.75703
.67
14.29923
11.65895
81.54%
-2.64028
.68
13.78464
11.25629
81.66%
-2.52835
.69
13.29182
10.87079
81.79%
-2.42103
.70
12.81943
10.50145
81.92%
-2.31798
.71
12.36629
10.14733
82.06%
-2.21896
.72
11.93129
9.807556
82.20%
-2.123734
.73
11.51337
9.481297
82.35%
-2.032073
.74
11.11156
9.167802
82.51%
-1.943758
.75
10.72496
8.866322
82.67%
-1.858638
.76
10.35266
8.576185
82.84%
-1.776475
.77
9.993879
8.296739
83.02%
-1.69714
.78
9.647812
8.027358
83.20%
-1.620454
.79
9.313733
7.76751
83.40%
-1.546223
.80
8.990898
7.516578
83.60%
-1.47432
.81
8.678654
7.274044
83.82%
-1.40461
.82
8.376314
7.039387
84.04%
-1.336927
.83
8.083248
6.812124
84.27%
-1.271124
.84
7.798814
6.591688
84.52%
-1.207126
.85
7.522376
6.377668
84.78%
-1.144708
.86
7.253296
6.16953
85.06%
-1.083766
.87
6.99092
5.966757
85.35%
-1.024163
.88
6.734556
5.768836
85.66%
-0.96572
.89
6.483499
5.575197
85.99%
-0.908302
.90
6.236905
5.385215
86.34%
-0.85169
.91
5.993871
5.198218
86.73%
-0.795653
.92
5.753285
5.013335
87.14%
-0.73995
.93
5.513883
4.829639
87.59%
-0.684244
.94
5.27376
4.645742
88.09%
-0.628018
.95
5.030555
4.459864
88.66%
-0.570691
.96
4.780486
4.269295
89.31%
-0.511191
.97
4.51724
4.069485
90.09%
-0.447755
.98
4.228144




Refs

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale,NJ: Lawrence Erlbaum. 

Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155-159.