Blogs on on psychology, psychometrics, and statistics that help me to remember what I've read and done and think.
Wednesday, 19 July 2017
Skinner on Science
I doubt B.F. Skinner* would have been a fan of what I do. I'm interested in the self-concept--which is about as inner as inner states get. Because he didn't think a science of inner states was possible, and also didn't think they were important, he'd have told me to work on something else. Nevertheless, I am a big fan of his work.
One book of hisI particularly like is Science and Human Behavio(u)r, first published in 1953. Because it covers a lot of ground it's hard to summari(s)ze all of the ideas in it in a blog post. So I won't try: I'll just focus on the first two chapters.
In those chapters Skinner does two things. First, he defends science, which he explains has recently come under attack. Second, he argues that, if we try hard enough, a science of human affairs is possible and, moreover, desirable.
Now, there are many takes on this thing called science. For Popper, science was about falsifiable theories. For Lakatos, it was about spectacular prediction. For Kuhn, science was periods of dull tradition-bound work punctuated by periods of rapid changes in thinking and method. For Polanyi, science was essentially an art, involving indefinable powers of thought. Feyerabend gave up all together on the idea that science had a special essence and concluded that anything goes**.
But, what was Skinner's view?
Well, for Skinner, science was crucially a mindset: "first of all a set of attitudes. It is a disposition to deal with the facts rather than with what someone has said about them... a willingness to accept facts even when they are opposed to wishes" (p. 12).
It was about honesty: "The result may confirm his theory, contradict it, or leave it in doubt. In spite of any inclination to the contrary, he must report a contradiction as readily as a confirmation. If he does not, someone else will... In the long run, the issue is not so much one of personal prestige as of effective procedure. Scientist have simply found that being honest--with oneself as much as with others--is essential to progress. Experiments do not always come out as one expects, but the facts must stand and the expectations fall" (p. 13).
It was about patience and a kind of intellectual chastity: "Scientists have also discovered the value of remaining without an answer until a satisfactory one can be found. This is a difficult lesson. It takes considerable training to avoid premature conclusions, to refrain from making statements on insufficient evidence, and to avoid explanations which are pure invention" (p. 13).
Science was about finding laws of nature: "It begins, as we all begin, by observing single episodes, but it quickly passes on to the general rule, to scientific law" (p. 13).
It was not concerned only with observation: "Science is not concerned just with "getting the facts", after which one may act with greater wisdom in an unscientific fashion... it leads to a new conception of a subject matter, a new way of thinking about that part of the world to which it has addressed itself" (p. 6).
It sought and assumed order: "it is an attempt to discover order, to show that certain events stand in lawful relations to other events... but order is not only a possible end product; it is a working assumption which must be adopted at the very start. We cannot apply the methods of science to a subject matter which is assumed to move about capriciously" (p. 6).
It predicted: "science not only describes, it predicts" (p. 6).
All in all, it was something special: "science is unique in showing a cumulative progress... a unique intellectual process which yields remarkable results" (p. 11).
* In this paper, recently identified by female psychologists as the psychologist with the most historical impact in psychology. For male psychologists he was the psychologist with the third greatest historical impact in psychology.
** Book-length philosophies of science, such as these, of course contain more nuanced visions of science than the sentence-long ones I have attributed to them. These are the core ideas or the ideas they are known best for.
Monday, 5 June 2017
80% Power for Pearson's r .01 to .98 According to R Package pwr
Statistical power and power analysis have been topics of discussion in psychological research since the 1960s (see Cohen, 1992 for a brief overview). Statistical power was defined by Cohen as "the long-term probability, given the population ES [e.g. r, d ], alpha [e.g. .05, .01], and N [sample size], of rejecting the H0 [the null hypothesis]" (1992, p. 156). The bits in square brackets that ruin that quotation are mine. Sorry.
The idea of statistical power makes most (perfect) sense from the classic NHST statistical framework, where the researcher is thought of as drawing random samples from some larger population. Power analysis provides an answer to the following question: if (a) a population effect size of some magnitude exists, (b) I have a significance criterion of x for my statistical tests, and (c) I draw samples from a population, what sample size do I need if, in the long term, I am to correctly reject the null hypothesis with a certain probability?
The theoretically required sample size depends on the effect size anticipated, the significance criterion employed, and the desired long-term probability of rejecting the null hypothesis. In practice, the smaller the effect size anticipated, the smaller the significance criterion employed, and the greater the desired long term probability of rejecting the null hypothesis, the larger the sample size the researcher will require. Statistical power thus depends on the nature of your phenomena (strong versus weak), the nature of your statistical tests (conservative versus liberal), and your approach to uncertainty in the long term (risk-averse versus risk-taking).
According to Cohen (1992, p. 156) power depends on one other thing too: whether you are doing a one-sided or two-sided test. In the context of a Pearson's r correlation a two-sided test is when you don't have an a priori hypothesis as to the sign (+ versus -) of the correlation. A one-sided test is when you do. I have a feeling that in social and personality psychology most tests of correlations will be one-sided (e.g. that self-esteem is positively associated with life satisfaction, but negatively associated with depression). So if you use R keep in mind that the pwr package computes power for a two-sided test as default and the values in the table of Cohen (1992) are also for two-sided tests. Sample sizes for .10 should be used for one-side tests (p. 156). In my Masters I planned a sample size for a research project involving only one-side tests without knowing that. The practical implications of ignoring the one-side/two sided distinction will depend on effect size: at the low end of the effect size spectrum you might need a few hundred more participants for a two-sided test; as soon as your correlation reaches medium effect size territory, however, we're only looking at a dozen more.
To help get a feel for the theoretically required sample size for 80% power for cross-sectional work with Pearson's rs and to concretely describe power analysis, I have made the table below. It contains the sample size required for all whole number Pearson's rs from .01 through .98, for two-sided and one-sided tests provided by the R package pwr (which is based on Cohen, 1988). It also presents one-sided N as a proportion of two-sided N, as well as the difference in N between the two. The sample sizes required are exactly the same for negative correlations.
For one-sided tests the following code is required:
pwr.r.test(n = , r = .10 , sig.level = .05, power = .80, alternative = "greater")
For two-sided tests the following code is required:
pwr.r.test(n = , r = .005 , sig.level = .05, power = .80)
The idea of statistical power makes most (perfect) sense from the classic NHST statistical framework, where the researcher is thought of as drawing random samples from some larger population. Power analysis provides an answer to the following question: if (a) a population effect size of some magnitude exists, (b) I have a significance criterion of x for my statistical tests, and (c) I draw samples from a population, what sample size do I need if, in the long term, I am to correctly reject the null hypothesis with a certain probability?
The theoretically required sample size depends on the effect size anticipated, the significance criterion employed, and the desired long-term probability of rejecting the null hypothesis. In practice, the smaller the effect size anticipated, the smaller the significance criterion employed, and the greater the desired long term probability of rejecting the null hypothesis, the larger the sample size the researcher will require. Statistical power thus depends on the nature of your phenomena (strong versus weak), the nature of your statistical tests (conservative versus liberal), and your approach to uncertainty in the long term (risk-averse versus risk-taking).
According to Cohen (1992, p. 156) power depends on one other thing too: whether you are doing a one-sided or two-sided test. In the context of a Pearson's r correlation a two-sided test is when you don't have an a priori hypothesis as to the sign (+ versus -) of the correlation. A one-sided test is when you do. I have a feeling that in social and personality psychology most tests of correlations will be one-sided (e.g. that self-esteem is positively associated with life satisfaction, but negatively associated with depression). So if you use R keep in mind that the pwr package computes power for a two-sided test as default and the values in the table of Cohen (1992) are also for two-sided tests. Sample sizes for .10 should be used for one-side tests (p. 156). In my Masters I planned a sample size for a research project involving only one-side tests without knowing that. The practical implications of ignoring the one-side/two sided distinction will depend on effect size: at the low end of the effect size spectrum you might need a few hundred more participants for a two-sided test; as soon as your correlation reaches medium effect size territory, however, we're only looking at a dozen more.
To help get a feel for the theoretically required sample size for 80% power for cross-sectional work with Pearson's rs and to concretely describe power analysis, I have made the table below. It contains the sample size required for all whole number Pearson's rs from .01 through .98, for two-sided and one-sided tests provided by the R package pwr (which is based on Cohen, 1988). It also presents one-sided N as a proportion of two-sided N, as well as the difference in N between the two. The sample sizes required are exactly the same for negative correlations.
For one-sided tests the following code is required:
pwr.r.test(n = , r = .10 , sig.level = .05, power = .80, alternative = "greater")
For two-sided tests the following code is required:
pwr.r.test(n = , r = .005 , sig.level = .05, power = .80)
r
|
Two-Sided
|
One-Sided
|
One-Sided as % of Two-Sided N
|
Two-Sided –
One-Sided
|
.01
|
78485.47
|
61823.42
|
78.77%
|
-16662.05
|
.02
|
19619.02
|
15454.24
|
78.77%
|
-4164.78
|
.03
|
8717.822
|
6867.354
|
78.77%
|
-1850.468
|
.04
|
4902.403
|
3861.944
|
78.78%
|
-1040.459
|
.05
|
3136.41
|
2470.869
|
78.78%
|
-665.541
|
.06
|
2177.105
|
1715.223
|
78.78%
|
-461.882
|
.07
|
1598.674
|
1259.593
|
78.79%
|
-339.081
|
.08
|
1223.25
|
963.8714
|
78.80%
|
-259.3786
|
.09
|
965.8606
|
761.1258
|
78.80%
|
-204.7348
|
.10
|
781.7516
|
616.1032
|
78.81%
|
-165.6484
|
.11
|
645.5317
|
508.8029
|
78.82%
|
-136.7288
|
.12
|
541.9252
|
427.1924
|
78.83%
|
-114.7328
|
.13
|
461.2951
|
363.6803
|
78.84%
|
-97.6148
|
.14
|
397.3175
|
313.2856
|
78.85%
|
-84.0319
|
.15
|
345.7037
|
272.6297
|
78.86%
|
-73.074
|
.16
|
303.4615
|
239.356
|
78.88%
|
-64.1055
|
.17
|
268.4522
|
211.7797
|
78.89%
|
-56.6725
|
.18
|
239.1141
|
188.6706
|
78.90%
|
-50.4435
|
.19
|
214.2852
|
169.1135
|
78.92%
|
-45.1717
|
.20
|
193.0867
|
152.416
|
78.94%
|
-40.6707
|
.21
|
174.8439
|
138.0468
|
78.95%
|
-36.7971
|
.22
|
159.0316
|
125.5922
|
78.97%
|
-33.4394
|
.23
|
145.2367
|
114.7266
|
78.99%
|
-30.5101
|
.24
|
133.1299
|
105.1909
|
79.01%
|
-27.939
|
.25
|
122.4466
|
96.77644
|
79.04%
|
-25.67016
|
.26
|
112.9721
|
89.31424
|
79.06%
|
-23.65786
|
.27
|
104.5307
|
82.66581
|
79.08%
|
-21.86489
|
.28
|
96.97745
|
76.71702
|
79.11%
|
-20.26043
|
.29
|
90.19199
|
71.37305
|
79.13%
|
-18.81894
|
.30
|
84.07364
|
66.55463
|
79.16%
|
-17.51901
|
.31
|
78.53776
|
62.19504
|
79.19%
|
-16.34272
|
.32
|
73.51267
|
58.23783
|
79.22%
|
-15.27484
|
.33
|
68.93741
|
54.63501
|
79.25%
|
-14.3024
|
.34
|
64.75983
|
51.34549
|
79.29%
|
-13.41434
|
.35
|
60.93514
|
48.33393
|
79.32%
|
-12.60121
|
.36
|
57.42466
|
45.56996
|
79.36%
|
-11.8547
|
.37
|
54.19491
|
43.02712
|
79.39%
|
-11.16779
|
.38
|
51.21669
|
40.68248
|
79.43%
|
-10.53421
|
.39
|
48.46455
|
38.51594
|
79.47%
|
-9.94861
|
.40
|
45.91614
|
36.50995
|
79.51%
|
-9.40619
|
.41
|
43.55186
|
34.64901
|
79.56%
|
-8.90285
|
.42
|
41.35434
|
32.91948
|
79.60%
|
-8.43486
|
.43
|
39.30827
|
31.30931
|
79.65%
|
-7.99896
|
.44
|
37.40001
|
29.80769
|
79.70%
|
-7.59232
|
.45
|
35.61745
|
28.40513
|
79.75%
|
-7.21232
|
.46
|
33.94976
|
27.09311
|
79.80%
|
-6.85665
|
.47
|
32.38727
|
25.86401
|
79.86%
|
-6.52326
|
.48
|
30.92127
|
24.71096
|
79.92%
|
-6.21031
|
.49
|
29.54402
|
23.6278
|
79.97%
|
-5.91622
|
.50
|
28.24841
|
22.60907
|
80.04%
|
-5.63934
|
.51
|
27.0281
|
21.64969
|
80.10%
|
-5.37841
|
.52
|
25.87738
|
20.74515
|
80.17%
|
-5.13223
|
.53
|
24.79101
|
19.89129
|
80.24%
|
-4.89972
|
.54
|
23.76423
|
19.08448
|
80.31%
|
-4.67975
|
.55
|
22.79277
|
18.32132
|
80.38%
|
-4.47145
|
.56
|
21.87264
|
17.59861
|
80.46%
|
-4.27403
|
.57
|
21.00035
|
16.91359
|
80.54%
|
-4.08676
|
.58
|
20.17253
|
16.26368
|
80.62%
|
-3.90885
|
.59
|
19.3862
|
15.64652
|
80.71%
|
-3.73968
|
.60
|
18.63858
|
15.05985
|
80.80%
|
-3.57873
|
.61
|
17.92718
|
14.50177
|
80.89%
|
-3.42541
|
.62
|
17.24958
|
13.97038
|
80.99%
|
-3.2792
|
.63
|
16.60367
|
13.46401
|
81.09%
|
-3.13966
|
.64
|
15.98746
|
12.98108
|
81.20%
|
-3.00638
|
.65
|
15.3991
|
12.52014
|
81.30%
|
-2.87896
|
.66
|
14.83688
|
12.07985
|
81.42%
|
-2.75703
|
.67
|
14.29923
|
11.65895
|
81.54%
|
-2.64028
|
.68
|
13.78464
|
11.25629
|
81.66%
|
-2.52835
|
.69
|
13.29182
|
10.87079
|
81.79%
|
-2.42103
|
.70
|
12.81943
|
10.50145
|
81.92%
|
-2.31798
|
.71
|
12.36629
|
10.14733
|
82.06%
|
-2.21896
|
.72
|
11.93129
|
9.807556
|
82.20%
|
-2.123734
|
.73
|
11.51337
|
9.481297
|
82.35%
|
-2.032073
|
.74
|
11.11156
|
9.167802
|
82.51%
|
-1.943758
|
.75
|
10.72496
|
8.866322
|
82.67%
|
-1.858638
|
.76
|
10.35266
|
8.576185
|
82.84%
|
-1.776475
|
.77
|
9.993879
|
8.296739
|
83.02%
|
-1.69714
|
.78
|
9.647812
|
8.027358
|
83.20%
|
-1.620454
|
.79
|
9.313733
|
7.76751
|
83.40%
|
-1.546223
|
.80
|
8.990898
|
7.516578
|
83.60%
|
-1.47432
|
.81
|
8.678654
|
7.274044
|
83.82%
|
-1.40461
|
.82
|
8.376314
|
7.039387
|
84.04%
|
-1.336927
|
.83
|
8.083248
|
6.812124
|
84.27%
|
-1.271124
|
.84
|
7.798814
|
6.591688
|
84.52%
|
-1.207126
|
.85
|
7.522376
|
6.377668
|
84.78%
|
-1.144708
|
.86
|
7.253296
|
6.16953
|
85.06%
|
-1.083766
|
.87
|
6.99092
|
5.966757
|
85.35%
|
-1.024163
|
.88
|
6.734556
|
5.768836
|
85.66%
|
-0.96572
|
.89
|
6.483499
|
5.575197
|
85.99%
|
-0.908302
|
.90
|
6.236905
|
5.385215
|
86.34%
|
-0.85169
|
.91
|
5.993871
|
5.198218
|
86.73%
|
-0.795653
|
.92
|
5.753285
|
5.013335
|
87.14%
|
-0.73995
|
.93
|
5.513883
|
4.829639
|
87.59%
|
-0.684244
|
.94
|
5.27376
|
4.645742
|
88.09%
|
-0.628018
|
.95
|
5.030555
|
4.459864
|
88.66%
|
-0.570691
|
.96
|
4.780486
|
4.269295
|
89.31%
|
-0.511191
|
.97
|
4.51724
|
4.069485
|
90.09%
|
-0.447755
|
.98
|
4.228144
|
Refs
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale,NJ:
Lawrence Erlbaum.
Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155-159.
Subscribe to:
Posts (Atom)
