Wednesday, 19 July 2017

Skinner on Science


I doubt B.F. Skinner* would have been a fan of what I do. I'm interested in the self-concept--which is about as inner as inner states get. Because he didn't think a science of inner states was possible, and also didn't think they were important, he'd have told me to work on something else. Nevertheless, I am a big fan of his work.

One book of hisI particularly like is Science and Human Behavio(u)r, first published in 1953. Because it covers a lot of ground it's hard to summari(s)ze all of the ideas in it in a blog post. So I won't try: I'll just focus on the first two chapters.

In those chapters Skinner does two things. First, he defends science, which he explains has recently come under attack. Second, he argues that, if we try hard enough, a science of human affairs is possible and, moreover, desirable.

Now, there are many takes on this thing called science. For Popper, science was about falsifiable theories. For Lakatos, it was about spectacular prediction. For Kuhn, science was periods of dull tradition-bound work punctuated by periods of rapid changes in thinking and method. For Polanyi, science was essentially an art, involving indefinable powers of thought. Feyerabend gave up all together on the idea that science had a special essence and concluded that anything goes**.

But, what was Skinner's view?

Well, for Skinner, science was crucially a mindset: "first of all a set of attitudes. It is a disposition to deal with the facts rather than with what someone has said about them... a willingness to accept facts even when they are opposed to wishes" (p. 12).

It was about honesty: "The result may confirm his theory, contradict it, or leave it in doubt. In spite of any inclination to the contrary, he must report a contradiction as readily as a confirmation. If he does not, someone else will... In the long run, the issue is not so much one of personal prestige as of effective procedure. Scientist have simply found that being honest--with oneself as much as with others--is essential to progress. Experiments do not always come out as one expects, but the facts must stand and the expectations fall" (p. 13).

It was about patience and a kind of intellectual chastity: "Scientists have also discovered the value of remaining without an answer until a satisfactory one can be found. This is a difficult lesson. It takes considerable training to avoid premature conclusions, to refrain from making statements on insufficient evidence, and to avoid explanations which are pure invention" (p. 13).

Science was about finding laws of nature: "It begins, as we all begin, by observing single episodes, but it quickly passes on to the general rule, to scientific law" (p. 13).

It was not concerned only with observation: "Science is not concerned just with "getting the facts", after which one may act with greater wisdom in an unscientific fashion... it leads to a new conception of a subject matter, a new way of thinking about that part of the world to which it has addressed itself" (p. 6).

It sought and assumed order: "it is an attempt to discover order, to show that certain events stand in lawful relations to other events... but order is not only a possible end product; it is a working assumption which must be adopted at the very start. We cannot apply the methods of science to a subject matter which is assumed to move about capriciously" (p. 6).

It predicted: "science not only describes, it predicts" (p. 6).

All in all, it was something special: "science is unique in showing a cumulative progress... a unique intellectual process which yields remarkable results" (p. 11).


* In this paper, recently identified by female psychologists as the psychologist with the most historical impact in psychology. For male psychologists he was the psychologist with the third greatest historical impact in psychology.

** Book-length philosophies of science, such as these, of course contain more nuanced visions of science than the sentence-long ones I have attributed to them. These are the core ideas or the ideas they are known best for.

Monday, 5 June 2017

80% Power for Pearson's r .01 to .98 According to R Package pwr

Statistical power and power analysis have been topics of discussion in psychological research since the 1960s (see Cohen, 1992 for a brief overview). Statistical power was defined by Cohen as "the long-term probability, given the population ES [e.g. r,  d ], alpha [e.g. .05, .01], and N [sample size], of rejecting the H0 [the null hypothesis]" (1992, p. 156). The bits in square brackets that ruin that quotation are mine. Sorry.

The idea of statistical power makes most (perfect) sense from the classic NHST statistical framework, where the researcher is thought of as drawing random samples from some larger population. Power analysis provides an answer to the following question: if (a) a population effect size of some magnitude exists, (b) I have a significance criterion of x for my statistical tests, and (c) I draw samples from a population, what sample size do I need if, in the long term, I am to correctly reject the null hypothesis with a certain probability?

The theoretically required sample size depends on the effect size anticipated, the significance criterion employed, and the desired long-term probability of rejecting the null hypothesis. In practice, the smaller the effect size anticipated, the smaller the significance criterion employed, and the greater the desired long term probability of rejecting the null hypothesis, the larger the sample size the researcher will require. Statistical power thus depends on the nature of your phenomena (strong versus weak), the nature of your statistical tests (conservative versus liberal), and your approach to uncertainty in the long term (risk-averse versus risk-taking).

According to Cohen (1992, p. 156) power depends on one other thing too: whether you are doing a one-sided or two-sided test. In the context of a Pearson's r correlation a two-sided test is when you don't have an a priori hypothesis as to the sign (+ versus -) of the correlation. A one-sided test is when you do. I have a feeling that in social and personality psychology most tests of correlations will be one-sided (e.g. that self-esteem is positively associated with life satisfaction, but negatively associated with depression). So if you use R keep in mind that the pwr package computes power for a two-sided test as default and the values in the table of Cohen (1992) are also for two-sided tests. Sample sizes for .10 should be used for one-side tests (p. 156). In my Masters I planned a sample size for a research project involving only one-side tests without knowing that. The practical implications of ignoring the one-side/two sided distinction will depend on effect size: at the low end of the effect size spectrum you might need a few hundred more participants for a two-sided test; as soon as your correlation reaches medium effect size territory, however, we're only looking at a dozen more.

To help get a feel for the theoretically required sample size for 80% power for cross-sectional work with Pearson's rs and to concretely describe power analysis, I have made the table below. It contains the sample size required for all whole number Pearson's rs from .01 through .98, for two-sided and one-sided tests provided by the R package pwr (which is based on Cohen, 1988). It also presents one-sided N as a proportion of two-sided N, as well as the difference in N  between the two. The sample sizes required are exactly the same for negative correlations.

For one-sided tests the following code is required:

pwr.r.test(n = , r = .10 , sig.level = .05, power = .80, alternative = "greater")

For two-sided tests the following code is required:

pwr.r.test(n = , r = .005 , sig.level = .05, power = .80)

r
Two-Sided
One-Sided
One-Sided as % of Two-Sided N
Two-Sided –
One-Sided
.01
78485.47
61823.42
78.77%
-16662.05
.02
19619.02
15454.24
78.77%
-4164.78
.03
8717.822
6867.354
78.77%
-1850.468
.04
4902.403
3861.944
78.78%
-1040.459
.05
3136.41
2470.869
78.78%
-665.541
.06
2177.105
1715.223
78.78%
-461.882
.07
1598.674
1259.593
78.79%
-339.081
.08
1223.25
963.8714
78.80%
-259.3786
.09
965.8606
761.1258
78.80%
-204.7348
.10
781.7516
616.1032
78.81%
-165.6484
.11
645.5317
508.8029
78.82%
-136.7288
.12
541.9252
427.1924
78.83%
-114.7328
.13
461.2951
363.6803
78.84%
-97.6148
.14
397.3175
313.2856
78.85%
-84.0319
.15
345.7037
272.6297
78.86%
-73.074
.16
303.4615
239.356
78.88%
-64.1055
.17
268.4522
211.7797
78.89%
-56.6725
.18
239.1141
188.6706
78.90%
-50.4435
.19
214.2852
169.1135
78.92%
-45.1717
.20
193.0867
152.416
78.94%
-40.6707
.21
174.8439
138.0468
78.95%
-36.7971
.22
159.0316
125.5922
78.97%
-33.4394
.23
145.2367
114.7266
78.99%
-30.5101
.24
133.1299
105.1909
79.01%
-27.939
.25
122.4466
96.77644
79.04%
-25.67016
.26
112.9721
89.31424
79.06%
-23.65786
.27
104.5307
82.66581
79.08%
-21.86489
.28
96.97745
76.71702
79.11%
-20.26043
.29
90.19199
71.37305
79.13%
-18.81894
.30
84.07364
66.55463
79.16%
-17.51901
.31
78.53776
62.19504
79.19%
-16.34272
.32
73.51267
58.23783
79.22%
-15.27484
.33
68.93741
54.63501
79.25%
-14.3024
.34
64.75983
51.34549
79.29%
-13.41434
.35
60.93514
48.33393
79.32%
-12.60121
.36
57.42466
45.56996
79.36%
-11.8547
.37
54.19491
43.02712
79.39%
-11.16779
.38
51.21669
40.68248
79.43%
-10.53421
.39
48.46455
38.51594
79.47%
-9.94861
.40
45.91614
36.50995
79.51%
-9.40619
.41
43.55186
34.64901
79.56%
-8.90285
.42
41.35434
32.91948
79.60%
-8.43486
.43
39.30827
31.30931
79.65%
-7.99896
.44
37.40001
29.80769
79.70%
-7.59232
.45
35.61745
28.40513
79.75%
-7.21232
.46
33.94976
27.09311
79.80%
-6.85665
.47
32.38727
25.86401
79.86%
-6.52326
.48
30.92127
24.71096
79.92%
-6.21031
.49
29.54402
23.6278
79.97%
-5.91622
.50
28.24841
22.60907
80.04%
-5.63934
.51
27.0281
21.64969
80.10%
-5.37841
.52
25.87738
20.74515
80.17%
-5.13223
.53
24.79101
19.89129
80.24%
-4.89972
.54
23.76423
19.08448
80.31%
-4.67975
.55
22.79277
18.32132
80.38%
-4.47145
.56
21.87264
17.59861
80.46%
-4.27403
.57
21.00035
16.91359
80.54%
-4.08676
.58
20.17253
16.26368
80.62%
-3.90885
.59
19.3862
15.64652
80.71%
-3.73968
.60
18.63858
15.05985
80.80%
-3.57873
.61
17.92718
14.50177
80.89%
-3.42541
.62
17.24958
13.97038
80.99%
-3.2792
.63
16.60367
13.46401
81.09%
-3.13966
.64
15.98746
12.98108
81.20%
-3.00638
.65
15.3991
12.52014
81.30%
-2.87896
.66
14.83688
12.07985
81.42%
-2.75703
.67
14.29923
11.65895
81.54%
-2.64028
.68
13.78464
11.25629
81.66%
-2.52835
.69
13.29182
10.87079
81.79%
-2.42103
.70
12.81943
10.50145
81.92%
-2.31798
.71
12.36629
10.14733
82.06%
-2.21896
.72
11.93129
9.807556
82.20%
-2.123734
.73
11.51337
9.481297
82.35%
-2.032073
.74
11.11156
9.167802
82.51%
-1.943758
.75
10.72496
8.866322
82.67%
-1.858638
.76
10.35266
8.576185
82.84%
-1.776475
.77
9.993879
8.296739
83.02%
-1.69714
.78
9.647812
8.027358
83.20%
-1.620454
.79
9.313733
7.76751
83.40%
-1.546223
.80
8.990898
7.516578
83.60%
-1.47432
.81
8.678654
7.274044
83.82%
-1.40461
.82
8.376314
7.039387
84.04%
-1.336927
.83
8.083248
6.812124
84.27%
-1.271124
.84
7.798814
6.591688
84.52%
-1.207126
.85
7.522376
6.377668
84.78%
-1.144708
.86
7.253296
6.16953
85.06%
-1.083766
.87
6.99092
5.966757
85.35%
-1.024163
.88
6.734556
5.768836
85.66%
-0.96572
.89
6.483499
5.575197
85.99%
-0.908302
.90
6.236905
5.385215
86.34%
-0.85169
.91
5.993871
5.198218
86.73%
-0.795653
.92
5.753285
5.013335
87.14%
-0.73995
.93
5.513883
4.829639
87.59%
-0.684244
.94
5.27376
4.645742
88.09%
-0.628018
.95
5.030555
4.459864
88.66%
-0.570691
.96
4.780486
4.269295
89.31%
-0.511191
.97
4.51724
4.069485
90.09%
-0.447755
.98
4.228144




Refs

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale,NJ: Lawrence Erlbaum. 

Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155-159.