PSY 429 Experimental Psychology
Class Exercise #10
Power and Strength of Association Tests
Answers
PSY 429 Experimental Psychology
Class Exercise #10
Power and Strength of Association Tests
Answers
1. If someone asks you, "What level of power did you achieve in your study?" and you answer, "Power turned out to be .90," what did the question refer to and what does your answer mean?
Power is the probability of correctly rejecting the null hypothesis when it is false. So when the power is .90, that means that 90% of the time we will correctly reject the null hypothesis when it is false.
2. What four factors affect the power of a statistical test?
a. Strength of association between the IV and DV: if strength is strong, power tends to be high--we will be more likely to detect the relationship between the IV and DV and hence, more likely to correctly reject the null hypothesis.b. Sample size: large sample sizes mean that we will be more likely to detect an existing difference between group means and hence, power will tend to be high.
c. Alpha level: with a low or conservative alpha level (e.g. .01), the chances we will reject the null hypothesis is also low since the differences between means must be larger in order to detect such differences. Thus, we will tend to increase the chances of overlooking a difference, or missing being able to detect a difference (Type II error) when we adopt a low alpha level. An alpha level of .05 results in more power than one of .01.
d. Directional versis non-directional test: when seeing whether there is a difference between means of a two-group experiment, a directional or one-tailed test results in more power since the value of t in the t-test will be larger in a one-tailed test than in a two-tailed test, everything else being equal, and thus, more likely to judge the difference between the groups as significant. However, this is only true in a two-group experiment when the t-test is used.
3. What does the eta-squared statistic reflect? (don't just say the strength of association between IV and DV)?
The stronger the influence of the IV on the DV, the more the overall total variability in the data (SS-total) should be due to treatment effects (SS-treatment) as opposed to error (SS-error). Therefore, the proportion of total variability due to treatment effects should be large. Eta-squared measures the proportion of total variability in the DV due to treatment (IV).
4. Translate into words what an eta-squared of .65 means.
An eta-squared of .65 means that 65% of the variability in the DV is explained by treatment effects or the IV. The remainder (35%) is explained by other variables not under study.
5. The following experiment measured manual dexerity in four participants over a series of three practice sessions. The goal of the experiment was to examine changes in manual dexerity performance as a function of practice. Results showed the following sum of squares (SS):
SS-treatment = 14.0
SS-error = 6.0
SS-subjects = 12.0
SS-total = 32.0
What is the strength of association between the IV and the DV and describe what this means.
eta-squared = .70That means that 70% of the variability in the dependent variable (performance on manual dexterity task) is explained by the independent variable (practice). Putting it another way, 70% of the variability in performance is due to practice.
6. Based on question #5 above, what is the level of power with an alpha level of .05 (assume an equal number of participants in each group)?
power = .99 (Table 7)
7. A school psychologist would like to test the effectiveness of a behavior-modification technique in controlling classroom outbursts of unruly children. A teacher is instructed to use the response-cost technique. Every time a child disrupts the class, he or she is told that the behavior has cost him or her 10 minutes of free time. That is, the free-time period is shortened for each unruly act. For a sample of n = 4 children, the number of outbursts is measured for a day before the treatment is initiated and again 1 week, 1 month, and 6 months after the response-cost technique began.
Results showed the following sum of squares (SS):
SS-treatment = 77.0
SS-error = 11.0
SS-subjects = 13.0
SS-total = 101.0
a. What is the F ratio?F = 21.04b. Is there a significant difference?
Yes. According to the F table, the critical value of F is 3.86. Thus, reject the null hypothesis.c. Conduct a strength of association test using eta-squared and describe in words what this means.
eta squared = .875; This means that 87.5% of the variability in the number of outbursts is due to the response-cost technique.d. Determine the level of power and describe in words what this means.
power > .99; This means that the chances are over 99% that we will correctly reject the null hypothesis when it is false. (Table 7)
8. It has been suggested that pupil size increases during emotional arousal. A researcher would therefore like to see if the increase in pupil size is a function of the type of arousal (pleasant versus aversive). A random sample of five subjects is selected for the study. Each subject views all three stimuli: neutral, pleasant, and aversive photographs. The neutral photograph portrays a plain brick building. The pleasant photograph consists of a young man and woman sharing a large lic cream cone. Finally, the aversive stimulus is a graphic photograph of an automobile accident. Upon viewing each stimulus, the pupil size is measured (in millimeters) with sophisticated equipment. The results show the following sum of squares (SS):
SS-treatment = 30.0
SS-error = 28.0
SS-subjects = 6.0
SS-total = 64.0
a. What is the F ratio?F = 4.29b. Is there a significant difference?
No. According to the F table, the critical value of F is 4.467. Thus, fail to reject the null hypothesis.c. Conduct a strength of association test using eta-squared and describe in words what this means.
eta squared = .52. This means that 52% of the variability in the number of outbursts is due to the response-cost technique. This is a moderate to moderately high eta-squared and perhaps indicate that the IV is exerting a definite effect on the DV. One conclusion to draw from this might be that significant differences may be found if the sample size were larger.d. Determine the level of power and describe in words what this means.
power > .70; This means that the chances are about 70% that we will correctly reject the null hypothesis when it is false. It also means that power can be increased to .95 relatively cheaply by increasing the number of subjects to 9. (Table 7)
To go back to Dr. Ryan's Home Page, click on kitty.
Celtic
Clip Art - http://artinfo@aon-celtic.com