Black dots are students who participated in test preparation (e.g., an expensive commercial course, or even a private tutor; see below) between taking the PSAT and taking the SAT. White dots are students who did not. Click for larger versions.
But who needs data like this when one can simply assert that it is obvious that test scores reflect SES or preparation, rather than actual ability? Even a casual investigation into this topic reveals that, at least on average, SAT scores are not easily improved, even through extensive effort. (Insert IQ or g score for SAT score if desired.)
Biggs: ... When researchers have estimated the effect of commercial test preparation programs on the SAT while taking the above factors into account, the effect of commercial test preparation has appeared relatively small. A comprehensive 1999 study by Don Powers and Don Rock published in the Journal of Educational Measurement estimated a coaching effect on the math section somewhere between 13 and 18 points, and an effect on the verbal section between 6 and 12 points. Powers and Rock concluded that the combined effect of coaching on the SAT I is between 21 and 34 points. Similarly, extensive metanalyses conducted by Betsy Jane Becker in 1990 and by Nan Laird in 1983 found that the typical effect of commercial preparatory courses on the SAT was in the range of 9-25 points on the verbal section, and 15-25 points on the math section.
One of the most remarkable aspects of this line of research has been the lack of impact it has had on the public consciousness. ...
... By far the largest effect sizes belong to the those preparation activities involving either a commercial course or private tutor [NEVERTHELESS THE SCORE CHANGES ARE NOT LARGE], and the effects differ for each section of the SAT. On average students with private tutors improve their math scores by 19 points more than those students without private tutors. The effect is less on the verbal section, where having a private tutor only improves scores on average by seven points. Taking a commercial course has a similarly large effect on math scores, improving them on average by 17 points, and has the largest effect on verbal scores, improving them on average by 13 points. With the exception of studying with a book, no other activity analyzed in this manner has an effect on test score changes that is statistically different from zero at a .05 significance level.
... Does test preparation help improve student performance on the SAT and ACT? For students that have taken the test before and would like to boost their scores, coaching seems to help, but by a rather small amount. After controlling for group differences, the average coaching boost on the math section of the SAT is 14 to 15 points. The boost is smaller on the verbal section of the test, just 6 to 8 points. The combined effect of coaching on the SAT for the NELS sample is about 20 points.
0 comments:
Post a Comment