> Once scholars control for all these factors as best they can, they find that coaching has a positive but small effect: Perhaps 10 or 20 points in total on the SAT, mostly on the math section, according to careful work by Derek Briggs of the University of Colorado Boulder and Ben Domingue of Stanford University.
> A 2010 study led by Ohio State’s Claudia Buchmann and based on the National Education Longitudinal Study found that the use of books, videos, and computer software offers no boost whatsoever, while a private test prep course adds 30 points to your score on average, and the use of a private tutor adds 37.
Also, managed funds are not analogous to interview/test preparation in any way.
10 to 20 points on a 2400 scale (at the time of study) test is not a substantial impact. I also think there are probably test-oriented lurking variables they are not able to capture.
Managed funds also sell you on the idea that you can buy your way into performance, and there is also little evidence that they actually work.
> Let’s assume the effects of short-term coaching are really just a 20- or 30-point jump in students’ scores. That means they ought to be irrelevant to college admissions officers. Briggs found otherwise, however. Analyzing a 2008 survey conducted by the National Association for College Admission Counseling, he noted that one-third of respondents described a jump from 750 to 770 on the math portion of the SAT as having a significant effect on a student’s chances of admissions, and this was true among counselors at more and less selective schools alike. Even a minor score improvement for a high-achieving student, then—and one that falls within the standard measurement error for the test—can make a real difference.
> Compare that with students from families that earn less than $20,000 per year, who have a mean score on the math portion of the SAT around 450. According to the same admissions counselor survey, a 20-point improvement to a score in this range would have no practical meaning for students who are trying to get into more selective schools. Even when it comes to less selective schools, just 20 percent of the counselors said a boost from 430 to 450 would make a difference. In other words, research has debunked the myth that pricey test prep gives a major bump to students’ scores, but it’s also hinted that whatever modest bumps they do provide are more likely to help the people who are already at the top.
======
As for managed funds, they promise to sell performance within an environment composed of many many people. Test prep and interview prep promise to improve performance for individuals. One is about mastery over an external environment, nature. The other is about internal improvement. I don't think promises of personal improvement through coaching, training, or teaching is comparable to promises of gains within a large chaotic unpredictable environment. They're entirely different categories of "buying your way into performance."
> he noted that one-third of respondents described a jump from 750 to 770 on the math portion of the SAT as having a significant effect on a student’s chances of admissions
But did the SAT prep really help people go from 750 to 770? I'd guess that most of that 20 point jump would be for people who start out at 500 or so, and going from 500 to 520 isn't terribly significant. Fixing the last points is much much harder than the middle, so I really doubt that people at that level would see the same absolute gain as people in the middle.
Agreed. Indeed, I would expect the "average" to mostly consist of large boosts for very low initial scorers rather than improvements among high-scorers.
So, in a poll about how their own industry works, 66% vs. 33% said that a 20 point difference (from 750 -> 770, which is not the standard score change from prep and the changes are lowest at the high end of the score range) would not make any significant difference in the chance that a student would have.
I don't follow the distinction between market as external environment and college admissions as internal environment. Could you elaborate?
> would not make any significant difference in the chance that a student would have
So a third is not statistically important?
> I don't follow the distinction between market as external environment and college admissions as internal environment. Could you elaborate?
It's a very simple point: fund managers claim to have mastery over a vast, complex external environment. Interview and test prep courses, like any sort of pedagogy, claim to teach and coach candidates towards self-improvement. The latter is self-evidently simply more achievable than the former. And even if you were to claim that SAT prep schools make no substantiative improvement for students, what makes interview prep most analogous to those, as opposed to other forms of tutoring, or even personal fitness training, which do yield greater benefits?
I'm just saying it's a lot easier to teach someone how to be better at something than to beat the market. They are completely different endeavors.
Is that true?
https://slate.com/technology/2019/04/sat-prep-courses-do-the...
> Once scholars control for all these factors as best they can, they find that coaching has a positive but small effect: Perhaps 10 or 20 points in total on the SAT, mostly on the math section, according to careful work by Derek Briggs of the University of Colorado Boulder and Ben Domingue of Stanford University.
> A 2010 study led by Ohio State’s Claudia Buchmann and based on the National Education Longitudinal Study found that the use of books, videos, and computer software offers no boost whatsoever, while a private test prep course adds 30 points to your score on average, and the use of a private tutor adds 37.
Also, managed funds are not analogous to interview/test preparation in any way.