This is a great step for accessibility, but will not go well with Collegeboard's level of incompetence. I took the GRE (administered by the same company that owns collegeboard) remotely online and as soon as I started the test their absolute spyware of a lockdown browser immediately closed. Despite me passing the system check immediately before the test, the lockdown browser thought I was cheating and immediately shut off the test without recourse. After waiting 2 weeks, they came back saying they would not refund me and despite some back and forth, the rep I was interacting with seemed convinced I was cheating and would not give me any further details on how to prevent the issue in the future or any way to further appeal my case or get a refund.
I retook the test a couple months later inperson at a testing center (where they used computers with the same lockdown browser) and got 99th percentile scores on everything, and I would recommend anyone planning to take the SAT to still take it at a testing center.
Ah, I misread the article. I completely missed the line mentioning that the test was still inperson, and thought that the "bring your own device" thing implied remote testing. I'm glad it's still an in-person proctored exam, since then if people have issues with their own devices they can hopefully switch to a provided one
If anything has to be installed, I’d say it would still be wise to use a provided device. I didn’t notice anything about it in the article, though the mention of a tablet option makes me assume it’s web based.
If you are based in the US, try to make a claim with your credit card or, if more adventurous, small claims court to sue. It seems pretty easy to win. Also, send a letter to your attorneys general office to explain what happened.
> got 99th percentile scores on everything
This part was unnecessary to include. Nothing but humblebragging there.
I disagree. It shows that the original test taker was less likely to be cheating than a lower scoring person and the system was at more likely to be at fault.
Not really, since if you don’t trust their account of the story in the first place (that they didn’t cheat), why would you trust that part? It adds nothing.
You should immunize yourself to devices like this having any impact for the same reason you should immunize yourself to cheap devices like “I normally hate X but” and “as an X, I think that” in other online comments.
I don't understand the '99th percentile' aside as a way to convince HN readers that the commenter wasn't cheating.
Rather, I take it as an expression of personal pride (in the positive sense) that there was such incontrovertible evidence that the commenter had no need to cheat in the first place, told with the satisfying (but probably unrealised) hope that the people who unjustifiably accused the commenter of cheating were later aware of the top marks he or she achieved after taking the exam in person.
I did seriously consider charging back, but considering it would probably get me banned from taking further tests and a couple grad schools I was applying to wanted the scores, I didn't pursue it.
I mentioned it in a few other replies, but my intent was just to show that I didnt need to cheat. I really dont think GRE scores are worth bragging about and I wish I could go back and edit my comment seeing as it clearly didn't come across as intended.
Your comment comes off as "people who score well on tests are more often cheaters", supported by anecdotal evidence. Above all, it's hardly relevant to the overall discussion and feels more like a grievance.
A student who has excess bandwidth after acing a test is more likely to want to poke and explore the limitations of that system than a student who, at capacity, is almost perfect.
Unscientific, but I'd bet there are two major cheating populations: 95%+, and something like 50-75%. By definition there are a lot more of the mid-tier-barely-getting-by cheaters, but they also aren't as smart and are only dangerous to societal trust in aggregate.
Come to think of it, I missed the third major population: pre-meds
You're right. I misremembered. I went back to check the reports. I ended up with a perfect score in math (which is only 96th according to the score report and 1 point off on the reading section, which was 99th.
I only brought up the scores to indicate that I didn't need to cheat. I don't think GRE scores are a valid metric of competency or anything worth bragging about. I'm sorry it was taken in another way
Maybe, maybe not. For this sort of test there is a huge selection bias since only people planning to work in math or an adjacent field bother to take the test. If you gave the same test to the general population this wouldn’t happen. I’m guessing most selective schools treat it as sort of a pass/fail, and I’m not sure whether that’s problematic for them or not.
It's definitely my recollection from the GRE (though this was many years ago now that I took it): for math a perfect score got you a percentile in the mid-to-high-90s, while on the opposite side you could lose several points and still be 99th percentile in verbal.
The percentile will vary by whatever the testing period is, since the percentile is the rank among other people taking the text within the same testing interval.
I don't think anyone in their right mind brags about GRE scores, and I'm sorry it came off that way. I mentioned it solely because as other comments said, I have no reason to cheat.
When I took the GRE in 2017, everything was digital as well. But I couldn't believe how locked down the experience was. The testing center in downtown SF had no windows, a metal detector, and you had to check all food outside of the testing room. Just in case you were going to try to cheat off of your banana. I think each computer was also recording a video of you taking the test although I might be mistaken here. Certainly there were cameras set up all around the perimeter.
I'm shocked at how laissez-faire we've gotten for college entrance exams, which I'd arguably say are much more important at getting you _into_ the graduate programs than the actual graduate program exams are themselves.
> I'm shocked at how laissez-faire we've gotten for college entrance exams,
to the point that they stopped requiring SAT altogether, but are starting to bring it back. there's a lot of things they've done is head scratching. i loved when they changed the scores so that your result from 10 years prior suddenly become a lower result so people had to know when you took the test to properly weight the result.
The laissez-faire attitude to drivers licensing when they stopped requiring road tests for new drivers was bonkers as well to me.
> i loved when they changed the scores so that your result from 10 years prior suddenly become a lower result so people had to know when you took the test to properly weight the result.
I took the GRE just before they switched from scoring out of 800 on each section to scoring out of 180. After the switch they kindly mailed me “converted” scores on the new 180 scale, which turned my perfect math score—I hadn’t missed a single question—into a 178. I was not amused.
I believe they percentile-mapped it. Like that guy, I too got a full score on Mathematics (it's very easy) and that was p96 when I took it so I bet I got downscored too. I would almost certainly have been full score if it had had the tougher questions.
I believe that most states require road tests to get your first license or to get a license if you have been unlicensed for a while but stopped requiring them for a renewal, at least if you've got a clean driving record.
In at least some states this was based on data that showed that for people that had clean records the rate of road test failures was very low, and those were almost always due to test anxiety and the person would pass on a second or third test.
Many states no longer require a road test. Instead you take a state approved driving course and the instructor certifies you as meeting the state driving skill requirements. Most states with this system do have an option to still take a state administered driving test instead but most new drivers just take the course. My daughter got her license last August and we paid for the course. They didn’t teach her anything. They had her drive around the local neighborhood for the state mandated numbers of hours for the course and the “instructor” then signed off. All real driving instruction she received came from me.
In my state the real benefit of taking the course it that scheduling a road test with the DMV often has a multi-month waitlist. Then, if you fail, you have to wait another 2-3 months to retest.
A better lesson than the state approved course was paying for a morning on the skid pad at the local raceway with a real driving instrcutor. Once your young driver understands how to regain control of the car, they are much better prepared for the first emergency maneuver they will need to make while driving solo.
Most of what they need to know includes knowing road signs and road markings, right of way, going through intersections, following traffic lights, speed limits, keeping safe distance, and even recognizing aggressive and impaired drivers to stay away from. None of that they will learn on the race track.
You can definitely push the car to the limit and see what happens when you go over in a _somewhat_ controlled environment (laws of physics still apply at the race track and you can still crash). Or you can teach them that a good driver is not the one who drives fast but the one who arrives alive.
You don't want your first experience at the traction limits of the vehicle to be a time when the road is wet and a wrong move causes a head-on collision with a school bus.
In my state the motorcycle licensing works similarly in that you can skip the state test, however unlike what you described for cars, the course is excellent (standard MSF course). Also, the MSF does administer a (pretty tough) test.
It has devolved into a pay for play scheme. I cut a check for a few hundred dollars and you stamp the form saying my kid can drive. You can also take the on the road driving course through your public school at no cost but the wait for that can be extremely long. Most people in my area just pay. A parent does also have to sign the form stating the kid has completed a certain number of hours driving under a certain mix of conditions with them. I think it is a total of 45 or 50 hours.
Over the last 15 years Virginia has tried to move as many DMV processes as possible to online and closed many DMV locations to save money. The elimination of the road test for most new drivers was part of this effort.
You should’ve lead with being from Virginia. That entire state is layers of pay for play schemes. They might’ve removed road tests to “save money” but they’re more than happy to keep up car inspection, emissions, registration and a vehicle value tax.
Our previous governor tried to eliminate state safety inspections but the auto repair shops were able to successfully lobby to block it. Emissions inspections are only required in parts of the state and are a federal requirement which the state can’t avoid. As for vehicle personal property tax that is a local tax and not a state imposed tax.
The actual drivers license (the one you can get at 18yo) does require road test in most states. The learner's permit that one can get at 14yo does not require road test, but when you apply for full driver's license at 18yo you will have to take it.
In my state you can’t get a learners permit until 16. You get your drivers license after having your learners for 6 month, completing a classroom and a road driving class. There is no road driving test involved at any point.
It is state dependent, yes, but in all the states I looked at 16 you get restricted driver's license. You can drive without an adult but you cannot have more than one passenger, cannot drive at night, cannot use your phone, etc etc. In many states this still requires road test.
The "adult" driver's license can be acquired at 18.
When I first got my driver's license in California, I had to do a test driving on the streets. The driving test also required driving on the freeway, but the examiner waived it. When I asked why, he said they waive all freeway driving because "it's too dangerous for the examiners". Sounds fair, if not legal.
I subsequently got my license.
When I moved to Oregon years later and re-applied for a license, a driving test was required but the DMV lady waived it. Why? Probably because the DMV was way out in the country and she was literally the only one working the office.
Yes, when I received my license, I had to do a road test with a state trooper riding shotgun. Four years later, my younger sibling did not have to do the road test. I don't know what the particulars were of qualifying to not have to do the test, but taking a certified driver's ed class was still a requirement. So it's not like there was someone just taking a multiple choice test with no behind the wheel training.
I’m sure it very much depends on locality, but I watched an elderly person backup over some cones and then drive out ahead of me with their new license. The road test also consisted of essentially going around the block a few times. I was 18 so there were no prerequisites. If you took the test and the evaluator gave you a pass, you got your license.
Explains all the collisions I pass on the way to dropping the kids off at school nearly every day.
There is also no testing or vision rechecking requirement, so there are just a lot of people on the roads who should probably have their license taken away.
Michigan takes car culture seriously. You have to parallel park on the drivers test, and the Secretary of State offices (there is no DMV, because what a waste of taxpayer funds...) have eye charts when you renew your license.
CA here. The eye charts are a joke. They’re visible at all times so you could easily memorize them during your long wait (even with appointment); and the specific line they have you read is huge. I’ve never looked into it but I suspect it’s just to show you can read a sign on the highway.
Your comment seems odd. If the vision test is enough to show you can read a sign on the highway, why does that make it a joke?
Are you suggesting that someone shouldn't have a license if their vision is not correctable to 20/20, even if it's good enough for all essential driving tasks?
It’s a joke because the testing method is inadequate to prevent even the simplest of cheating and does not adequately test that the driver’s vision legally qualifies for an unrestricted license.
I never realized our test was anything strange. Parallel parking was the big scary part of the test but I remember it being huge, almost two car lengths long or something. It was very easy.
The eye chart machines (they are electronic and usually at each clerk window) look like they are from the 70s for what its worth.
WA allows renewal by mail without even stepping into an office.
WA also allows transferring of drivers licenses without additional testing. To me, this is the most problematic part, because different states have different rules around priority, proper usage of the left lane, etc. and WA is a transplant destination, and the resulting mix of driver backgrounds is chaotic. As a general example, in WA pedestrians have priority at all intersections, including where there is no visibly marked crosswalk, but this is not true in all states.
Young people get into more crashes (and more deadly crashes) than old people, we should probably just flat out not allow people under 25 to drive long before we get around to worrying about things like vision checks.
Young people have more accidents because they have less experience driving. Raising the driving age doesn't help with that. If anything it should be lowered so that more of the learning period occurs when people are still under parental supervision.
It's also because younger people are more impulsive. If the driving age were raised to 25 there would be fewer accidents among beginner drivers. Of course it would be terrible in so many other ways, but the point is the accidents are not just about lack of experience.
I agree that having more of a supervised learning period would be a good idea.
> It's also because younger people are more impulsive.
This is more of a stereotype than a universal constant, and you could just as easily say things like "younger people have faster reflexes and better eyesight" etc.
The question isn't whether some group in the aggregate has different statistical properties, it's whether we're willing to discriminate against an entire class of people, including the ones who aren't the cause of it, because of that statistical correlation.
Which is the same reason people saying things like "people over 65 should have their license taken away" are just as wrong.
> This is more of a stereotype than a universal constant,
It's a well-known fact.
You may also observe how prisons are full of young people. Brains that are still developing are far more likely to make really stupid decisions without regard for consequences.
If I were to pick qualities for a driver, and had to choose between 'Makes good decisions' versus 'Faster reflexes', I'd pick the former any day.
Note that we don't let 12 year-olds drive. We have arbitrary age cutoffs that have nothing to do with quantifiable capability.
> Brains that are still developing are far more likely to make really stupid decisions without regard for consequences.
Then how come there are so many more 40 year olds in prison than 20 year olds?
> If I were to pick qualities for a driver, and had to choose between 'Makes good decisions' versus 'Faster reflexes', I'd pick the former any day.
But now we're back to experience, and you have to be allowed to do the thing before you can get that.
> Note that we don't let 12 year-olds drive. We have arbitrary age cutoffs that have nothing to do with quantifiable capability.
We do, but we shouldn't. If a 12-year-old can pass the driver's test then either that 12-year-old is competent to drive or your driver's test is inadequate in testing drivers.
> how come there are so many more 40 year olds in prison than 20 year olds?
You'd want to look at the age at which someone committed the crime for which they are imprisoned, not their current age, to assess the affect of age on crime-related statistics. Also, the juvenile justice system typically lets people out at 18 or 20, unless they were charged as an adult. So you don't have the same buildup of late teenagers being imprisoned at 20 as you do for people who are older.
It is factually the case that brain development continues until around age 25. The impulsive areas of the brain develop/myelinate earlier than than the prefrontal cortex, for example, which exercises executive function and impulse control.
> It is factually the case that brain development continues until around age 25
No, its factually the case that brain development continues for most of life, with different capacities peaking at wildly different ages on average (with some peaking after others have gone into significant decline), and considerable variation between individuals in each capacity, with executive function, specifically, reaching a plateau starting around the mid-20s on average (but again with considerable variation among individuals.)
> Then how come there are so many more 40 year olds in prison than 20 year olds?
Because age demographics of prison are very distant from age demographics of people committing crimes at the time they commit them for a variety of reasons, starting with the fact that imprisonment happens after conviction which happens after the crime. But also because of both discretionary leniency to first time offenders (which younger criminals are more likely to be) and younger criminals even apart from whether they are first time offenders, and because of formal enhancements to punishment for repeat offenders, which combined mean that, for otherwise similar immediate predicate offenses, older criminals are more likely to be imprisoned at all and likely to be imprisoned longer.
Most of those would explain a delay of a couple years, not a couple decades, and the rest is rather the point. If this was disproportionately youthful indiscretion then there wouldn't be many repeat offenders in their late 20s and 30s, since they'd have grown out of it.
In some cases, it was a good idea for top students applying to college or grad school to take a gap year if possible around 2020. When things get lax, mediocre students fare better.
I applied to Canonical a few months back and they made me write an essay describing my high school grades, extracurriculars, and test results. I've never had that experience in any other job app so it's pretty Canonical specific but it does happen.
Canonical's interview process is legendarily invasive and dumb, and they get very touchy when you tell them so. I enjoyed how, when I pointed out that the questions about high school extracurriculars would narrow down many of my demographics and let you guess at a few others (and just the way I write would let you make strong inferences about others), they clammed up and told me I could do it or drop. I dropped.
I've heard similar things from former employees too. One glassdoor reviewer (so take with a grain of salt) said that they were called stupid by leadership for suggesting that the process might be biased, which of course it is. It also feels dehumanizing. It may take months to go from applying for a job to talking to a real human being.
I applied for a job with the FAANG that rhymes with "pajamas on" and their interview process was dumb as well. I told them so and got the interview ended right there. Good riddance, I suppose.
Similarly, I had an interviewer (same place) who just oozed “I despise being here”. Dismissed everything I said and gave me only the last 2 minutes to solve the problem he was supposed go give me. Didn’t bother to check back in with the recruiter after that.
i worked at one startup that decided to emulate the Canonical interview process because one of the managers was the kind of guy that loved cargo-culting this kind of stuff. he decided to start doing these interviews without coordinating with the rest of the hiring team.
it's one thing for a large company to do these kind of interview when the interviewee is knows what they're getting into, but it feels very different when a you're interviewing at a random early-stage startup and out of nowhere this guy starts asking you invasive personal questions. it really starts pushing the boundaries of legal interview questions.
anyway... this guy has since founded his own company writes inane linkedin-friendly blog posts filled with inspirational stories about "leadership" that make me gag.
When I interviewed at Red Hat is was almost entirely technical. There was a computerized technical test (essentially "here's a VM, set up <several things>" and then a script grades it), a verbal technical interview, and then a panel/presentation that was specifically about the process I took my previous company through in adopting k8s.
I thought that part of the application was a bit odd. High school me is so different from current me that I might as well be writing about a different human being entirely.
These kinds of interviews just scream ageism. For younger applicants, High School may be the most significant bit of life experience they have yet had. For older employees, high school is just a distant memory that was at most a seed that has grown into a much more branched experience
What possible insight would they gain knowing what someone did both as a youth and more than a decade ago? I'm a 1000% different person than I was in high school.
That’s insane. I barely graduated high school, my extracurriculars were smoking pot and programming. Then I graduated summa cum laude at a top cs school and a very successful career doing pretty cutting edge stuff as IC and manager up to senior exec levels at FAANG and adjacents. Digging into someone’s high school career is frankly bizarre but convinces me to never consider working there and to stop using their products.
I wish I hadn't found and read these posts. I have been an Ubuntu fan for the last 10 or so years. I will not be able to look at Ubuntu the same again...
I literally couldn't tell you what my SAT scores were. Those and my high school grades were good enough to get into $SCHOOL. I'm sure not going to bother to try to dig them out. Here are some extracurriculars I did--not that it's relevant at this point.
If I compared my SAT score my child's, that would easily compare past the 10 years. If I compare my score to a Millennial or younger.... Lots of ways to compare a score. You're putting yourself in a box. You should really try not doing that.
When I took the GRE in 2017, I got a really bad bloody nose right when the test started and I couldn't pause the test.
The person monitoring wouldn't let me have a bunch of tissues at the computer (Perhaps due to cheating concerns), so each time I needed a new tissue I had to raise my hand and she gave me one.
Sounds like you should have bled all over the computer and let them deal with it. The lack of empathy they showed is abhorrent. Ugh. I’m sorry to hear that.
This happened to me during a written final exam, and yes I bled on the test a bit. Was using one hand to control the bleeding, but sometimes I had to turn pages, plus it was dripping all down my arm. Even the digital scan showed my bloody fingerprints on every page plus some large drop stains.
That's like suggesting you trash a hotel room because of some grievance with corporate policy. What actually happens is some innocent housekeeper gets stuck cleaning up the mess you made. The local test proctors are tasked with following very strict procedures and not given any room for discretion, especially if the testee isn't voluntarily withdrawing. It has nothing to do with an individual lack of empathy.
I had this experience as well outside the US. And while the police stations in my country are famous for its bad experience and treatment, They still much better and you get to deal with more reasonable rules and people (although it is oppressing regime) than what I saw during my GRE test.
Hint: To be fair all my few visits to stations were me reporting things (finishing paperwork) not being arrested.
Its def for cheating. You can find examples of replaced ingredients lists, products labels, barcodes, and people writing on the inside peel of oranges and other fruit on reddit and other places.
What would you write on a cheat sheet for the SAT? Equations for volume of a sphere or something? I don't remember there being anything that required memorizing, and since you could bring a graphing calculator there would have been no need to write things down anywhere else.
The exams for engineering licenses have mostly switched to computer based testing. You have to go to one of those specialized Pearson testing centers and they check to see if you wrote anything on your arms and legs. The old test was famously open book, with people taking carts full of reference material. The new one gives you a searchable pdf.
Been taking certifications exams (Microsoft, then AWS) since early 2000s. Certification centers (eg Prometric) have cameras everywhere, and personnel watches you while you're taking the test, they make you put all your belongings into a locker, including watches. Workstations are locked down to the point that they can only run proprietary testing software.
I honestly prefer it to doing it remotely at home, where you have to show them your entire room (in the beginning and whenever they ask), install their software that wants full control to everything, and it still doesn't work half the time. One time I tried it told me bandwidth is insufficient - well, normally it's sufficient to have 2 or 3 Rokus stream in HD.
Yes, the GRE has been way ahead of the curve for a long time because it's an adaptive test: it actually adjusts the difficulty of the questions it presents you based on your performance up until that point in the test, as it tries to hone in in exactly what your correct score should be.
I don't know if the algorithm has changed since the early aughts, and I don't know if that approach is better or worse than the one-size-fits-all non-adaptive SAT, but the difference was always striking to me. You'd think that if the GRE adaptive approach was better statistically (by whatever measure) they'd move towards it soon for the SAT as well since they're going full digital finally.
My friend took an important law school exam (GRE or BAR, not sure which) in the past 2 years that was self administered, and she described her experience as "harrowing." The kernel level malware they installed to monitor the exam crashed her device 4 times during the exam. This was after she got it running on a secondary device because it wouldn't even start on her M1 MacBook Air. The software recorded her screen and camera the entire time, and would invalidate her results if her husband entered the frame.
I can't imagine the stress this causes even on top of, y'know, having your entire career dependent on the results.
I hope they're using their savings to purchase managed devices and hire good IT support, but I'm not holding out much hope.
That's crazy because I imagine cheating is trivial for those that want to do it. All you would need is a clean install, an HDMI splitter, and a tiny earbud stuffed in your ear. Then someone in an adjacent room can feed you answers.
Even if they start making people put their ears up to the camera, that won't help much. Bone conducting headphones built into the arms of glasses will be virtually undetectable. Do you ban glasses? That won't help. I just tried putting my bone conducting headphones against my upper jawbone inside my mouth and they work fine at 30% volume. How do you prevent that?
What's the endgame? Do we end up with everyone completely hairless, naked, and forced to expose every orifice for the digital proctor?
The education industry is becoming satire. Bring back common sense IMO.
It's practically impossible to defeat cheating on a technical front with their current design. Anti-cheating methods strike me as depending on emotions: I can probably find a way to cheat successfully but, if my future depends on it, my fear will make my body language stick out like a sore thumb. I also wouldn't do it to begin with since it's unlikely to have a better EV than putting the same effort into studying, especially if I actually need the knowledge in the future, or if there are social repercussions. Some won't have this hurdle but the combination of 'needing to cheat' and 'being skilled enough to get away with it' is rare.
The big problem is when cheating is so easy and widespread that everyone knows it. Then it becomes the low friction path. This would explain what I've heard about some Indian school exams.
There are probably a lot of people doing minor cheating and a few people doing major cheating. In most cases, it's not enough to undermine the reputation of the qualification.
Oh, it's the same shape as the fraud problem that's popular here:
The problem is that the people who imagine cheating aren’t asking anyone technical how they would cheat. They mostly target imagined methods of cheating.
Cheating in online games is big business and is accessible to more than the technically inclined through purchase. There's no reason to think the same won't happen for tests that have actual monetary value.
The point is to provide a meaningful metric schools can use to determine how likely a student is to succeed at their University, and how much success they are likely to achieve.
Bad or average cheaters are unlikely to find success. Maybe great cheaters are likely to find success.
It will just catch the dumbest and most people will cheat successfully. Just catching the most blarant and naive is stupid when the alternative of people being supervised in a physical room means there is next to no cheating at all.
Catching all cheaters is impossible for any reasonable amount of cash spent (remember all the possibilities people came up for that chess thing?) - which means they want to reduce cheating to a level that is acceptable.
Things they're not doing that could allow some high-level cheating:
I've taken some tech certifications in the past and had the option of taking it at home (PSI) or at a test center (Pearson). The amount of intrusive demands the at home tests required and the absurd amount of monitoring made sure I never take a at home exam. Test centers I just bring my ID, leave my phone an keys in a locker and go to the assigned PC.
One memorable instance, I have a 37" 4K screen. In full screen mode the questions are in the top left and the multi choice answers are bottom right. The test proctor kept interrupting my exam because they assume I was looking away from the screen. I had to remind them repeatedly I'm using a large screen.
Ouch that seems even worse than the lockdown browser some of my classes used during covid. I ended up digging out an older laptop for it since there was no way I was going to deal with cleaning up that invasive mess off my regular computers.
Luckily by the end most professors just switched to Zoom with cameras on, and instead modified the tests (eg making them harder, but open book, or multiple versions of the exam distributed randomly - similar to how some in-person exams will alternate between two question sets so no one can cheat off their neighbor)
The amount of surveillance/security needed when students take tests at home (like your friend, and many others during COVID) is awful. But it sounds like the SAT won't be administered at home, only in high schools and test centers.
Students will use either school-owned or student-owned devices, which is not great. I would certainly opt for the former, since I wouldn't want to install this crap on my own computer. We'll find out in the next couple years how smoothly this goes. They've been doing the SAT online for a while outside the US, so hopefully the ironed out some of the bugs already.
But they won't have to worry about someone entering the frame, since testing will happen exclusively at proctored centers.
I suspect the Bar exam. My wife went through that twice in two different states, and it's an insane amount of stress, because the consequences of failure rest entirely on the test taker.
Can't her husband just stand in front of her, behind the laptop webcam? There's often ridiculously simple real-world workarounds these complex device security process
There is no one else supposed to be in the same room as the test taker - for obvious reasons that they should receive no help on the test in any way.
Some households -- especially if you have small children, or live in a small house without the luxury of separate rooms, noisy neighborhood, etc. -- may pose a challenge.
But outside those scenarios the candidate should know to dedicate one room to themselves for the duration of the test and preferably keep it locked from inside and inform others in the house not to disturb them for those 2 or 3 hours.
I am surprised to see this simple requirement -- that there should be no other person in the video frame (which will be audited for it, both manually and through automatic processing) -- is considered draconian? How? Are test takers expecting to take tests in rooms where anyone else can casually walk in, move around, etc?
To be sure there are other quirks like no bathroom breaks, no glancing away from scree, no mouthing the words as you read, no covering your face or sometimes no resting your chin on your hand as you think etc that all can become very tedious and stressful sure.
They don't actually disqualify you (most of these places like https://www.proctoru.com or similar) just report to the exam administrator what they saw/noticed.
And that same (or similar) test software doesn’t have any accessibility features for those with limited vision. And if you mention it to the company, they brush you off. They have the contract to gate-keep certain careers, what’re you going to do? Fucking slimeballs.
I don't know if there was a remote LSAT option, but I could see this being the bar exam. It's pretty common to have to use crummy testing lockdown software for actual law school exams: they want you to write a bunch and nobody can handwrite anymore, nor does anyone want to read it, but they also don't have the capacity or inclination to provide computers for this purpose, so it's all BYOD. That's just for regular in-person exams, and the remote context only amps things up with the surveillance aspects.
A friend just recently described to me a similarly stressful experience of taking an AWS certification exam remotely and worrying about accumulating too many "strikes" in the eyes of the remote proctor and not being allowed to finish. So they actually preferred going to a testing center.
Ive taken a few online tests and they have never failed to have some issues that add a ton of stress and waste time.
Even online ides have had significant issues for me in the past which have eaten up 5-15 minutes of interview time. It sucks because I don't know how much cheating any of these measures actually prevent and anyone determined to cheat will be able to bypass them.
Damn it, I just realized AWS online exams for certification are pretty much the same if you read the TOS, I just might book a exam in a physical exam center. Too many unknown ifs and I don't have to expose my machine.
Doesn't stop an industry of cheating by having crib sheets or entire whiteboards with accessory equations, information, or potentially gasp answers on the other side of the screen. Without a 360 camera recording the entire room for the entire duration of the exam, proctor-based exams at a public venue or off-site corporate testing center should be the gold standard. (A few institutions like Stanford continue an absurd tradition of proctor-less exams.)
The online SAT was much easier than the written version, because (1) You don't have to read big essays anymore in the reading part, and (2) both math sections have the Desmos graphing calculator, with absolutely no features taken out. Desmos is far more convenient to use than a TI-84 for many things, so it made all the problems, especially the hard ones, much easier.
Also, in my opinion, their Bluebook testing software is actually pretty good, especially compared to other digital testing software I had to use over the years, like ClearSight.
When I first heard they were shortening the exam by 30%, I thought that was unfortunate, since it would either make the test less reliable or impose a lower ceiling on the range it can assess.
However, I later learned that it will be dynamic — depending on how a student does in the first section on a given topic, he will be given either easier or harder versions of a subsequent section. That means that they'll be able to better assess students of all abilities with fewer total questions. Sounds like a win to me.
Of course, dynamic testing like this is not possible with hard copies. Seems like a decent rationale for moving to computerized testing, though I would still be fine if they did it hard copy, for 3 hours.
Shorter is good, but dynamic tests (called "computer adaptive tests" or CAT in the industry) make it difficult to follow longstanding best-practices for test taking. When I took the SAT, I could quickly scan through the questions in the current section and mark those that looked difficult, and save them for the end. With a CAT test, it's typically one-question-at-a-time, and if you're a strong test taker, they get successively more difficult. They are more efficient tests, but (IMHO) less pleasant to take.
That is true for adaptive tests that operate on a per-question basis. What I have read of the new SAT is that it operates at a session level. So if you do well on the entire first session, you get a harder version of the next session on that topic (reading or math).
Students should still be able to use the technique you mention to identify hard questions and save them for last.
> What I have read of the new SAT is that it operates at a session level. So if you do well on the entire first session, you get a harder version of the next session on that topic (reading or math).
That is positive. I've seen issues with adaptive tests not being resilient to accidents and brain farts: if a kid fat-fingers the response to a question early in the test, they may never get the computer to give them questions of a suitable difficulty later in the test. Thus they never even have a chance to get a score that accurately reflects their ability.
If the SAT is session-based, each session needs to be large enough that a single question can't tank the whole thing, but at the same time there also needs to be enough sessions to allow for properly dialing-in the knowledge level. In reality, the ideal situation is that the test should allow for an unlimited number of sessions, but each session is time-limited. Something along the lines of stopping once the test-taker gets below a certain score on 2 consecutive sessions.
> If the SAT is session-based, each session needs to be large enough that a single question can't tank the whole thing
I think each session is either half or a third of the test, for that subject. There could actually be more than two versions of the later session, to ensure that a student who makes a mistake or two isn't prevented from getting a relatively high score.
> make it difficult to follow longstanding best-practices for test taking.
Isn't that good? The effectiveness of test-taking strategies is a (minor) flaw in the SAT. People taking an ideal test would not benefit at all from learning strategies.
(The flaw is minor because all our methods for assessing people are gameable to some extent, and standardized general tests like the SAT are among the least gameable.)
Some of the math competitions I participated would penalize for skipping around. You were awarded +5 for correct answers and -4 for incorrect or skipped questions. It was not uncommon for negative scores. In one test, any stray marks were also marked as incorrect. No erasing, no changing a 7 to a 9, or anything was allowed. The questions leaned toward an increasing difficulty, but there could something very difficult followed by a string of much easier questions. So additional math had to be done to see if it would be better to stop or skip.
It's still equivalent to "normal" tests, because the cost of skipping and the cost of answering incorrectly are the same. A test in that format with N questions and a test in the normal "1 point for a correct answer, 0 for anything else" with N questions are related as follows, assuming C correct answers:
Score in the +5/-4 system: 5 C - 4 (N-C) = 9 C - 4 N
Score in the +1/0 system: 1 C + 0
It's basically a "9 points for a correct answer, 0 for anything else" test that simply starts with a score of -4 N.
It's very similar to the difference between Celsius and Fahrenheit.
how many normal tests would ever result in a negative score? it's quite devastating to any shred of morale one might have. test had 80 questions so a max score of 400. i personally never witnessed someone receiving anything over 300.
you're really trying to make something into something it is not.
Don't know how wide spread they are in other states, but it was part of the UIL Academics teams as statewide competition between school districts in Texas.
It's cute but doesn't make any difference. Add 4 to the score for every question and you get 9 or 0. The factor of 9 makes no difference so call it 1 or 0. Now your score is just the number of questions you got right, but the ranking is exactly the same as the original scoring system.
Again, another person responding with the same trope of a response totally missing the fact that assigning 1 or 0 means an always positive score. You will never know the shame of receiving a negative score. It's part of the competition whether it was designed that way or not, it is exactly what teenage boys have turned it into. The concept of a negative score is a pretty good motivator.
It depends if skipping means questions not answered or if it means questions skipped to answer later questions. But either way it's more like playing gameshow host than making a meaningful exam.
Howdy fellow Texan! I still use some of the "tricks" learned to do that test. However, I struggled for a long time in my higher math classes for not having any work to show as I was just doing it in my head. It was finally solved when told that the AP exams gave partial credit based on the work shown.
I also did the Calculator tests. It's why I learned to 10-key.
You're right - I still use number-sense tricks for fast multiplication :)
For Calculator, I wrote https://git.io/ti84rpn so that I could use the fast parenthesis-less Reverse Polish Notation without having to adapt to a whole new calculator keyboard.
They get extra time allocation to take tests - 50% more in most cases. The numbers I hear tossed around is that a substantial fraction (20% to 40%) of school kids in some of the elite public and private schools have IEPs. Clearly there are kids who need the extra time. But that many?
ADHD is the reason that I hear the most. The extra test taking time is not reported to colleges.
Combine 50% extra time with adderall. 1500+ SAT scores are not uncommon.
> Combine 50% extra time with adderall. 1500+ SAT scores are not uncommon.
That's not a great take. If you want to complain about overprescribing, go ahead. But Adderall is meant for people who need it to get closer to baseline. People abusing the prescription is not a good reason to criticise IEP adjustments.
Are stimulant prescriptions for ADHD carefully calibrated in dosage and time to get patients to a baseline? In my experience, doctors ask you questions to determine if you have attention deficits, then give you a prescription, which will be changed in various directions (different medication, increased dosage, etc) if you report problems; they very rarely in practice have you take an in-depth "attention measuring" test and then prescribe on that basis.
What is the baseline? Would anyone consider it cheating if they took extra or changed their dosing schedule to maximize their test score? I think most people would consider that rational behavior, not abuse. But it does call into question whether that's "fair" to people without the drug.
Fair comment and distinction between IEPs, medical support and over prescribing. I'm all for helping people get to a healthy baseline in their life with medical support and medication as necessary. IEPs are necessary and benefical to many, many kids.
> They get extra time allocation to take tests - 50% more in most cases. The numbers I hear tossed around is that a substantial fraction (20% to 40%) of school kids in some of the elite public and private schools have IEPs. Clearly there are kids who need the extra time. But that many?
An IEP is a catch-all term for anything that requires special adjustment for a student. What the adjustment is varies depending on the situation--there's a reason it's called individualized.
Having gone to one of those "elite public schools", it wouldn't surprise me if that large a fraction of kids had an IEP (hell, I did). However, in my experience, not one kid had an IEP that allotted them extra time to take tests. The only specific test adjustment I'm aware of was one kid who was so blind he needed the exams to be large print to have a hope of reading them.
My daughter has an IEP due to her dyslexia. She gets extra time on in class writing assignments and tests which have a writing component. So no extra time in general for her math or science tests. Although for science there are sometimes writing components for which she gets additional time. English, history, government, etc which are writing heavy she gets extra time. For several of her classes she is dual enrolled with a local college and we had to apply to the college to get accommodations for those classes. They were actually much more generous with the accommodations granted than her high school. The only one she takes advantage of is the increased time.
The College Board also granted her accommodations so she gets 50% longer on the SAT and AP tests. April will be her 1st attempt at the SAT. She has taken a few AP tests.
ADHD is not considered a specific learning disability so should not qualify someone for an IEP or accommodations from the College Board.
> However, in my experience, not one kid had an IEP that allotted them extra time to take tests.
There's a difference between the IEP that the school uses and the accommodation request for standardized testing. It would be pretty obvious to you if a student in your class were given extra time for in-class tests, but much harder for you to know if that student qualified for extra time on the SAT. If you knew when they were taking it then perhaps you could figure it out, but people who bend the rules to get extra time tend to not go around advertising when they're going to take the SAT, or at which test center, since that might out them as receiving extra time.
All this to say: I'm not so sure a classmate would know if a fellow student received extra time on the SAT.
You're aware that we generally took the SAT in the same kind of classrooms that we took our class tests, or the state standardized tests, or the APs? (Okay, the APs were largely in the gym instead of classrooms because there's several hundred people taking the AP exam at the same time, but same thing really).
If kids were getting extra time on standardized tests, we'd know.
> You're aware that we generally took the SAT in the same kind of classrooms that we took our class tests, or the state standardized tests, or the APs?
Sounds like a very different experience than what I had. SATs were on Saturday, at a different HS. There were various HS's to choose from, depending on where you lived. I went to a magnet HS (sounds like you did too), so kids were from all over, and they took tests at whatever school was near their house. Within each test center, we were then broken into rooms based on last name, IIRC. I typically knew only 1 or 2 kids in the room I was in (the rest were from other HS's), and would have had no idea whether any of my classmates had extra time.
Even in your situation, you wouldn't have known that a kid who took the SAT with you didn't also take it again, at a different test center, with extra time. Lots of kids take it more than once, and you can choose where to take it. Kids tried to keep this stuff on the down-low, either due to shame (because they actually had a disability) or to hide their undeserved accommodation (because they didn't have a disability).
If you can't opt out of the dumb-dumb version regardless of your answers to previous questions then the test is rigged. Not only would it not be directly comparable to previous test years, it would not be comparable to other versions of the test. Defeating the entire point (ranking students)
Comparability across test years is taken seriously by the folks that make these tests (see e.g. [1,2]). There are valid criticisms to be made, but flippantly calling the test 'rigged' is not constructive.
As a former educator, I look forward to the day that we can simply track student ability in a trustable manner based on long term data collection in the classroom rather than requiring these additional, often expensive, exams. We would need to mediate our classroom interactions more than we do now - but the tools do so usefully are there if packaged correctly.
This is a nice ideal to strive for, but until everyone goes through the same education system there will always be a need for some form of...standard testing across the board.
I think the closest we’d be able to get to this is maybe smaller, more granular standardized test that do a better job of testing multiple specific dimensions over an academic career vs a big bang at the end.
Somehow that actually sounds kind of creepy, but if that’s what you meant that’s the only way I see this done more effectively.
In your experience, what other ways or ideas have you heard?
"I think the closest we’d be able to get to this is maybe smaller, more granular standardized test that do a better job of testing multiple specific dimensions over an academic career vs a big bang at the end."
This is the heart of the concept, yes. We should be attempting to make in-classroom assessments as trustable as standardized tests which would allow them to continuously implemented during instruction. The quantity of data will provide a significantly more robust representation of a student's abilities today, and over time, compared to the infrequent snapshots of the ACT/SAT/etc.
> I think the closest we’d be able to get to this is maybe smaller, more granular standardized test that do a better job of testing multiple specific dimensions
Don’t we pretty much have this via AP tests and SAT II Subject tests?
> over an academic career vs a big bang at the end.
I hate to say this, but describing the SAT as a “big bang” strikes me as funny. It’s a test of fundamental literacy and fundamental numeracy for well-educated people. It’s not that hard.
The catch is that the preparation that yields high scores is developed over years.
For math, it’s taking the subjects and understanding the fundamentals of these subjects. Sadly, overall math education in the US is very weak and very skewed towards males. Note that most Asians (East and South) that have to take the SAT or GRE tend to and expect to get very high scores — it’s just that easy.
For the language sections, it boils down to having read and understood enough high quality “high brow” content. This high brow content covers most if not all of the vocab, reading comprehension, grammar, etc. that are needed for the test. Sadly, most young people are not exposed regularly to this type of content, and they rarely read it critically if they do have access.
I’m not sure what exactly we are trying to finesse with your suggested changes.
> I’m not sure what exactly we are trying to finesse with your suggested changes.
Maybe you should ask then instead of going on some dumb rant?
Maybe that was an attempt at a rhetorical question, but given how much you missed the mark, I took it more literally.
Big bang btw just refers to doing it all at the end all at once vs in smaller steps over time, it’s not to be taken literally. It doesn’t imply grandeur or anything like that.
Also, you missed the entire point of OPs message, as an educator, they were suggesting that perhaps the SAT doesn’t do all you suggested.
I don’t know what we’re trying to finesse either, can you explain what you mean by finesse?
> Also, you missed the entire point of OPs message, as an educator, they were suggesting that perhaps the SAT doesn’t do all you suggested.
Except that it does, and there is data to back it. As an example, some elite schools have stated that they really like using the SAT since it predicts success at their respective schools.
“On average, SAT scores added 15% more predictive power above high school grades alone when attempting to understand how students will perform in college. Within narrow high school GPAs, SAT scores further helped predict a student's future success in college.”
> I don’t know what we’re trying to finesse either, can you explain what you mean by finesse?
What do “granular tests” tell us that the SAT, SAT IIs, and AP tests do not already tell us?
The short answer is “not very much”.
The results of these tests are accurate enough to make meaningful predictions and decisions.
Note that there is some interesting work in learning portfolios (probably an optimal variation of “granular test”) that can be useful for structuring individual learning plans, but that sort of misses the point of broad-based standardized tests, which is normalizing widely varied experiences.
Note that standardized tests fail two types of people at the margins, but these are largely addressable, and the benefits outweigh the costs imho:
1. Folks who have test anxiety. Familiarization with the test reduces this substantially for most people affected by this.
2. High-performing folks (think top 5% or better) who don’t have high attention to detail on computer adaptive tests. Also addressed by test familiarity. I’ve seen folks who should be maxing a test get lower than the max because they were careless in some early questions.
Isn't the issue that you need a shared test to measure students from across different classrooms, let alone different cities, states, countries, etc.? I agree that it would be great if student ability could be evaluated in some other way that met the same requirements as a standardized test, but I just don't see how to do that.
Apart from, I guess, a one-on-one interview for every student with the colleges they apply to. That wouldn't scale, and people would complain about bias with that, and not without reason.
There are distinct psychometric techniques used to validate the trustability of assessments. Those same techniques could be used to validate the classroom assessments, but - obviously - no one has the financial or time budget to implement such a thing. So, the idea is that you would begin by providing an assessment creation tool that can be seamlessly implemented in the classroom that also has the ability, in a more aggregated fashion, to develop assessments that pass psychometric evaluation. Would it replace the ACT/SAT immediately? Of course not. The effort would be a classic innovator's dilemma scenario in which the continuous collection of curriculum, instruction and assessment content creates a unique (and, currently, silo'ed/analog) data set against which to continuously improve the quality of the assessment tool. From there - you branch out into instructional support more broadly, but that's a much broader and more difficult context.
Classrooms are not built equal. The grading is highly subjective based on the teacher and there is a strong incentive to make it seem like kids are doing better than they really are. Nothing can replace a standardized test that puts everyone on an objectively measurable plane.
If there were a common tool for integrating assessment into instruction, it would be possible to continue to rank/sort students regardless of their educational context. Understanding the rigor of the course itself would necessarily be quantifiable as well. Think of it more like "we do a standardized test every lesson/week/month" instead of once a year.
How would you compare raw intelligence or cross schools? That kind of calibration and opportunity for smart students who had other reasons to not focus or care about high school and other remedial grades is really important.
This is what we're working on building at AutoMark (https://automark.io). We're currently building grading tools for teachers, but eventually want to build out a tool to accurately monitor a student's performance over time. There's a lot of potential for AI to disrupt the education system from high school to college to a career.
Currently, a GPA and a few test scores (SAT, ACT, etc) are the most in depth view into a student's high school performance that colleges have, but in the future, we hope to see continuous tests throughout high school that don't just show a single letter/number, but a holistic view of a student's progress, effort, etc. (Obviously the data should be controlled by the student) These could also help a student decide where to focus their study efforts.
SAT And ACT and other standardized tests are the single best thing for giving poor students a leg up against rich students. We should be putting more emphasis on those tests. A poor but smart student can study for a test and do well on it, while they can not have continuous tutoring throughout school, paid grad students to do or proofread their homework, or extracurriculars that cost more than their parents pretax income.
Your business is selling an advantage to the rich students coupled with increasing student stress throughout school. Your "holistic" view can't account for "students grades are slipping because they were sleeping in their friends basement after their parent killed another parent" or "students grades are slipping because they are working 40 hours a week, illegally, at 14", but it will holistically show "student got a B+ so their dad who makes $500k a year hired a tutor for 10 hours a week".
This is what I don't understand about the movement that wants the SAT/ACT removed for college admissions, or claim that it's biased.
Sure, it's biased. Everything is. But it's biased in an open, auditable way that students of any background can prepare for. The rich will always have an advantage of course, but this is one of those areas where grit and determination can actually close the gap considerably.
Remove that, and a benchmark that anyone can understand and prepare for is just going to be replaced by something more opaque. You think a standardized test is biased? Do you think the application committee at most universities is going to be unbiased when they have nothing to go off but a student's essay and zip code?
It's incongruous to me that the same people who I think have a deep understanding of unconscious bias also seem to be the same ones wanting the standardized tests gone. I don't think it's going to help the people they think it will.
I absolutely agree. I grew up in a single parent household in poverty. But I was able to beat people on the SAT who grew up in extremely wealthy families. I have met plenty of people that graduated from highschool with a great gpa and went to good schools solely because of their family, and I have known dozens of people that I know are pretty sharp who didn't get to school because of family circumstances.
Anything that seeks to further widen the rich / poor gap like this is in my mind, completely unethical.
It’s still paying in time for a poor person. The wealthy hire an SAT tutor for a hundred+ an hour to bump their kid’s scores up by a third for maybe 20 hours of work. The poor student will spend much more time. It’s like many things where the poor pay in time rather than money.
Your situation is not the same for everyone, though. It should instead be more carefully worded, "[We didn't] need any more resources for SAT." Imagine the more extreme case of a student who took it first try and got a perfect score. You wouldn't necessarily expect everyone else to be able to do the same.
Being on this site, you're probably someone who values education. And you probably raised your kids to be capable. Or maybe they just lucked out and got the correct weights to succeed in that particular scenario, but you can't assume that the situation for everyone is the same as you!
Many people simply can't learn by just doing practice tests and watching videos. If it were so easy, then we wouldn't need teachers or schools in general. Mentorship and guidance are important, and it's the reason why so many people want to get into universities in the first place.
Essentially, yes you can do without, but having it helps. Overall, that would create a disparity that favors more wealthy people.
Poor people's parents don't have the money to bring them to vacations, enlist a violin teacher or drive them to baseball on the weekends, so clearly they have more time to allocate to studying and thus are advantaged. Conclusion: do away with meritocracy and just exclude poor people altogether. Did I get the "logic" right? Of course put this way it can't be valid because that wouldn't support the American Neomarxist delusion, sorry I meant worldview.
I really don't get how the context follows through on your point here. Music, sports, and travel all have been shown to increase your mental health and thinking capabilities.
All I get is that if you start from an extremely unfounded point you'll get to the wrong solution. But that doesn't say anything about the points people are making here
This is a very hard problem to solve - nobody has successfully solved it before. It's not just a practical problem, because there are also a lot of political motivators. It's difficult to get districts to adopt new technologies, and teachers have a way they already like to teach.
There are already some great quality resources online to learn subjects like math, history, and science, but we believe the missing piece is grading - you're never actually evaluated whether you learned something online.
We think by giving students unlimited practice and feedback, along with quality learning materials, we can work on chipping away at this issue. AI will allow scaling access to quality tutoring, practice, and exams, all of which significantly improve eduction (currently, this costs $40-$X00/hr, but will cost much less with AI assistance). It's definitely not a quick fix to solve the issue of instructional quality.
That seems like a really bold claim that I was excited to read about in the source. But this seems like it is only saying that for one school district and one particular choice program, they weren't able to see a difference on a test.
With what, a standardized curriculum? That is a hell of a can of worms as you probably know.
Apply an AI agent to the sum total of student output? Can't wait for the entrenched racial bias to leak into the AI and a host of other issues.
If you could get an AI implemented that didn't involve reporting to the greater regulatory bodies, I can see that working, but that's the issue with things like that: management wants to see them and then the stats rapidly get watered down to meaninglessness.
It would be crazy to see if an AI can start diagnosing home issues, abuse, etc from student output and performance. I can see a lot of benefits, as long as the AI output is private.
No standardized curriculum. Utilize the instructor’s curriculum and a local, live transcription of the instructor to continuously create assessment content that can be rapidly curated and proctored within the flow of the lesson.
The vision is not as grand as your last paragraph.
I'd go a step further and decouple education from social status stratification as much as possible. Maybe not realistic with evolution and primate dominance hierarchies, but we can hopefully (although I'm not positive...) overcome that.
In an ideal world, we wouldn’t have a tiered education system and that everyone will have access to master any subject/curriculum regardless of age. Therefore, we wouldn’t need an arbitrary and flawed aptitude tests.
> In an ideal world, we wouldn’t have a tiered education system and that everyone will have access to master any subject/curriculum regardless of age.
I’m not sure where you are, but I’m in the US (specifically California). This dream of yours pretty much exists.
> Therefore, we wouldn’t need an arbitrary and flawed aptitude tests.
The aptitude tests are more for matching folks who will be more closely matched in terms of quantity and speed of learning. There is definitely some virtue signaling as well via elite admissions, but that doesn’t exclude anyone from learning pretty much most subjects.
I agree. If a child understands and excels at some topic, they should have the choice to move on to the next? (I believe this is what you mean, correct me if I'm wrong)
How do you do that data collection, though? And more importantly, how do you effectively compare across different schools and districts? Some high school give out As in math to students that can't do basic algebra.
Analyzing curriculum contents and local instruction transcription.
Ranking and benchmarking between educational environments would require an additional psychometric evaluation layer. This would not a Day One function, so the tool would focus on maximizing utility as an assessment assistant tool for instructors. There is a bridge to be built between formative and summative assessment. If this thing doesn't make them better instructors, it would be dead in the water anyway.
At that point you're just describing another form of standardized testing. The whole point of a test like the SAT is to administer the same test so that results are comparable.
Everyday, all across the country - we pay teachers to teach and assess their students. Those assessment results aren’t trustable for benchmarking and ranking students, so we have standardized testing.
Obviously, yes - one intention of this concept is that it serves the same function as standardized testing, but it does so while eliding the externalities of cost and single, high stakes exams by allowing classroom assessment - made and proctored by the instructors during their standard lessons - to become trustable.
Same. This god-damned nightmare of needing to maintain perfection is great for some kinds of people but not me. I blew off homework often, then killed it on the tests, and killed it in grad school, and quite successful in real life. But the ideal American education is a never ending series of constant evaluation - intended to measure adherence to curriculum over everything else. Absolute nightmare. I'm really considering homeschooling my kids for this.
Yup. Take a test and show you should be a good fit for a school, awesome, have to be tracked throughout a relatively boring (classroom-wise, except for programming class) couple years of high school, no thanks.
Eh I fucked around in high school a bunch (prob showed up to 40% of it) but still aced all my tests. I like the standard test gateway paradigm. Without it I wouldn’t have achieved the success I have today.
> "Today's students, they do a lot of their living digitally, they do a lot of their learning digitally and they do a lot of their test taking digitally," says Priscilla Rodriguez, who oversees the SAT for the College Board, the organization behind the test.
How do kids work out math problems these days? Do they draw equations with their finger on a tablet/phone? Tablet pen on tablet? Are they learning math typesetting at a young age?
I used to use huge amounts of scratch paper, in college I had a convertible laptop/tablet that I used for taking notes in class, sometimes handwriting everything, sometimes LaTeX & import the generated pdf into onenote where my handdrawn figures were.
But I never used that setup for scratch paper, scratch paper was something sacred and always paper + pencil.
But, now paper is nonexistent and I use a wacom tablet with OneNote when I need scratch paper. So there's been a huge revolution in my life and I'm curious what today's kids do.
As a current high school student who will be taking the digital SAT this March, I can assure you that paper is alive and well both in the classroom and in testing. I don’t love the new digital testing system, and I see it as a failure of their system that students can no longer self pace through the test.
>Is this change motivated by issues regarding self pacing?
I think that to the Collegeboard, the change to digital was to make it easier to distribute and easier to take (digital, shorter time). However, I think that this has some unintended consequences in how it will limit student freedom to self-pace, and the possibility of unjust scores if the student does not make the qualifying 'second stage' difficulty on the adaptive part.
>Good luck with your upcoming test!
Thanks for the kind words! When I heard that the test would be digital, I rushed to sign up for a paper section, and was lucky enough to get a slot (and a good score). I'm only now taking the digital because my school is offering it on campus, and that's just too convenient to pass up.
It makes sense to have tried to get in for a paper test in case the digital rollout is a mess. It will be interesting to learn how the two tests differ — I hope students like you write blog posts or articles in school newspapers about the experience.
My kid is doing prealgebra now online (AoPS). When she's on an iPad Pro, she uses the Apple Pencil in the Notes app, side-by-side with the AoPS window. When she's on a non-Pro iPad she uses pencil and paper.
I assume kids will use scratch paper with the SAT, along with calculators.
> Are they learning math typesetting at a young age?
My kid loves doing LateX on AoPS, for some reason. I guess it will be useful if she's still doing online math when she gets to higher levels (it's not really necessary for prealgebra).
Due to wrist injuries I used to type and turn in printed copies of my homework for a variety of math heavy engineering classes. It was quite nice once I got used to it especially because it reduced the friction of fixing a mistake and reduced the cost of including connecting or explanatory prose statements. It really improved the quality of my written work (especially the diagrams and the greek letters).
Anyway, I just got excited because I hadn't heard of anyone else doing that, I hope it remains a useful technique for your daughter for her classes.
It was pretty common at my uni to TeX math & physics problem sets, I had phases of doing both depending on my mood. I usually felt that the one I wasn't currently accustomed to made me more careful, so as TeX-ing became too easy I'd handwrite, as my handwriting deteriorated I'd go back to TeX.
Kids write on paper just like everyone used to, or they use a tablet or something. There’s literally no difference except that sometimes they click radio buttons instead of filling out a scantron.
I think these types of tests should only be administered on computers
provided to the students.
Not bring your own device.
The enormously draconian spywear that is installed makes me jumpy.
Not just due to the nature of the software, but for unknown errors and
crashes happening due to some incompatibility.
There has to be a million different ways a computer could be set up,
Oh which only a small subset has been properly tested.
It also gives rich students a potential benefit, since they may have
computers with bigger screens, faster cpu, which might be advantageous.
(Of course they already have thousands of other benefits)
I think similarly specced computers provided to the students would make this fairer and more reliable.
How will this devalue the test? The only way in which the test is easier is that it's shorter, so students who lose steam toward the end will perform somewhat better. But this shorter test is dynamic, so it will still be able to assess students fairly, at all levels. In truth, it could even open the door to allowing for a higher ceiling, by offering even harder questions to students who score super well on earlier portions. That could make the test (and a coveted 1600+ score) even more trusted as a marker of academic ability/readiness.
Would not 'losing steam' on a long test be indicative of motivation and also be part of the test? Remember, the test is a lot more than the questions; the test is also about pacing and focus.
There is something to be said for endurance, and I admit that the new test will not test endurance as much. But I think most people don't think of the primary role of the SAT as an endurance test.
When I was in HS, we had IB tests that were much longer, and were seen as grueling tests of endurance. And after law school, I took the CA bar exam, which was at the time the only 3-day bar exam. It was (in)famous for endurance.
But do people think the SAT is measuring endurance? I don't, but perhaps others do.
One thing that won't change is any effect on pacing. The SAT previously had sections, and it will continue to have sections. You still have to pace yourself through each section. But there was never a way to pace yourself across sections, through the entire test. You had to wait until one section was done before starting the next.
"Students will still take the exam at a test center or at a high school."
You're not going to be able to take this with an iPad on your couch (or with a friend/parent nearby). Likely just as draconian as today, except no pencils required.
I don't know why the above got downvoted - it's absolutely true. There is a reason I have to go into a testing center or sit there with my camera on with a proctor watching when taking cloud exams.
I don't see how hard it can be to keep organizing SATs once a year - plus it really does generate a shared identity, which is something we are all sorely lacking. It was a big event to go to a different school to take the SAT.
This are big statements to be making with no rationale. These will still be proctored exams in person, so there is no reason to believe that cheating will increase. Additionally, there is low indicators from the start that standardized tests correlate with academic success, hence the number of higher ed institutions no longer requiring them.
People who want to cheat will always find a way to cheat. All you need to do now is go to the bathroom and lookup answers on your phone.
Standardized tests actually correlate better with academic success than a number of other measures. Some Ivies have recently been reinstituting standardized testing requirements.
That’s just straight false, or at least I have never seen any evidence showing a strong correlation. What has been shown is the negative advantage minorities, impoverished, ESL, disabled people have when taking the test and the advantages that wealthy people who can pay for tutors have.
>What has been shown is the negative advantage minorities, impoverished, ESL, disabled people have when taking the test and the advantages that wealthy people who can pay for tutors have.
How does this contradict the GP? This doesn't necessarily seem inconsistent with standardized tests correlating better with academic success on its face. Because I would expect that such people tend to have not just worse standardized test scores but also worse academic success (because of the advantages that wealthy people have that these people lack).
There is a difference between aptitude (intelligence) and academic success (grades). There is also the underlying issue of equity.
If someone with a high aptitude but low academic performance is put into a situation where they have more access to resources, they will perform better than someone with low aptitude that has the means to make up for it in their academic success.
For instance, the popular college prep hack is take the SAT three times. Study for a different section each time, and colleges will take your highest score of each section. That’s not possible without the moderate financial means to take the test multiple times.
Doing these online will create Lots of opportunities to cheat. You guys underestimate how much money there is in cheating these tests. It’s easily worth 100k per head maybe more.
It’s worth a lot to a small subset of people, but that’s an issue today. The biggest way the group you’re referring to cheats will not be made worse by having the system online (bribe proctors, have someone else take the test, SAT training, etc.)
Your average high school student does not have thousands, let alone tens of thousands, of dollars at their disposal to cheat on the SATs.
What angle? I'm talking about the definitions of the words.
Pandemic: occurring over a wide geographic area (such as multiple countries or continents) and typically affecting a significant proportion of the population; characterized by very widespread growth or extent
Endemic: characteristic of or prevalent in a particular field, area, or environment; a disease or outbreak of disease that is typically present in a particular region or population : an endemic disease
COVID doesn't even really qualify as endemic based on the dictionary definition, but it's certainly not a pandemic.
None of those numbers indicate growth. None of those numbers indicate a significant proportion of the population. I'll give you geography, but that's also characteristic of an endemic illness - distributed across a large geographic region among a small percentage of the population with relatively little impact.
The severity has dropped more than the prevalence.
Hospitalizations suggest that in the US the peak in Jan 2024 was around 80% as big as Jan 2023. If that's accurate, then Jan 2024 had more cases per day than most of 2020 and a good chunk of 2021. That's a lot of extent. There had been 120 thousand cases in the entire world when pandemic was originally declared, and the US is currently doing a multiple of that each week.
How does this cut costs? I would think they'll be spending a ton on online security and such. Imagine the fallout if this is hacked or goes down on test day. They will save some money on proctoring, since the exam is much shorter in duration.
It would not surprise me if the test was delayed last minute due to a security issue. I've seen firsthand the level of security in the testing application, and it is worryingly lacking. Did you know that they literally link 'desmos.com/calculator' as the calculator in the math section? That's opening the door to all sorts of security issues. And they're certainly saving money, the digital is the same cost as the paper[1].
They've already rolled out in international testing, but yeah the comments here make me think there are going to be some massive cheating if they don't tighten things up.
"you are not allowed to bring any electronics into the testing center (except a computer that you own, possibly running a VM)!" isn't likely to deter cheaters.
Anybody that's taken a proctored online exam in high schools knows that this is absolutely not about efficiency. With universities rolling back test-optional pandemic policies, the College Board is likely to raise the price of exams and drive down the cost of administering them by gutting support. I'd predict more frequent exam dates once people have absorbed the price increase. Driving that velocity requires online administration.
In the noughties, at my Bay Area high school, students had a microphone wired up to their cell phone to someone they paid outside to Google answers, who was watching a feed of stills relayed via MMS every few seconds from a camera hidden in their hoodie. (Or, alternatively, the room was pre-bugged.)
I wonder if colleges will begin differentiating between at-home and trusted-facility scores.
So my kid (7th grade) was taking a standardized test yesterday at her school. School rules have them keep their phones in the locker - not allowed in class. Locker is far away from testing room so that blue tooth is ineffective and they cut the wifi during test time so watches cannot be used to cheat.
I have consulted in this sector. An exec I met who has three decades of experience explained to me that software fit for purpose has to operate within these parameters:
1. The highest value exams (like lawyer licensing) can only be in-person proctored, not remote which can’t reasonably stop even moderately clever cheaters, and where any suspicion of cheating undermines the legitimacy of the entire exam.
2. No exam-giver wants to be responsible for the complaints they will receive if they provide the computers, since some number of people who do poorly will seek to blame the hardware, so exam-takers must be responsible for making sure the software works on their own machine.
3. Essay exams (like the bar exam) have to be the same day for all takers, because it is trivial to share meaningful information about the questions sufficient to give late takers a huge advantage. It is possible to use different questions on different days, but this creates major challenges for calibrating the answers fairly.
4. Large exam halls where thousands of people can take the exam simultaneously don’t have good enough wifi to hold all those connections, and even if they did, any blip would be a nightmare, so the software has to be able to work offline.
Ah, I remember this test fondly. Mostly because I "cheated" my way to a much better score than I "earned" on my presat.
I remember walking into the classroom to take the test and I recognized a student who ranked in the top of the class. So I sat down next to them. We didnt know each other. We were both taking the test and through "random" peripheral vision I noticed that none of our answers were lining up. So I continues on with the test. I was on the math section with which I am quite confident so I didn't even consider questioning myself and just assumed we had different tests. Then I started my English section for the second half of the exam. Now this is the only language I can speak but I am still terrible at all those words whose definitions mean to lookup another word (is there a word for that?). But again none of my answers lined up with theirs. I started to doubt myself but it's at this point I noticed all my first math questions are lining up perfectly with their second section... No way I think. They don't make multiple versions of the test? No way I think, it can't be this obvious. Then I start checking every English answer I am undoubtedly sure about with their first half and they all line up perfectly. Needless to say I took a "gamble" and if I didn't know all the stupid words in the analogy I just put what they put.
This long, verbose story later, I cheated my sat got 300 points higher than my pre-sat and it's all because they made 1 version of this terrible test.
If I would have had to take it digitally I would of probably cheated it in some other way. It's absolutely no way to determine intelligence. Simply test "taking" skills.
If you are smart enough to cheat and not get caught, then I think the test results accurately determined your intelligence. At least the type of intelligence one needs to make it in a world ran by liars, crooks, and thieves.
Loathe: [0] "to dislike greatly and often with disgust or intolerance"
Abhor: [1] "to regard with extreme repugnance : to feel hatred or loathing for : loathe"
Basically any word that means exactly the same as another word.
The SAT liked to do questions like Abhor is to Loathe like X is to Y.
> [A high school student] says she's still nervous and anxious for the test, because she feels a lot of pressure to do well. "It's a very important test," she says. "It dictates what's going to happen for your college life.
Man oh man, do I wish I could convince my high school self: the particular college you go to matters much less than you think. The SAT matters much less than you think. Grades matter less than you think. Pursue friendship and genuine interests, not standardized tests!
I think I would have told my high school self the exact opposite: Go for a better GPA. Go for a better SAT. Get into MIT or some other "elite" instead of the state school that I had to fall back to. Between companies that recruit at top schools, starting salary, salary progression, and compounding interest on savings, I'd probably be retired by now if I spent a just a tiny little extra effort in high school. You can get friends and hobbies anywhere and any time, but you can't re-do what undergrad you went to and what your first job was, which both have an outsized influence on your entire life's financial trajectory.
If it's just about salary, at least for software engineers, you don't need to get into MIT or anything on par with it. Not nowadays. At universities far below MIT, plenty earn six-figure starting salaries, and a few people earn upwards of 300k starting (yes, only a few, but only a few from places like MIT as well, because this is a difficult thing to do whether you're at MIT or not. Perhaps the proportion of people who get such jobs is higher at MIT, but I would wager that this is because MIT tends to attract highly talented students moreso than other universities, not because of the MIT name).
Not to say MIT isn't a good goal to shoot for, of course. But doing so primarily out of financial interest isn't just heavily discouraged (I'm sure any MIT recruiter would tell you this) but also probably not worth it (particularly if you have to pay!). At least, not for software engineers, though I wouldn't be surprised if the same applied to other fields.
Yeah like, Satya Nadella got his master's at UW Milwaukee. I went to a big state university and got a great education. If you're smart and you're not desperate to get into one of the "elite" schools, it's gonna be fine.
Wow. Education in the US is turning into an even bigger joke, making cheating effortless, and minimizing cognitive exercise of fluid and crystalized intellect.
Nothing can or should replace tests or writing on paper and a (cleared) HP 4x because Erable is the superior CAS.
When I took my SAT someone crashed into a utility pole and the power went out. We ended up having to retake, which was super annoying since that test was so damn long. I wonder what they'll do this year if the same thing were to happen?
I did a WGU CS degree, which had a lot of online testing.
It was honestly fine. They made me use an HD webcam, show my drivers license, and pan around the room to ensure I was by myself and didn’t have an easy way to cheat. They also made me share my screen on Zoom and open up the Application Monitor to ensure I was only running Firefox. They didn’t let you take notes on paper, just using a portable whiteboard that they watched me erase before leaving.
I’m not saying it’s impossible to cheat, but I would say it’s probably roughly as difficult as cheating in person. Presumably the SAT stuff would be comparable.
I feel like people who cheat on digital tests could learn a thing or two from cheaters in video games. They have cheats that are invisible on screen-shares and that are super hidden (this mostly applies to AC bypasses, but some cheats run at a kernel or hardware level)
Yeah, as I said it's obviously no impossible to cheat in WGU, I don't even think that they're claiming that. I think they're just doing a reasonable level of due diligence to make digital cheating comparatively difficult to in-person cheating.
Could someone do some kind of hacked video driver (or something similar) to fool screen sharing, or have a fake ID made so that someone else could take the test in their place? Of course, but that's difficult enough to where fundamentally a vast majority of people simply won't do it. I don't think most cheating deterrents have the goal of eliminating 100% of cheating, but rather the 95+% of potential cheaters who would do the easy stuff.
I mean, if you're willing to put in a lot of effort to cheat on a test, at some point it's easier to just study and pass it legitimately.
If I read correct this is about taking the test physically in a testing centre on a computer from the testing centre. It is not about taking the test remotely online from your house or on your own device.
Still relying on the test being taken in a monitored environment, like at school or a test center. They emphasize that this new digital delivery means questions are less likely to be taken out of the test environment and published.
The standardized test cheating industry will still pay people to take the test and memorize certain questions, just like they've been doing forever. But that form of cheating might become a bit less useful when adaptive testing means that individual test takers are less likely to see the same questions.
Hi HN, I’m currently sitting in a testing center right now, and the technicial issues are real. Among other things, the testing app needed a last minute update, the preloaded test on the app failed to load for everyone, and the Wi-Fi is down/at a crawl from people using it to download said update. This is fun.
Also, for anyone wondering, the password for every student (the one that prevents people from taking the test at home) is 8 characters and *extremely* easy to guess. I won’t say it here, but given that this code grants access to take the test in the first place, you’d think it would be more complicated.
If the software doesn’t report that it suspects it’s in a vm at test taking time, but instead records data for later analysis the test taker is leaving themselves open for a lot of problems months down the road.
Who is going to pay someone to look for problems months down the road?
Ultimately if malware doesn't think it's a VM, then test taking software isn't going to either. Malware is trying to resist analysis by trained professionals. Test taking software is just trying to sell to the next test; the sale is made by the company that takes the testing committee to the best sportsball game, not by how well they detect being in a VM.
(Interesting corollary; you should probably ALWAYS run inside a VM. There is so much malware that tries to detect it that it actually helps you; the malware will just shut itself off out of fear of being analyzed by a security researcher.)
Perhaps they are following market pressure to streamline operations. The Classical Learning Test (CLT) is a growing SAT/ACT/PSAT competitor that can be taken online or on paper, with either remote or in-person proctoring. It is accepted by 250+ colleges and universities.
They are making it quite a bit easier for the students to score high.
Which seems unfair to anyone who has already taken the test in the old way
> The test is also one hour shorter (down from three hours), has shorter reading >passages and uses digital tools, like a highlighter, a graphing calculator and a >bookmark to go back to skipped questions.
That's what I thought at first also. But the test is adaptive, so if you score well on the first part of the math, for example, they give you harder questions in the second part.
In theory, it should actually be possible to make the test harder than it was before, using this framework.
From what I've heard the new test is more volatile/less informative. I wonder if this is a result of College Board reacting to the trend that started a few years ago where universities started saying that the SAT was a bad indicator for equity reasons.
> I wonder if this is a result of College Board reacting to the trend that started a few years ago where universities started saying that the SAT was a bad indicator for equity reasons.
Most of those universities have flipped flopped into requiring the SAT/ACT again because the other indicators (GPA, class ranking, extra curriculars) were actually even worse for equity reasons.
How do you even monitor who is doing the test? Do they have a ChatGPT window open? What additional tools they are using etc. seems like a terrible idea.
Essentially spyware that potentally records your screen, camera, keystrokes, mouse movements, running processes, and connected peripherals. It might also run constant integrity checks on itself to ensure no modifications are made.
I'd rather go into a testing center and use someone else's device than infect my own system with whatever spyware they'll insist students install to try and prevent cheaters.
Yeah, good luck with that. If there's one thing remote teaching during covid taught me it's that you can't stop someone from cheating during an online assessment.
I graduated from WGU and I wouldn't have known how to cheat on the many proctored tests. The only special software I installed was Zoom. I had to have a detached USB camera and show myself and my testing area, and then during the test the camera was to my side showing both myself and the laptop screen.
I'm sure it's possible, but that seems just as secure as a random highschool teacher, which was the only anti-cheat of decades past.
I wonder when they changed that. I was there around 2014, and back then you either had to install their anti cheat software, or go to a local testing/proctor center.
How is that experience relevant? Students aren't taking the test remotely, they are taking it in a controlled test center, just possibly with their own device.
It will only be offered in testing centers and high schools. However, students will use either school-owned devices or their own. I believe they can download and take a practice test on the device in advance, to make sure everything works smoothly. I still wouldn't want to install the stuff on my device!
"This week, students in the U.S. will begin taking the new SAT on their own devices — including a tablet or a laptop — or on school devices."
I'd thought "never trust the client device" was the golden rule to prevent cheating. A background process that reads the display and fills in the field when you start typing would be difficult to detect.
They have a ‘secure environment’ but the testing app ‘Bluebook’ is also available for download for practice tests and even to the general public. They have a chromeOS PWA, a Mac Universal app, and a Windows version.
I can also personally attest that the app will run perfectly happy inside a Windows virtual machine, even inside the ‘secure testing mode’, as this is how I personally took the 11th grade PSAT a few months ago.
So what you're saying is someone could record their screen the entire time, with no one the wiser? It would be tricky to use any external programs because the Bluebook app might be able to detect that it's not in focus, and other students/proctors could see something else on the screen.
Yes. A virtual machine specifically bypasses any on-device protections, protections which the terms of service indicate they have. However, I think the primary deterrent to cheating here would be as you mentioned the proctor and the other students.
To get around this, I theorize a more subtle overlay could be fashioned (imagine some sort of screen reader that speaks chatGPT-generated answers through the provided earbuds). However, at that point you could spend your time studying for the test in the first place.
If this is (and remains) true, I would guess that someone will build it and sell it to others. There are enough people out there who will never get above a 1200 — with any amount of studying/coaching — and who would happily pay $2k for a guaranteed 1500+. You wouldn't even need an LLM, really. Just a cellular-enabled laptop that can take screenshots every few seconds and transmit them to a confederate who whispers in your ear. Or to be even less detectable, just faintly displays a letter at the bottom of the screen once the problem has been solved.
1. if you can figure out how to do this, you're intelligent enough to score in the top 90th percentile on the exam.
2. good luck getting away with this in person at the proctored exam.
With the kind of money involved sophisticated cheating by professionals is a thing. This isn’t kids in the bathroom level cheating it’s mob and organized crime level.
So how is it worse than what happens now? These tests are worth a lot to a very small subset of people. The average high schooler (even the bottom 85%+) does not have access to those resources
Yep, the article says the exam is still on-site and proctored. Any cheating would have to be self-contained to the laptop and not obvious to someone looking over the cheater's shoulder.
Hypothetically they could require that you use a device with an included monitor (laptop, tablet) and require that no HDMI cable is plugged it.
This won’t protect against somebody who is willing to open the device up and probe the embedded DisplayPort (or whatever standard) pins, but that’s probably beyond the ability or interest of most people.
I don’t like any of this; if I were asked to install some school spyware I would refuse. But it looks like the plan is for the schools to also allow the students to take the tests there. People should do that instead. Hopefully some parents will be super pedantic and require schools to provide accessible testing space and not use the “you can test at home” option as an out.
People dont talk about this, but the SAT used to be more of a signal of intelligence and academic ability. In the 70-90s, you could really trust a good SAT score. Its been nerfed, they purposefully removed the performance aspect, its now very common for not that intelligent people (relative to the true intellectual elite) to get perfect scores.
This changes who ends up at elite institutions, and effectively who makes decisions in society.
Certain hedge funds and elite companies stopped looking at SATs as a hire indicator. Its well known that its no longer a reasonable measure of top talent
I retook the test a couple months later inperson at a testing center (where they used computers with the same lockdown browser) and got 99th percentile scores on everything, and I would recommend anyone planning to take the SAT to still take it at a testing center.