Welcome to College Confidential!

The leading college-bound community on the web

Sign Up For Free

Join for FREE, and start talking with other members, weighing in on community discussions, and more.

Also, by registering and logging in you'll see fewer ads and pesky welcome messages (like this one!)

As a CC member, you can:

  • Reply to threads, and start your own.
  • Post reviews of your campus visits.
  • Find hundreds of pages of informative articles.
  • Search from over 3 million scholarships.
Please take a moment to read our updated TOS, Privacy Policy, and Forum Rules.

Reuters: How Asian test-prep companies swiftly exposed the brand-new SAT

Roger_DooleyRoger_Dooley Founder Posts: 106,392 Senior Member
"Booklets for the redesigned exam leaked online within days of the test. The ongoing failures to secure the SAT are prompting some college officials to question the validity of exam scores."

http://www.reuters.com/investigates/special-report/college-sat-two/

CC gets mentioned in passing. Looks like College Board has a major issue to deal with, and that improved technology is the only solution. Obviously, single-use tests are essential. But, as long as the exact same tests are administered across the globe in different time zones, leakage will be a problem.

Certification tests for companies like Microsoft and Cisco have used randomized tests offered on-demand electronically for many years. That would be a huge change and would require major infrastructure investment, but it would avoid test-leaking scandals like this one.

It probably wouldn't work for TCB, but language instruction firm Duolingo is allowing people to take tests on their own device. They reduce cheating by monitoring the test taker in real time from a central proctor who views the device's front-facing cam. One proctor can monitor multiple test-takers.
«13456710

Replies to: Reuters: How Asian test-prep companies swiftly exposed the brand-new SAT

  • PlotinusPlotinus Registered User Posts: 892 Member
    edited March 2016
    As I wrote in the thread about the test registration cancellations, barring adult test-takers was just for PR and not at all an effective measure against cheating. At least Reuters has brought a couple of the gaping holes in SAT test security to public attention. There are still others.

    There are different test security models used by different tests. The other main paper test is the LSAT - a one and done test. Obviously, it is also a much more expensive test. However, I don't see how CB could move the SAT online, even to the test-taker's own device, in the near future. That would go completely against the CB campaign to "open doors" to underprivileged students. Would underprivileged students have suitable devices and know how to use them sufficiently well? Would not digitalizing the SAT to own-devices reintroduce special hurdles for the underprivileged?
  • Roger_DooleyRoger_Dooley Founder Posts: 106,392 Senior Member
    Good points, @Plotinus . Of course, mobile devices have been used to make services like banking available to millions on underprivileged people with limited access to physical banks. Obviously, this is a hugely complex effort with different security concerns. I wouldn't expect to see it happen for many years, even if a practical approach emerged.

    One option that would provide easy access would be a proctored exam using physical computers or tablets in existing schools. Test times could be booked in advance and conducted over a period of days or longer so you wouldn't need one device per test taker. Randomization of questions from a large database would be essential, though, to prevent earlier test takers from providing significant help to later ones.
  • MYOS1634MYOS1634 Registered User Posts: 30,561 Senior Member
    Underprivileged people and especially developing countries have been on the forefront of mobile technology because that's often the only bank, source of information, source of education, map... there is. Literally.
    (Kenya was about 5 years ahead of the US in terms of mobile banking, for instance).

    ETS digitalized the TOEFL years ago - and its format is similar enough to verbal (CR) on the SAT that I don't see why the SAT is on paper, not
    Devices could be provided - which school in the US doesn't have computers/tablets? A time to practice with the device would be included before each test, just in case.
    Abroad, the situation could be case-by-case, depending on country. Kenya, Germany, China, and Nepal have different problems.
    Front-facing cameras monitoring test takers and randomization of questions would be an obvious solution, too.
  • PlotinusPlotinus Registered User Posts: 892 Member
    The TOEFL has a very different target population of test-takers than does the SAT.
    Do you think so many underprivileged internationals are taking the TOEFL and the SAT? The problems for digitalizing the SAT are mainly the underprivileged in the US.
  • PlotinusPlotinus Registered User Posts: 892 Member
    If the real concern were test security, wouldn't it be easier and less expensive to stop recycling tests than to move the SAT to a digital platform? Oh, I forgot. Moving to a digital platform would mean more money for Big Data. Bill Gates must be smiling.
  • NotVerySmartNotVerySmart Registered User Posts: 1,670 Senior Member
    I do wonder if a digital test wouldn't be more vulnerable to cheating, in some ways. I can see two.

    1. Questions would have to be separated into different levels of difficulty. Otherwise, a "true 600" student might see more difficult questions than the average and get a 500 on a given section, or a "true 700" student could dodge the handful of questions that your average test would include to separate 700s from 800s, and get an 800. Rather than separate questions into tests (300-400 questions each), the CB would have to separate them into difficulty levels (each a large fraction of the question bank, likely 10-20%) in order to randomize the SAT. Leaking a single difficulty level compromises the test until the full level can be rewritten.

    2. The question bank would have to be small enough for the CB to write the questions and test their difficulty with an extensive control group. That fact limits its size; compiling 5,000 questions would be a time-consuming and expensive endeavor. This in turn makes it beneficial for a student to book a time at the end of the test period and read questions as they're posted online. This is true whether the test is on-demand for 9 months of a year or for any shorter period - say, a week. An on-demand test would mean a larger question bank, but also give cheats more time. A 5-day test period (for instance) gives cheaters less time, but also makes their job easier by limiting the size of the question bank, because the CB would need several sittings each year to accommodate all test-takers.

    Realistically, a student can cover 100 questions in a day, at the very least - the SAT itself asks them to answer far more, in far less time. 300-400 is a more likely figure. This means a 1,000-question database for a 2-3 day test period is wholly inadequate to prevent cheating; even 5,000 questions would be iffy, and I doubt the College Board has the time, the resources, or the will to produce and test 35,000 questions and accommodate 7 yearly sittings.

    The other option is a massive question bank, well into the tens of thousands, available on demand. But this couldn't be updated all that often, making it vulnerable to cheating, an issue solved only by reducing the benefits of such an approach with 100,000+ questions.
  • NotVerySmartNotVerySmart Registered User Posts: 1,670 Senior Member
    In that same year, the College Board used the December 2012 exam internationally in January of 2014, an exam that previously had been recycled on at least two occasions both in the U.S. and elsewhere, even despite overwhelming evidence obtained from the South Korean Prosecutors' Office of South Korea, District of Yeok-sam-dong (Prosecutor **** oversaw the confiscation and seizure of illegal materials from at least 44 teachers/academies in South Korea, 2013 and this exam was one of many included in that blacklist) that the exam was seriously compromised and should never be used again.

    Ugh. The more you know...

    Also, the venerable prosecutor's last name isn't going to escape CC's filters.
  • 2018eastorwest2018eastorwest Registered User Posts: 333 Member
    CC is mentioned more than just in passing. Aside from other references about how test material is shared on this Board, the closing of the article states: “Have you read College Confidential?” Roy said in an interview. “All the detailed material is there.”
    This makes me sick.
    Is this Board perpetuating the cheating as the article suggests?
    If so, what is being done to stop it?
  • suzyQ7suzyQ7 Registered User Posts: 2,925 Senior Member
    Reddit had SO MUCH more detailed information than CC.

    I don't get why CB does not use several different forms abroad - randomized by location. That would help. Isn't that what they did here in March?
  • BusyNappingBusyNapping Registered User Posts: 150 Junior Member
    I understand the need for security and absolutely agree that no tests should ever be recycled by CollegeBoard. However, I don't know if giving the test online or on a electronic device is really the answer.

    I'm sure it is possible, but it would be complicated and might lower scores for students. My D17 has to take state testing online every year and she hates it. Often times, the reading or science passage is not visible with the questions, so it requires clicking back and forth between windows. The math sections are quite annoying when scratch work has to occur completely separately from the actual problem, requiring the student to look from the screen to the paper over and over again, transferring the information between each surface. Also, staring at a computer screen for 4 hours is enough to strain anyone's eyes.

    I'll be interested to see what ends up happening in the realm of electronic testing, but one thing is for sure, CB needs to stop recycling tests.
  • actisnowkingactisnowking Registered User Posts: 4 New Member
    edited March 2016
    http://twelvebytwelve.net/xts/finger-of-blame.html

    Written in 2013.


    ABSOLUTELY PROPHETIC.


    "And the contemporary testing regime inflicts injury even when no cheating has taken place. For example, a test may be cancelled in anticipation of cheating, with all examinees being forced to take the test at a later time, which may result in the missing of application deadlines and therefore delay of further study by a year. Or an examinee who scores extraordinarily high may be suspected of cheating, and distrusted, even though he came by his score honestly."


    "The Educational Testing Service has historically accepted no responsibility for the cheating that takes place on its exams, instead blaming students and their tutors. However, today's students do not have to take this blame lying down because the Internet has made available to them information that students in earlier times lacked — today's student writing an ETS test is more likely than ever before to know that some, perhaps many, of the students being tested along with him are cheating. In some examination settings considered in a broader discussion of cheating, an examinee's wondering whether he is the only one in the hall not cheating may spring less from paranoia than from being well-informed. It is the arrival of examinee awareness of massive cheating that is able to reverse the attribution of responsibility, as explained below."


    http://www.reuters.com/investigates/special-report/college-sat-one/

    According to the PowerPoint, College Board officials also considered another approach: Push ahead with all scheduled tests in every country, regardless of the security risks. Under this scenario, one of the “benefits” listed was giving the “appearance that security situation is under control.”

    UNBELIEVABLE.


  • jym626jym626 Registered User Posts: 51,687 Senior Member
    edited March 2016
    Lets again blame CB for the test hacked and copied . Its always their fault. No matter how tight they make their security, someone (s) from certain countries will hack/steal/compromise it.
  • skyovermeskyoverme Registered User Posts: 319 Member
    @periwinkle
    I think the US Congress should give them the will

    Exactly!

    Congress's role is to force companies to create better products and services!

    That is exactly why Congress was created.

    After Congress fixes this, I hope they can force Apple to not have my apps update so often on my phone!! Senator McCain asked about this but Congress has NOT yet "give them the will" to fix this!!

    http://politicalticker.blogs.cnn.com/2013/05/21/mccain-why-the-hell-do-iphone-apps-need-updating/

«13456710
This discussion has been closed.