right arrow
Examples: Monday, today, last week, Mar 26, 3/26/04

Reuters: How Asian test-prep companies swiftly exposed the brand-new SAT

Roger_DooleyRoger_Dooley 6084 replies100309 postsFounder Senior Member
"Booklets for the redesigned exam leaked online within days of the test. The ongoing failures to secure the SAT are prompting some college officials to question the validity of exam scores."

http://www.reuters.com/investigates/special-report/college-sat-two/

CC gets mentioned in passing. Looks like College Board has a major issue to deal with, and that improved technology is the only solution. Obviously, single-use tests are essential. But, as long as the exact same tests are administered across the globe in different time zones, leakage will be a problem.

Certification tests for companies like Microsoft and Cisco have used randomized tests offered on-demand electronically for many years. That would be a huge change and would require major infrastructure investment, but it would avoid test-leaking scandals like this one.

It probably wouldn't work for TCB, but language instruction firm Duolingo is allowing people to take tests on their own device. They reduce cheating by monitoring the test taker in real time from a central proctor who views the device's front-facing cam. One proctor can monitor multiple test-takers.
edited March 2016
146 replies
· Reply · Share
«1345678

Replies to: Reuters: How Asian test-prep companies swiftly exposed the brand-new SAT

  • PlotinusPlotinus 939 replies19 postsRegistered User Member
    edited March 2016
    As I wrote in the thread about the test registration cancellations, barring adult test-takers was just for PR and not at all an effective measure against cheating. At least Reuters has brought a couple of the gaping holes in SAT test security to public attention. There are still others.

    There are different test security models used by different tests. The other main paper test is the LSAT - a one and done test. Obviously, it is also a much more expensive test. However, I don't see how CB could move the SAT online, even to the test-taker's own device, in the near future. That would go completely against the CB campaign to "open doors" to underprivileged students. Would underprivileged students have suitable devices and know how to use them sufficiently well? Would not digitalizing the SAT to own-devices reintroduce special hurdles for the underprivileged?
    edited March 2016
    · Reply · Share
  • Roger_DooleyRoger_Dooley 6084 replies100309 postsFounder Senior Member
    Good points, @Plotinus . Of course, mobile devices have been used to make services like banking available to millions on underprivileged people with limited access to physical banks. Obviously, this is a hugely complex effort with different security concerns. I wouldn't expect to see it happen for many years, even if a practical approach emerged.

    One option that would provide easy access would be a proctored exam using physical computers or tablets in existing schools. Test times could be booked in advance and conducted over a period of days or longer so you wouldn't need one device per test taker. Randomization of questions from a large database would be essential, though, to prevent earlier test takers from providing significant help to later ones.
    · Reply · Share
  • MYOS1634MYOS1634 41554 replies447 postsRegistered User Senior Member
    Underprivileged people and especially developing countries have been on the forefront of mobile technology because that's often the only bank, source of information, source of education, map... there is. Literally.
    (Kenya was about 5 years ahead of the US in terms of mobile banking, for instance).

    ETS digitalized the TOEFL years ago - and its format is similar enough to verbal (CR) on the SAT that I don't see why the SAT is on paper, not
    Devices could be provided - which school in the US doesn't have computers/tablets? A time to practice with the device would be included before each test, just in case.
    Abroad, the situation could be case-by-case, depending on country. Kenya, Germany, China, and Nepal have different problems.
    Front-facing cameras monitoring test takers and randomization of questions would be an obvious solution, too.
    · Reply · Share
  • PlotinusPlotinus 939 replies19 postsRegistered User Member
    The TOEFL has a very different target population of test-takers than does the SAT.
    Do you think so many underprivileged internationals are taking the TOEFL and the SAT? The problems for digitalizing the SAT are mainly the underprivileged in the US.
    · Reply · Share
  • PlotinusPlotinus 939 replies19 postsRegistered User Member
    If the real concern were test security, wouldn't it be easier and less expensive to stop recycling tests than to move the SAT to a digital platform? Oh, I forgot. Moving to a digital platform would mean more money for Big Data. Bill Gates must be smiling.
    · Reply · Share
  • NotVerySmartNotVerySmart 1608 replies62 postsRegistered User Senior Member
    I do wonder if a digital test wouldn't be more vulnerable to cheating, in some ways. I can see two.

    1. Questions would have to be separated into different levels of difficulty. Otherwise, a "true 600" student might see more difficult questions than the average and get a 500 on a given section, or a "true 700" student could dodge the handful of questions that your average test would include to separate 700s from 800s, and get an 800. Rather than separate questions into tests (300-400 questions each), the CB would have to separate them into difficulty levels (each a large fraction of the question bank, likely 10-20%) in order to randomize the SAT. Leaking a single difficulty level compromises the test until the full level can be rewritten.

    2. The question bank would have to be small enough for the CB to write the questions and test their difficulty with an extensive control group. That fact limits its size; compiling 5,000 questions would be a time-consuming and expensive endeavor. This in turn makes it beneficial for a student to book a time at the end of the test period and read questions as they're posted online. This is true whether the test is on-demand for 9 months of a year or for any shorter period - say, a week. An on-demand test would mean a larger question bank, but also give cheats more time. A 5-day test period (for instance) gives cheaters less time, but also makes their job easier by limiting the size of the question bank, because the CB would need several sittings each year to accommodate all test-takers.

    Realistically, a student can cover 100 questions in a day, at the very least - the SAT itself asks them to answer far more, in far less time. 300-400 is a more likely figure. This means a 1,000-question database for a 2-3 day test period is wholly inadequate to prevent cheating; even 5,000 questions would be iffy, and I doubt the College Board has the time, the resources, or the will to produce and test 35,000 questions and accommodate 7 yearly sittings.

    The other option is a massive question bank, well into the tens of thousands, available on demand. But this couldn't be updated all that often, making it vulnerable to cheating, an issue solved only by reducing the benefits of such an approach with 100,000+ questions.
    · Reply · Share
  • NotVerySmartNotVerySmart 1608 replies62 postsRegistered User Senior Member
    In that same year, the College Board used the December 2012 exam internationally in January of 2014, an exam that previously had been recycled on at least two occasions both in the U.S. and elsewhere, even despite overwhelming evidence obtained from the South Korean Prosecutors' Office of South Korea, District of Yeok-sam-dong (Prosecutor **** oversaw the confiscation and seizure of illegal materials from at least 44 teachers/academies in South Korea, 2013 and this exam was one of many included in that blacklist) that the exam was seriously compromised and should never be used again.

    Ugh. The more you know...

    Also, the venerable prosecutor's last name isn't going to escape CC's filters.
    · Reply · Share
  • 2018eastorwest2018eastorwest 301 replies32 postsRegistered User Member
    CC is mentioned more than just in passing. Aside from other references about how test material is shared on this Board, the closing of the article states: “Have you read College Confidential?” Roy said in an interview. “All the detailed material is there.”
    This makes me sick.
    Is this Board perpetuating the cheating as the article suggests?
    If so, what is being done to stop it?
    · Reply · Share
  • suzyQ7suzyQ7 3948 replies55 postsRegistered User Senior Member
    Reddit had SO MUCH more detailed information than CC.

    I don't get why CB does not use several different forms abroad - randomized by location. That would help. Isn't that what they did here in March?
    · Reply · Share
  • BusyNappingBusyNapping 147 replies3 postsRegistered User Junior Member
    I understand the need for security and absolutely agree that no tests should ever be recycled by CollegeBoard. However, I don't know if giving the test online or on a electronic device is really the answer.

    I'm sure it is possible, but it would be complicated and might lower scores for students. My D17 has to take state testing online every year and she hates it. Often times, the reading or science passage is not visible with the questions, so it requires clicking back and forth between windows. The math sections are quite annoying when scratch work has to occur completely separately from the actual problem, requiring the student to look from the screen to the paper over and over again, transferring the information between each surface. Also, staring at a computer screen for 4 hours is enough to strain anyone's eyes.

    I'll be interested to see what ends up happening in the realm of electronic testing, but one thing is for sure, CB needs to stop recycling tests.
    · Reply · Share
  • actisnowkingactisnowking 4 replies0 postsRegistered User New Member
    edited March 2016
    http://twelvebytwelve.net/xts/finger-of-blame.html

    Written in 2013.


    ABSOLUTELY PROPHETIC.


    "And the contemporary testing regime inflicts injury even when no cheating has taken place. For example, a test may be cancelled in anticipation of cheating, with all examinees being forced to take the test at a later time, which may result in the missing of application deadlines and therefore delay of further study by a year. Or an examinee who scores extraordinarily high may be suspected of cheating, and distrusted, even though he came by his score honestly."


    "The Educational Testing Service has historically accepted no responsibility for the cheating that takes place on its exams, instead blaming students and their tutors. However, today's students do not have to take this blame lying down because the Internet has made available to them information that students in earlier times lacked — today's student writing an ETS test is more likely than ever before to know that some, perhaps many, of the students being tested along with him are cheating. In some examination settings considered in a broader discussion of cheating, an examinee's wondering whether he is the only one in the hall not cheating may spring less from paranoia than from being well-informed. It is the arrival of examinee awareness of massive cheating that is able to reverse the attribution of responsibility, as explained below."


    http://www.reuters.com/investigates/special-report/college-sat-one/

    According to the PowerPoint, College Board officials also considered another approach: Push ahead with all scheduled tests in every country, regardless of the security risks. Under this scenario, one of the “benefits” listed was giving the “appearance that security situation is under control.”

    UNBELIEVABLE.


    edited March 2016
    · Reply · Share
  • jym626jym626 55107 replies2859 postsRegistered User Senior Member
    edited March 2016
    Lets again blame CB for the test hacked and copied . Its always their fault. No matter how tight they make their security, someone (s) from certain countries will hack/steal/compromise it.
    edited March 2016
    · Reply · Share
  • skyovermeskyoverme 311 replies8 postsRegistered User Member
    @periwinkle
    I think the US Congress should give them the will

    Exactly!

    Congress's role is to force companies to create better products and services!

    That is exactly why Congress was created.

    After Congress fixes this, I hope they can force Apple to not have my apps update so often on my phone!! Senator McCain asked about this but Congress has NOT yet "give them the will" to fix this!!

    http://politicalticker.blogs.cnn.com/2013/05/21/mccain-why-the-hell-do-iphone-apps-need-updating/

    · Reply · Share
  • marvin100marvin100 8558 replies1246 postsRegistered User Senior Member
    Periwinkle wrote:
    According to their 990 for the year ending in June 2014, the College Board made a profit of about $99 million in that year. Total assets in that year were about $1 billion.

    I think the College Board has the resources to create one-time tests. If they want to.

    I think the US Congress should give them the will. The College Board is a non-profit.

    Exactly. The CB is on the record as saying that it costs about $1 million to create a test. With clear, untaxed profit of $99 million, the CB absolutely has the resources to use unique tests in every test administration. The College Board must stop recycling tests.
    · Reply · Share
  • jgoggsjgoggs 208 replies22 postsRegistered User Junior Member
    Good for Reuters for doing some real reporting here, but the March 2016 cover page they reproduce in their story looks fake to me. A scan wouldn't look that crisp, and I don't think College Board says things like "SAT Test," which sounds more like non-native English. I could be wrong, of course--but it looks to me like someone took the cover page of one of the College Board's practice tests and just doctored it a bit.
    · Reply · Share
  • marvin100marvin100 8558 replies1246 postsRegistered User Senior Member
    They're on the record saying that the contents of that exam match reports of the test that was given in March and have run it by students who took the March exam, all of whom said it was the exact same test. I'm very inclined to believe the report, partly because of what I've heard from students here in Asia, but also because, hey, this is Reuters, not Buzzfeed, and Steve Stecklow has won a Pulitzer and three Polks. @jgoggs
    · Reply · Share
  • jgoggsjgoggs 208 replies22 postsRegistered User Junior Member
    I'm not doubting the report. I'm only doubting that cover. But again, I could be wrong.
    · Reply · Share
  • marvin100marvin100 8558 replies1246 postsRegistered User Senior Member
    Yeah, the cover may be synthetic.
    · Reply · Share
This discussion has been closed.

Recent Activity