Took a similar test at a recruiting company in ... 2006 (IIRC). Mostly on PHP. And the test was wrong. IIRC, I got 24 out of 25. They were ecstatic - "wow, no one in this office ever got such a high score - you were almost perfect"
"I was," I said. "One of those questions is wrong".
"Oh, no, it couldn't be - we have a team of experts who create these to the highest standards, blah blah blah.".
I asked again to go back, take a picture (crappy camera phone) and reviewed it again when I got home. It was wrong. Something to do with how references behaved in PHP5, and the exam's answer was right for PHP4, but the test was for PHP5.
Anyway, it was a bit depressing, and I don't want to have to go through those sorts of tests again if I don't have to.
"One of your questions is wrong" is an excellent filter. People should be thrilled to get that information.
I corrected two questions on a multiple-choice test that a finance firm gave me. They flushed me out after I handed in my personality test, so maybe it was for the best.
Yeah, you'd think it would be, but it's also indicative of someone who might be 'too big for their britches', or some other such nonesense. Given how many fakers I've encountered doing crap work for people (not just in software, but any service), I'm sadly inclined to realize that when I say "one of your questions is wrong", I'm very likely to be dismissed.
In the early 2003 time frame, I ended up taking a bunch of multiple choice tests about Java, C and C++. Every single one of them had things wrong. As near as I can tell, the auto-capitalization feature of Microsoft Word caused most of these problems. For example 'boolean' and 'Boolean' denote very different types in Java. In one test, they meant 'boolean' but had 'Boolean' written out.
I would definitely be thrilled. I wonder if intentionally putting an incorrect assumption in order to see if someone brings it up would be a good indicator. The idea wouldn't be to penalize someone who didn't notice the assumption, but to reward anyone who noticed the mistake and brought it up.
"I was," I said. "One of those questions is wrong".
"Oh, no, it couldn't be - we have a team of experts who create these to the highest standards, blah blah blah.".
I asked again to go back, take a picture (crappy camera phone) and reviewed it again when I got home. It was wrong. Something to do with how references behaved in PHP5, and the exam's answer was right for PHP4, but the test was for PHP5.
Anyway, it was a bit depressing, and I don't want to have to go through those sorts of tests again if I don't have to.