We tried it and cheating happens, but for us it was bearable. We were running candidates through Codility as the very first level of screening (3 programming problems to be solved from home), then through 2 or 3 interviews. People were cheating on a test once in a while, but not badly enough to spoil the whole filtering process. At some point we started sending out a welcome e-mail before the screening session to make it clear that cheating is nothing more than a recipe for a seriously bad interview, that you really do not want to have. Helped somewhat, but still somebody cheats once in a while. I guess such is life.
For a non-programming recruiter it is way to easy to get impressed by a good talker who knows how to throw in appropriate buzzwords in a conversation. You can judge communication skills, but how on earth would you judge whether the prospective hire can code correctly if you cannot code yourself? IMHO the best way to go is to find a fried of a friend who is a programmer and get an opinion from him/her. It would also not hurt to run a candidate through 3rd-party test that require actual programming (my favorite is codility.com, but there are several other websites that provide similar services).
There is already codechef.com, codility.com, spoj.pl, online-judge.uva.es, ideone.com, turingscraft.com, etc. How is your service going to be different?...
