Also, "crowd-sourced curated lists of websites" sound like the old Yahoo directory of yore. They will either become obsolete very quickly or spammers will find a way to penetrate and dominate them.
I believe the Yahoo lists were actually centralized. Hypothetically a fully-distributed approach could work better, in the style of Wikipedia. Realistically, well, I have no idea but it's worth a try.
Not only that, but Yahoo didn't think search was important, so it's not even a real try of enhancing search with curation.
The truth is, if some company comes up with a better search engine, whatever ideas behind it are not going to sound like an obvious win up front—if they did then Google would already be doing that. Instead they'll have to create a search engine that is better, but somehow antithetical to Google's business model so that they can't just copy it, because there's no way for a startup to come up with enough resources to stay materially ahead of Google in pure search. And of course that's only half the battle; then you have to be better enough that users can be bothered to switch (or a browser deal coup).
Personally I haven't found the spam problem to be nearly as bad as the echo chamber makes out. I think silicon valley types just have a good imagination about how good it could be.
I've been wanting to build a decentralized curated social search (buzzword soup I know). The idea is that security has to be based on trust, and adversarial search is a security problem. Here's my idea, it starts with delicious style bookmarks and a social graph. the search engine indexes the pages you've bookmarked so your bookmarks are searchable. AND, you can expand your search to include your friends bookmarks. Obviously it wouldn't be comprehensive, but it would work really well for shared interests like programming.
But it would be useless if you're the first person in your social graph to have a need for particular information; similarly, for it to be very useful you would want your users to push almost every website they visit into it, or any website which includes useful information (otherwise you would get too many searches filled with junk, or just with nothing). It's also useless if a user doesn't have friends using it (as compared to other programs; see below), and so bootstrapping up would be a bit hard.
Essentially, you're talking about something like Chrome's history search (it indexes the content of every page you visit, and allows you to do full-text searches of them), but with the ability to expand the search to other users indexes. You'll want to make it entirely painless for users to add pages - a keyboard shortcut at most - and integrate into the browser as much as possible. It's also probably a good idea to add some mechanism to identify users with reliable indexes and allow others to use them; a karma system could be used for this (add karma if a result is useful, remove it if it's not - similar to how HN works with comments). Users with really low karma (indicating that they're pushing lots of spam sites into the service) would have their results biased against in full-index searches, or outright removed without them being aware (similar to how users on HN can be killed, so that the see all their posts normally but no one else does).
Disclaimer: I'm not quite awake, so take this advice with a huge grain of salt. It's just my thoughts on the idea.
" It's also probably a good idea to add some mechanism to identify users with reliable indexes and allow others to use them"
Yup, social graph wasn't the right choice of words on my part. I was thinking about something like twitters "follow". For example, you could go around a follow a bunch of well known programmers.
I don't like the karma idea, at least not at a global level. I think it has to be about trust, at an individual level. These are the people I trust, search their bookmarks.
Also, "crowd-sourced curated lists of websites" sound like the old Yahoo directory of yore. They will either become obsolete very quickly or spammers will find a way to penetrate and dominate them.