I think that the real problem in operation over here is information deluge. People aren't reading stuff on that page because it overflows on a minute by minute basis. So let's say, if I click to read a long article and I come back to upvote it. This would mean that I am putting a flag for others to read it, but in this time frame it has moved beyond the first new page. Hence, the stories that tend to get upvoted are the ones with a recognized tilt that appeals to the lowest common denominator (which is still pretty high, but for how long?)
However not all stories are created equal. Most of the stories on that page are from noob users who want to jack up their karma. Hence, the deluge HN on the weekdays (only dedicated ones stay around on the weekends) leading to this difference between the weekday and weekend articles.
How can we solve this?
One simple way to do this would be to post noobstories only to the noob stories page and sort the entries not by time, but by karma of the user as well as the votes given like the comments.
This way we could have a quasi front page which will create a positive feedback loop. Hence, there will be an implicit reward in going to the new page which will ensure that only the good stories were posted. Also, since older users (by karma, PG's metric) who are more settled in and are less likely to do karma antics it won't overflow every minute or so.
However, this might create a barrier to entry on HN and perhaps that's a good thing. This would force people to comment and get karma before it shows up over there. Hence, ensuring that the quality get maintained.
All in all with the same code behind the comments the new page problem can be solved.
It seems like eliminating the ability for noobs with less than, say, 20 karma from submitting stories would fix this and get rid of a lot of the spam that gets posted.
I know it seems unwelcoming to noobs, but we already have a tiered system where you need N karma for your votes to count, to downvote, to submit polls, etc.
I didn't submit anything for months after joining, which I think is the best way to go. Through actively participating in the comments, I got a good idea of what the HN community was really like.
And even now I don't submit that much (though since I discovered the bookmarklet my rate of submissions has increased). HN isn't about submitting to me, it's about comments (though obviously I realize we need good stories to comment on).
The biggest drawback I see is that it might impact the quality of comments. If users desperately want to hit that submission threshold, they might comment a lot or comment group-think-ily trying to pick up a few points here and a few points there until they hit that threshold.
I've thought about this too and it looked like a tough problem to solve, but it isn't.
Why do people do that?
They usually do it for Ask HN posts and well you can easily discriminate for them. Further if anyone tries to spoof the system by adding a URL there then we can parse it to see if there is any text there according to the stupidity filter. So, essentially it will still work.
Instead of having a link directly to the article, have the link go to the discussion page instead, but with an added header.
That extra header would check for a cookie. If that cookie is set, no action. If the cookie is not set, set the cookie, then forwards the user to the actual article.
Then, when you finish reading and decide it's worth an up vote, when you go 'back', you're already at the discuss page where you can vote the story itself up, as well as comment on it.
I often go to the comments page for a submission first, then click the article link. Particularly if it looks like it's going to be one that drifts off. That way the back button puts me in the right place, even if I come back to it hours later.
I must say, that's a mighty fine idea you have there.
Although it may surprise some folks at first. If HN had ads it would seem to be click fraud. Might work as a bookmarklet or browser extension.
It does seem like things fly by too fast. Here's an idea: what if every person saw only a random subset of the new stories?
Kind of like how Facebook has "Top News" and "Most Recent," but instead of intelligent filtering there would a simple random selection. The selection would be persistent, so that you'd see the same articles when you went back to the page.
However not all stories are created equal. Most of the stories on that page are from noob users who want to jack up their karma. Hence, the deluge HN on the weekdays (only dedicated ones stay around on the weekends) leading to this difference between the weekday and weekend articles.
How can we solve this?
One simple way to do this would be to post noobstories only to the noob stories page and sort the entries not by time, but by karma of the user as well as the votes given like the comments.
This way we could have a quasi front page which will create a positive feedback loop. Hence, there will be an implicit reward in going to the new page which will ensure that only the good stories were posted. Also, since older users (by karma, PG's metric) who are more settled in and are less likely to do karma antics it won't overflow every minute or so.
However, this might create a barrier to entry on HN and perhaps that's a good thing. This would force people to comment and get karma before it shows up over there. Hence, ensuring that the quality get maintained.
All in all with the same code behind the comments the new page problem can be solved.
(update: wrote a lot more detail)