The slowness is a self-defense mechanism. Slow down new signups and submissions.

Its genius really

Arc can heal.

At these moderate loads the problem is more likely to be in the design of the web server than the language itself.

In fact it was a design flaw, though not in the server.

The code for checking whether a username was taken was inefficient, so it was taking a huge amount of time to create a new account. The culprit was this line:

  (some [is dcuser (downcase _)] (keys hpasswords*))

The reason the TechCrunch post slowed the site down so much was not the traffic per se but the number of those visitors who were creating new accounts. I pasted a fix into the repl, so the site should be faster now.

Thanks to Rtm for narrowing the problem down to account creation.

   The reason [the site was slow] was not the traffic... 
   but the number of visitors who were creating new accounts.

The only better startup problem than that is "the site was slow because so many people were trying to give us money that our credit card processor crashed".

My friend Mike once told me: "When you can't tell the difference between public opinion and a DDoS, you're either doing something right, or doing something horribly wrong."

Are you coding live on the server? Or do you have a test platform that you work on first, and then past working code to the repl on the webserver?

Any chance you could fix a bug in the header handling as well? It's not accepting "Content-length" headers (lower case l) which is perfectly acceptable HTTP and happens to be what w3m produces. Means I can't post comments from w3m.

What type of tests do you guys maintain for news.yc other than the live deployment? Did this hiccup result in something like an account creation stress test being added to an automated test suite?

Hope you'll actually commit the change to the codebase so that the problem doesn't resurface on server restart like it happened once...

I wish my webserver had a repl.

Django has a shell that automatically loads all model objects. If you've got a decent shared-nothing architecture and point your config to the live DB (bad idea, but for the sake of argument...) that's effectively a REPL for the webserver.

For that matter, you can get the same effect with the MySQL prompt and a shared-nothing PHP architecture.

>For that matter, you can get the same effect with the MySQL prompt and a shared-nothing PHP architecture.

I see how you could, say, drop a user from the users table using this; but how would you fix a login bug like the one under discussion?

  EXPLAIN SELECT username FROM users WHERE username = $dc_user

Then look at the output, and either

  CREATE INDEX users_username ON users(username)

Or cache the result, or do something else to fix the performance problem...

If you meant "How do you find out it was in the registration check?", that's what the slow query log and query profilers are for...

FWIW, you can log into Erlang's Yaws.

or Seaside: http://www.seaside.st/documentation/tools

ruby script/console

I would agree. I just wanted to tongue in cheekly get this post over with.

I think your web server might be broken.

Rather a self-refuting comment...

I take that back. Though the problem was not in the server itself, the inefficiency in the account creation code was bad enough that it should be called a bug.

I was hoping a big blog would NOT post about news.YC I would rather have this community be small and focused rather then become something like Digg.

This was bound to happen, I'm afraid. If TechCrunch didn't do it, some Digg kidd would have.

The TC front-paging: A clever Arc stress-test organized by Paul Graham? Investigative report at 11...

For a language that was released a month ago I would be surprised if it did scale.

Time to write it in an enterprise language -- I suggest C++.

(Seriously, though, this should give pg and company some valuable benchmarking data for later on.)

maybe it was just the server?

What about arc makes anyone think it can scale? Is there an example of a continuations based web server that can?

Seaside (http://seaside.st), which powers dabbledb (http://dabbledb.com/).

Hacker news was a welcome relief from Reddit and Digg, Reddit is so political now that there are few articles anymore about "the craft". Aside from the the prerequisite preachings from the "GREAT!" Paul Graham (amen brothers), there was some very germaine content at times. I will be curious to see how they weather the coming storm, of course they do have have an ARC. Lol, I know ver ver bad pun.

programming.reddit is not bad.

But, yeah, if you're a hacker, hacker news is better yet.