In general, you always have to take into account the Law of Leaky Abstractions ( http://www.joelonsoftware.com/articles/LeakyAbstractions.htm... ).

No matter how convenient a new abstraction might make things, it will some day "leak". If you don't know anything about the layer underneath, at that point, you're pretty shafted.

Some abstractions are more leaky than others. ActiveRecord is a very leaky abstraction. Make sure you have a solid grounding in database usage, db model design, normalisation, etc

As for the web in general, your stack of abstractions reads something like: ActionController > MVC for web apps > CGI-BIN/Servlets > HTTP > TCP > IP > ...

You can probably stop around HTTP or TCP and be ok.

I don't think you'll find a single book that covers all that, but maybe someone will have some suggestions.

Please note you don't need to become an expert at all this stuff - but you need to have a pretty good idea how it all fits together.

Good points, but you made that sound kind of complicated.

Here's an example of a Leaky Abstraction. My programming language is called Ben (hey that's my name!) --

# build_webpage()

Tada. Fire up your browser, point it to localhost and bam! you have your own personal webpage with a photo section and a forum. See how easy it was.

Then you think to yourself, wait -- I want a classified ads section, too!

Ah, this is when the big leak happens.

My Ben framework doesn't include that bit, so you are going to have to dive down into the C that I wrote Ben with and muddle about it to write that code yourself.

It gets nastier too.

You could probably hack together a classified ads section in your favorite language in a couple hours, but since you've bet the farm on Ben, you have to get up to speed on how my framework works and you have to use my language of choice.

The abstraction is now leaking so much that you get out PHP and just do it all by hand.

What point are you trying to make here? That using frameworks is bad because they won't magically solve all your problems? That no magic fix exists? That everybody should use PHP? That C is a bad language for writing a web programming DSL?

It takes 10 years to get good at anything. This quote is from OOAD.

>>The amateur software engineer is always in search of magic, some sensa-tional method or tool whose application promises to render software development trivial. It is the mark of the professional software engineer to know that no such panacea exists. Amateurs often want to follow cook-book steps; professionals know that such approaches to development usually lead to inept design products, born of a progression of lies, and behind which developers can shield themselves from accepting responsi-bility for earlier misguided decisions. The amateur software engineer either ignores documentation altogether or follows a process that is docu-mentation-driven, worrying more about how these paper products look to the customer than about the substance they contain. The professional acknowledges the importance of creating certain documents but never does so at the expense of making sensible architectural innovations. The process of object-oriented analysis and design cannot be described in a cookbook, yet it is sufficiently well defined as to offer a predictable and repeatable process for the mature software development organization.

Frameworks are just fine, tools are just fine, any language is just fine. It is all about the skill of the person using it. I have seen good PHP code. I have seen bad Python code. It is about the developer, their understanding and how much they can handle of the system. If a framework lessens what they need to know to something they can manage at their level then it will work.

The job of a good engineer, is to take something complex and make it simple. So there are and will always be great tools to help people get better and the best will dig deeper and after 8-10 years it seems like most of the marketing and b.s. and jungle of software starts to come clear.

Add DNS to the list of necessary bits. I'm always stunned by how little most web developers know about DNS, and how often it bites them in the ass. It comes into play on all sorts of performance, redundancy, reliability, scalability, and server migration questions. It's not a particularly hard or deep subject (at least, to grasp as much of the topic as a web developer needs to know), but if you don't know the basics, you will run into problems at the worst possible time.

Our products touch all aspects of a web developers stack, and so I suspect our forums are pretty representative...and DNS problems are the second most common type of problem we hear about (trailing well-behind email, but well before PHP, Ruby, Python, Perl, performance, security, and pretty much everything else). I pretty much expect to recommend O'Reilly's DNS and BIND daily, and keep the BIND page of our wiki open at all times for referencing in my responses.

+1 for studying database design/normalization. I took a class on it and the analyses of performance on different queries for different organizations was something I probably wouldn't have understood just by reading a book. Understanding how it works will also give you the courage not to be a slave to other people's recommendations; sometimes you want to duplicate data for performance reasons, but you have to make sure to manage that in your control logic.

Agreed.

Essentially, any web framework or web programming is a way to give a GUI front end to your database. Learn the database layer, learn how to write sql, learn how to break down your problem into a database model, and be able to think about it in terms of rows and columns in a database.

All the database stuff that you learn is going to scale pretty linearly with any type of web application that you're doing. That is, the more database stuff that you learn the more you're going to know about web programming.

I'm in somewhat the same boat as you, except using python and django. I've found this book to be pretty helpful in terms of a general understanding of web programming: http://philip.greenspun.com/seia/

I second the SEIA recommendation - it helped me out a lot when I started. I also suggest the HTTP definitive guide book. http://www.http-guide.com/ .

You might want to learn PHP first, especially if you're new to programming. Not only is PHP easy to learn, but it is also useful for quickly testing out simpler webpages.

By learning PHP, you'll also learn how web programming is traditionally done and the basics of how the web works. Once finished, you'll clearly see how Rails makes certain aspects of web programming easier, even for things like form validation.

I recommend a book like "PHP 6 and MySQL 5" which is very easy to read, and definitely type in the examples! http://www.amazon.com/PHP-MySQL-Dynamic-Web-Sites/dp/0321525....

Over and over and over, I've made this same recommendation when someone is new to the art and is banging on the Rails train: Do PHP first.

You have nothing to lose learning PHP. I love Rails, but I use PHP in a pinch. Hell, even DHH said he fired up PHP recently and liked how quickly he could wire up small bits.

Rails is too magical and mystical. If you don't understand the lifecycle of web requests, you are going to be shafted really quickly when Rails starts its magic tricks.

I am reluctant to do PHP because of time contraints. I am working on this stuff after work, and at some point I would like to actually be able to build something. Anything. When someone says do PHP, my visceral reaction is no.

However, if the experts here think that PHP would be a worthwhile detour even though I plan to go with Rails, then I'll go with that. I guess I'd just like some assurance that I can pick up the basics this way and that I can do so in a reasonable amount of time.

Yes, I know patience is a virtue, but I just want to make sure that my time is spent as productively as possible. I do very much appreciate the advice from everyone.

I actually learned Rails after knowing PHP, and I felt that my knowledge of web fundamentals (generally, how web sites are built and architected) was greatly enhanced by my PHP knowledge. It also gives you an appreciation of what Rails makes easy!

If you're familiar with a C-based programming language, then PHP will be very fast to pick up. And, if you want to "actually be able to build something," with PHP you'll absolutely at least feel like you can haha.

Well, you'll probably need to add 'server administration' to your list of things to learn, since by going with Rails you'll probably need your own VPS and have to manage all of your mongrel servers and make sure they stay running. They tend to use a lot of memory.

Why not do "Ruby sans Rails" instead? You can keep the Ruby experience, but get all of the lower-level request/response experience you like.

Ruby is a good language for CGI programming.

That's a really good question and one that I had as well. The main problem for me up until now is finding info on web programming that does not mention Rails.

Sloppy is a function of the guy at the keyboard. I've seen beautiful PHP source and shitty Ruby.

Don't skimp on the database design. It can make your life a lot easier if you're model is tight.

Learn how to think about your data model as Entities and Attributes and the relationships between them.

Then find a tutorial on database normalization and learn it. It's not hard, but it might take 2 or 3 run throughs before it clicks.

Learn to be comfortable at the *nix command line if you plan to be hosting your app on a shared host or Virtual Private Server.

Most of all; don't get caught up in fads. It's easy to get the impression that you need to learn AJAX, Rails (no wait, Merb!), S3 and EC2, distributed programming, jQuery, ad infinitum, all at once. You don't.

Here's some Google terms:

"Database normalization"

"entity relationship diagram"

"Unix shell commands"

It's certainly useful to know how things got to where they are, but it's not always necessary. Even in history circles people debate how important history is if the lessons can be handed down via knowledge of good-practices.

If you want to know how good you've got it now, I'd start with building a site that uses entirely static content. After you've got a few pages, make a change that affects all of these pages. You'll see how static pages lead to redundancy and a lesson will be learned.

Then setup a cgi based commenting system in either c or perl, which would arguably represent the start of interactive web sites. From here, you'll see that there are things you'd like to store, like a username, and you'll understand how frameworks make following a user easy.

Once you've done these two things, add some database support for the comments and a way to search the comments.

I'd say, after these three things, you'll have a pretty good understanding of how slick things like rails are and you can go back to being as productive as possible with respect for the problems it's solving. There's definitely more to learn on this topic, but it might border on distraction at this point.

In other words, you need to understand the problem before you can make sense of the solution.

And cookies! If you want to know what pain is, you should try managing cookies by hand.

Start with these 2 books from Philip Greenspun:

- Philip and Alex's Guide to Web Publishing

- Internet Application Workbook,

http://philip.greenspun.com/books/

The key to writing good web apps is to remember that the web part of the web app is just that, a part. It's not the whole application. An ideally-structured web application will be a normal application with a thin layer of glue to make it talk to the web. Usually your framework provides the glue, but it seems that most developers try to make the framework a religion and way of life instead of a glue. This leads to unmaintainable and untestable applications.

Anyway, the rule of thumb is... if critical code can't be executed without starting a web server, your app is designed wrong. (I've seen developers that write all their tests in Selenium. This is a huge waste of time; make sure you can test the backend code without making web requests. Only use Selenium to test JavaScript! Even then, Test::More from the JSAN is quicker.)

I've given lots of talks about this, if you want more details. Here is a web-specific one:

http://www.jrock.us/fp2008/catalyst/start.html

(left click to advance the slideshow)

Here is a non-web-specific one: http://www.jrock.us/ppw2007/testmore/testmore.txt

If you want to see it as a slideshow, try this:

http://www.jrock.us/ppw2007/testmore/takahashi.xul?data=test...

but on my machine, recent versions of firefox seem to not execute takahashi.xul correctly. So just read the text file ;)

Anyway, have fun.

>> I am attempting to teach myself Rails and have also made a concerted effort to understand Ruby.

There's your problem number one - trying to use and excel at Rails without understanding Ruby means that even if you become a moderately proficient Rails user, you won't know what you're doing, and will be unable to process past something you have a cookbook answer for. You can drive a car without understanding how an engine works (and assume that when you press the pedal down, "magic happens" and you accelerate), but to become a great driver you have to understand what makes the car go, and how to react to and optimize your behaviors for that.

Ruby is a language that does more than Rails. Rails is a framework that's written in Ruby.

Learn the language first; then you'll know what's actually happening with Rails and what those commands mean. Net result is that you'll be a better Rails user, and you'll be able to use Ruby for many other things and in novel ways.

Separately, learn about database normalization and design. For this, start with the mantra "down, not across" - your tables should be much longer than they are wide, even if this means multiple tables.

Sound advice. I recommend 4 books: Black's ruby for Rails and Matz's first edition in English Ruby in a Nutshell. Black's book is a great intro to exactly the techniques of method_missing, mixins, class_eval, #included, etc, you need to understand the world of rails source and plugins/gems. 1st edition Ruby in Nutshell covers ruby 1.6, is small enough that you can read the whole thing carefully several times. The 2nd edition, by Flanagan and Matz, is also a must-have reference, but not something you read straight through.

Also recommended: the ruby/metaprogramming chapters in Ediger's "Advanced Rails" (oreilly) and Rappin's "Professional Rails" (Wrox). All excellent resources

Also, for javascript/ ajax: Flanagan's JS reference, and Crockford's "JS, the Good parts"

Great question... I am somewhat in the same boat you are, but about a year further. One really good website that I found has filled in some gaps for me is http://www.buildingwebapps.com (although I haven't seen the specific issues you've mentioned).

EDIT: Now that I think about it, I probably need more fundamental knowledge of algorithms, and computer science in general than the things you mentioned.

Architectural Styles and the Design of Network-based Software Architectures (http://www.ics.uci.edu/~fielding/pubs/dissertation/top.htm) is a very readable dissertation on Web architecture. It is the definitive source on REST. Architecture of the World Wide Web (http://www.w3.org/TR/webarch/) is also worth a visit.