I like Clojure (and its community) mainly because every time I saw a github Clojure project, it comes with a few things: test-automation (/test folder) and some way to build the project: get dependencies, run tests, package it up, and ready for deployment (using leiningen, a'la Maven).
That for me feels like home. (Plus I put premium on projects with test automation: it shows to me that the developer cares).
But I don't necessarily agree with Neal's first few minutes regarding "I'll choose technology X, knowing it will fail, over technology Y, knowing there's a risk but might achieve big win".
There's no big win when it comes to the so-called "Enterprise Project".
Either you met the requirement (pray it doesn't change too often, even if it does, pray that the management extends the deadline), or you under-deliver but within budget (cut features).
And most of the time it, the failure, is rarely because of the technology (.NET, Java, Ruby, Clojure) any more these days (excluding SAP, PeopleSoft, and some of the truly heavyweight). It's usually something else: management, people, developer's skill, overall culture, process.
>And most of the time it, the failure, is rarely because of the technology (.NET, Java, Ruby, Clojure)
I disagree, while it won't come down to C# vs Java - things like libraries, tools (that are tied to languages/platforms) and their suitability for the task that you're trying to solve can make or break your project, especially with a short deadline.
What kind of "snowflakes" (very unique) project you're talking about?
And how short is short?
I'd like to know the silver or gold bullet we're about to discuss here, I'm curious. (just don't bring ASM/C vs Java/Ruby lame example though, you know, them kind of example that is too obvious and too weak to disprove generalization-but-not-100%-always type of statement).
Quality UI tools and frameworks ? Ever tried to use those fancy thin-client frameworks like Cappuccino or ExtJS ? Spend days figuring out esoteric language/syntax/API and then fight with fragile implementations and no error checking + the ridiculous amount of boilerplate and bad structure. Compare that with XAML/C# with VisualStudio and tell me you don't see the productivity boost. Not to mention entirely different platform characteristics/capabilities.
As for libraries - just a week ago I was writing an app for image processing, among other things it needed to do indexing on images by extracting meta data in image headers and different formats (IPTC,EXIF,XMP) and there was no library to do that on Mono/C# that actually worked and after a day and a half of mucking around with half-assed wrappers I wrote the app to feed the DB in clojure with a Java library in few hours.
Sure in some abstract 10000 feet view tools don't make a difference, or if you're doing the simplistic CRUD stuff, but in day to day work it matters a lot that you get to use tools (eg. debuggers, intellisense, profiling), libraries, frameworks etc.
Uh hang on. Cappucino or ExtJS vs XAML/C#? Are you trying to compare Apple vs Orange or JS vs Flash/SilverLight?
One of the previous companies I worked for bet on WPF and it didn't go well. Memory hog, crashes, boilerplate code (sometime hidden by VS.NET), and the consensus was that WPF pretty much sucks out of the box (have to work very hard to make it look good).
By the way, another company I used to work for used GWT + Ext-GWT (GWT port of Ext-JS). It was pretty good as long as you don't have to modify the Look-n-Feel to be something it is not.
>Uh hang on. Cappucino or ExtJS vs XAML/C#? Are you trying to compare Apple vs Orange or JS vs Flash/SilverLight?
Silverlight vs Cappuccino and ExtJS. We tried both before silverlight and eventually gave up as the code became a complete mess (javascript and that monstrosity objective-j), debugging was hell (especially with objective-j) and documentation substandard (cappuccino had the assumption that you were keeping the apple cocoa reference handy I guess). OTOH Silverlight and MVVM Prism worked like a charm, we were able to redo the months front-end work in few days (everyone needed to pick up Prism). Things were super easy to mock and provide concepts with the Blend tool. If they only had TPL it would have been ideal.
>One of the previous companies I worked for bet on WPF and it didn't go well. Memory hog, crashes, boilerplate code (sometime hidden by VS.NET), and the consensus was that WPF pretty much sucks out of the box (have to work very hard to make it look good).
So you agree then - tools and platforms do matter ? If there was no time to "work very hard" would the project succeed ?
I want very badly to love Clojure and have done my best to try and use it in a number of medium-sized projects composed of several services... unfortunately it seems that deploying JVMs on EC2 (under Ubuntu, anyway) is just not reliable and we would always get into situations where the VM was locked up and pegging the machine at 100% CPU. Not sure if this was a Clojure problem or a JVM problem but it definitely was a huge headache.
That, and the fact that many of the libraries are immature (so you end up spending the time to fork them and add whatever they're lacking), and many of the error messages and exceptions it throws are pretty cryptic and useless... I'd be wary of Clojure in a production environment at this stage, unfortunately.
Which is really too bad because it is such a great language!
Excuse me if I'm totally off (I haven't actually used clojure, just evaluating it), but regarding the lacking libraries: isn't the fact that it runs in the JVM nullify this by having access to the rich java libraries?
The java interop is very verbose and kind of ugly (next to "pure" clojure code anyway) so many clojure wrappers or reimplmentations for existing Java libraries exist. So yes, sometimes it's as simple as extending the wrapper to expose the part of the underlying Java library you need, but other times there are fundamental issues with the design of the library that prevent it from being useful in a multi-threaded multi-user server environment (instancing etc).
Your problems with the exceptions is a problem with Java. As far as the libraries, Clojure can use any Java library natively.
Setting aside all that, how can you say it's a great language if in your experience it doesn't run reliably, has an immature library and troubleshooting is nearly impossible?
Well, the idea of leveraging the power and pragmatism of the JVM using a nice functional language is certainly appealing, and the moments in which it does work it's really nice. Not to mention the pragmatic and mostly idiot-proof approach to state and concurrency is awesome.
I guess I don't mean exceptions encountered at runtime, btw. I mean more like syntax errors during development. The Clojure compiler does not yet provide useful error messages or other information like line numbers in most cases when you have a bizarre paren-balancing issue or something.
What are people's experiences in writing such dense functional code in a production environment? What does maintainability/debug-ability/bringing other functional folks up to speed like?
Maintainability is immeasurably boosted by referential transparency; most of the time you can look at a single function in isolation and understand everything that's going on rather than having it be affected by other pieces of the codebase. I find it's much easier to work on a large codebase in Clojure vs Ruby just because it encourages a clean separation of concerns.
Of course in Clojure a 30k LOC project is considered very large. It's an open question whether this is due to Clojure's expressiveness vs the fact that it might not have been around long enough to spawn any massive projects.
Debuggability is helped a fair bit by having access to the repl in production contexts, but for the most part needing an actual stepping debugger is very rare, precisely because of referential transparency.
Bringing other folks up to speed is definitely the biggest problem. People who have experience with other FP languages usually don't know much about the JVM runtime and ecosystem, while people with Java backgrounds have a lot to learn about immutability and lisp.
Maintainability? Good. Yes, functional code can get weird and unmaintainable, but it's less common for it to turn to garbage, and the interactive mode makes it easier to fix modules before they get into states of advanced failure. Average-case Java or C++ turns to shit as soon as there are two people on the project who don't know each other. With functional languages, REPLs, and a good code review process in place, that's far less likely.
Most "bad" functional programs exist either because people apply FP ideology too strongly (i.e. eschewing mutable state on ideological grounds, despite the fact that it often makes programs better to model things that are semantically mutable with mutable state) or try to do things in a functional way when it's inappropriate (e.g. trying to beat C++ into functional-ness, generating code that's neither good C++ nor good FP).
Debugging? Referential transparency is a huge win here. It's not even close here. In large Java projects, it often gets to a state where no one knows WTF is going on. Java also does a lot to discourage testing (such as the requirement by many Java build systems that tests live in a separate directory). With a properly-written functional program, it's easy to test individual pieces.
Learning curve? Not an issue for smart people. Don't hire not-smart people.
That said, studying Scala (and especially its message-passing concurrency) has opened my eyes up to the relationships between programming paradigms: functional programming and OOP shouldn't be conceived-of as opposites. They actually work really well together, for a reasonable definition of each. The default means of abstraction should be a referentially transparent function. Objects should be used for large mutable systems intended to communicate through message-passing. (When possible, avoid sharing mutable state among more than one object.) Where Java failed is by taking OOP too far, and forcing people to create objects and classes in places where those (more powerful) abstractions are inappropriate to the problem being solved.
It will be most effective to grow the language from the bottom up by: propagandizing the benefits among developers, building bridges to other technologies (simple APIs), befriending other alternative language communities who are selling the idea that Java isn't necessarily the best and only way, and bury Clojure implementations in larger contexts.
That for me feels like home. (Plus I put premium on projects with test automation: it shows to me that the developer cares).
But I don't necessarily agree with Neal's first few minutes regarding "I'll choose technology X, knowing it will fail, over technology Y, knowing there's a risk but might achieve big win".
There's no big win when it comes to the so-called "Enterprise Project".
Either you met the requirement (pray it doesn't change too often, even if it does, pray that the management extends the deadline), or you under-deliver but within budget (cut features).
And most of the time it, the failure, is rarely because of the technology (.NET, Java, Ruby, Clojure) any more these days (excluding SAP, PeopleSoft, and some of the truly heavyweight). It's usually something else: management, people, developer's skill, overall culture, process.