Hacker News new | past | comments | ask | show | jobs | submit login
Three unique features of Lisp in 2010 (john.freml.in)
69 points by gnosis on Jan 28, 2011 | hide | past | favorite | 39 comments



Perl has #1, it's called "local". Any "our" global variable can be automatically overridden for anything in the current scope, and is automatically returned to its original value once the scope ends. One of my favorite uses is to store some concept of the "user's role" in such a variable, but when I temporarily need superuser access for some reason, I give myself a scope, "local" the superuser role, do my thing, close the scope, and don't spend a lot of time worrying about whether the user will get accidentally upgraded to a superuser.

I'm not sure any other language has the exact Lisp behavior for #3, but all the pieces in various combinations are certainly around. Perl actually has the ability to do some kooky leaping around its closures, though it requires some syntax:

   sub x {
       my $y = sub { goto TEST; };

       $y->();

       print "will not see\n";

       TEST: print "here we are\n";
    }

    x();
Which is not exactly what was described, but can be used in some of the places where you'd use that.


Lua gets closures right. The implementation (http://www.lua.org/doc/jucs05.pdf, pgs. 8-10) is a bit novel - essentially, closures are stack-allocated, but migrated to the heap if still active when the function returns. (This is partially a trade-off to keep compile-time analysis very brief; compiled Lua source is often used as a data serialization format.)

Lua also has support for calling code with a user-defined error handler before unwinding the stack (xpcall), but AFAICT there isn't a way to use it to resume where the error occurred. I tried making a library for restartable exception handling in Lua, but the complete infrastructure isn't there (yet). Maybe a "power patch"...


Perl has #1, it's called "local". Any "our" global variable can be automatically overridden...

And not just our variables but also subroutine calls can be localised in the scope:

    sub bar { 'bar' }
    sub baz { bar() }

    sub foo {
        no warnings 'redefine';
        local *bar = sub { 'My Bar' };
        baz();
    }

    say foo();   # => 'My Bar'
    say bar();   # => 'bar'
    say baz();   # => 'bar'
Thus you have localised monkey patching.


Once you have dynamically scoped variables, you can implement dynamically scoped functions. There's a really great paper by Pascal Costanza showing how to to this, and why it's the mechanism behind aspect-oriented programming (no, it's not just monkey-patching!): http://www.scribd.com/doc/47747298/Dynamically-Scoped-Functi...

You can generalize this to dynamically scoped object slots or whatever else you want, which Pascal does in the ContextL library: http://common-lisp.net/project/closer/contextl.html

Actually, you don't even need built-in dynamic scoping, you can emulate it using unwind-protect (try-finally), which can itself be emulated: http://home.pipeline.com/~hbaker1/MetaCircular.html

Once you have all that, you can implement Common Lisp's condition system. Here's Red Daly's implementation for JavaScript (Parenscript): https://github.com/gonzojive/paren-psos/commit/6578ad223515d...

Of course, none of this is very useful without macros. For the condition system, you also have to be careful because any intermediate function that doesn't follow the protocol might swallow the exception.

The reason you want dynamic scoping is because it completely obviates the need for dependency injection or the service locator pattern. DI is IMO by far the stupidest fad in object-oriented programming, which is no small feat.

The Common Lisp condition system is great for doing things like writing network libraries. For example, an HTTP client library can throw any exception, like timeout, transfer error, encoding error, whatever, and offer a restart to let your retry the request (in just one place, independent of the exceptions). If you just want to retry but don't care why the thing failed, you do that. If you don't care about retrying, you just catch the error and report it, or whatever you want.

The other thing about the Common Lisp condition system is that it provides introspection on conditions and restarts. This means I can do things like offer correct condition support for programs running across thread (and, eventually, network) boundaries transparently in Eager Future2 (http://common-lisp.net/project/eager-future/), in a portable way. By contrast, it's going to be impossible for C++ libraries using C++0x futures to be correct wrt exceptions without leaking their abstractions all over the calling code (if you don't wrap the exception, the information is not correct because who knows what thread/environment the future executed in, and if you do wrap it, you change the entire API).


And array/hash slices/elements, including the deletion of a key.


Clojure brings a couple other unique features to the table:

  * Pervasive use of performant persistent data structures.
  * Pervasive "time" (state/identity) management.
  * Dynamic type classes (w/o punting on the expression problem)
And in 2010 Clojure has something unique among other Lisps - it's core data structures are built on proper abstractions. They are extensible and you can even swap in your own implementations.

EDIT: modified the points to make them more specific. From what I can tell Haskell doesn't ship with persistent datastructures w/ Clojure's perf guarantees. And the concurrency stuff (STM etc) is tucked away in libraries that need to be imported. So pervasive here is key in the sense that these features are considered core languages primitives.


None of those are really unique, per se: Haskell has the first two, and several languages have all or almost all the benefits of the third by way of fully dynamic typing.

The first two are certainly very rare, though, and Clojure certainly has a different approach to the third than I'm used to seeing.


> And in 2010 Clojure has something unique among other Lisps - it's core data structures are built on proper abstractions. They are extensible and you can even swap in your own implementations.

To the extent this makes sense, it's equally true of Common Lisp.


> To the extent this makes sense, it's equally true of Common Lisp.

I'm afraid not. Christophe Rhodes has a proposal for an extensible sequence protocol, but AFAIK it is implemented only in SBCL. There's no portable way in CL to extend the built-in hash tables. And you certainly can't have your own cons class that responds to the built-in CAR and CDR.

Of course you can always shadow the builtin names and redefine them in your package as generic functions; this is what I did in my FSet functional collections library. But that's not quite the same thing as having extensibility designed in.


1. Special variables are pretty unique, but I think that's because people designing languages these days don't think they are a good idea. Debug flags seem to be the "killer app", as it were, and I do often wish I could have a special variable in another language for just that reason.

Of course, it's pretty easy to implement that functionality on top of C++ if you really want to. It's a little ugly, but it works.

2. I've never used that before, but it looks really nice. It seems pretty straightforward to implement in any language with full continuations, but such languages are sadly very rare.

3. That's kind of vague. He mentions mutating the state of the lexical closure, which doesn't appear to have any downsides, save potentially making a program harder to reason about and dangerous race conditions (say, with pmap). I'd be all for getting those in more languages.

The other thing he mentions looks like basically a multi-level return, which (while certainly a powerful feature) seems like a great way to break encapsulation. That sounds like one of those features that we might be better off having, but only if most people don't know about it. Fortunately, it's pretty straightforward to implement in terms of continuations (even the limited continuations some JVM languages have based on the throw/catch primitives).


Why not mention

- macros

- compiler macros

- reader macros

What other languages have these? What was done with compiler macros in cl-ppcre is something that doesn't seem possible anywhere else.


As was mentioned in the last Lisp macros thread, Forth's defining words. All of the flow control constructs and any words that use a unique syntax are defined through this mechanism.


I understand that Forth has things like reader macros.


1. Special variables seems really like a take it or leave it idea. I'm really not convinced that keeping these as explicit arguments of a function are a bad idea. I really like the ability to reason about a function by simply knowing its arguments (including the mplicit self/this).

2. Condition handling seems generally useful. There are definitely cases I could have used this. It's probably pretty straightforward to add to languages that have exceptions. Although I'd be surprisd to see it appear (pleasantly surprised).

3. I'm a little confused about this one... He's talking about just modifying the bound variables? C# and VB do this. And presumably Ruby and Python can too -- although he seems to allude to the fact that Python can't, which doesn't make sense (otherwise it seems odd to call them closures) -- or did I miss what he is really talking about (which I suspect I am)?


"1. Special variables seems really like a take it or leave it idea. I'm really not convinced that keeping these as explicit arguments of a function are a bad idea. I really like the ability to reason about a function by simply knowing its arguments (including the mplicit self/this)."

Sometimes it is nice to influence the entire call stack. The canonical example of this is probably re-directing "system out." Special variables provide a nice solution to the problem of making all the printlns in a call stack go to a file instead of the console or REPL, for example. That would be really awkward if you had to chain the System Out variable through all the calls.


> Condition handling seems generally useful. There are definitely cases I could have used this. It's probably pretty straightforward to add to languages that have exceptions.

Actually, it's not. In "normal" exceptions by the time an exception gets to you the computation that threw it is gone. The exception mechanism has to travel up the call stack to find the handler, and can't get back to where it was.

You can fake it in many languages, but you can't retroactively make the language support the behavior. You end up with a nasty mess that, although it may be powerful, is so far removed from the normal idioms of the language everyone you work with will want to kill you for doing it.


Actually, it's not. In "normal" exceptions by the time an exception gets to you the computation that threw it is gone.

This depends on how you model control flow. What's become more common now is to have very explicit control flow modeling so you have pseudo-edges from every place an exception can occur to the catch block (of course fewer with synchronous exceptions).

So when the exception is thrown you keep the frame on the stack, shove your IP, and go to the new handler -- like a function call. From the handler you have five potential exit points:

1) Pop stack, jmp... continue control flow from where the exception took place,if recovery succeeds.

2) Fall out the bottom if it was handled, but not "recovered".

3) Go to next recovery handler in list, if there is one.

4) Unwind.

5) New exception.

With that said, there are as many ways to implement exception handling as there EH mechanisms. Every one of mine I've done pretty differently, but in everyone of them I think could implement condition handling pretty easily.


The point I was making was more about the prospect of adding condition handling as a language user to an existing language. Basically that most language implementations make it impossible to add something like this using the tools provided by the language. I suppose at points go it was a bit of a silly one.

The difficulty in adding it to the language when modifying the language implementation is an open option is much lower.


OK, gotcha. I misunderstood. With that I agree.


>Special variables seems really like a take it or leave it idea. I'm really not convinced that keeping these as explicit arguments of a function are a bad idea. I really like the ability to reason about a function by simply knowing its arguments (including the mplicit self/this).

Haskell has an equally (arguably better) way of dealing with this but a lot of people find it a bit heavy weight.

When you compare it to OO languages, it becomes and obvious win. Any private data member of an object is an implicit/global variable to all methods of that class. If you have a stack trace in Java where one of those members seems to have an impossible value you have no way of knowing how it got there.

In Lisp this problem doesn't have to exist (ironically, I don't think class members in CLOS can be special, but I'm not certain). If you make your implicit variables special then any time that impossible value comes up and breaks the program you need only look in the stack trace. The culprit must be there because if it wasn't the bad value wouldn't be in effect.

Now, if you've ever programmed with the standard OO languages think about how much time that would have saved you.


Ruby has at least 1. and 3.

1. "usable global variables" exist in two forms; the rarely used $ sigil, as in $foo (automatically global), and the more commonly used hack of defining attributes and accessors on a class (for which there's standard macrology, cattr_accessor, in Rails).

3. Everything he mentions as properties of Lisp closures is commonly done with Ruby blocks (modifying variables in enclosing scopes; use of "return" in a block to return from the enclosing method).

That leaves the condition system. I'm not aware of another commonly used environment that lets you say, for instance, "if you were about to throw a FileNotFound exception, try doing this instead first". But unlike the other two, I'm not sure that's a feature of Lisp dialects in general, as Common Lisp in particular...


You're missing the point of 1. It's not simply "you can use globals", it's that you can rebind them in a thread-isolated way that stacks.

Ruby blocks aren't really first-class without getting converted into procs, and it's pretty un-idiomatic to pass these around like actual values. Even then they're not "first-class" as most calls happen via method calls rather than function invocation.


Smalltalk has restartable exceptions, but they're not as nice as CL's condition system.


What he describes in the condition system is also available in many implementations of Exceptions. Everything he describes has been available in Smalltalk for longer than a decade.


Dynamic variables can be done in Smalltalk via restartable exceptions but they aren't part of the language. Also, restartable exceptions are not nearly as nice as the condition system. That being said, Smalltalk is a beautiful language.


I'm not sure I get the global variables bit?

I know in Clojure you can rebind a var in a certain context with the 'binding' form... is that what they mean?


Yes. They're often called "special variables" or "dynamic variables" in a Lisp context - their value is checked within the current scope, not the (lexical) context in which they were defined.


I point out the value of them here: http://news.ycombinator.com/item?id=2152329


These are great features of Lisp. But I am going to make a bold statement and say: although they are extremely great features, you do not need any of them to build awesome applications.


"It would be great if some these ideas would be massaged into other languages."

Of course, this would be possible if they had macros!


Not really. The first has to do with how variable scope is managed. The second involves exception handling, and if your language unwinds the stack while looking for a handler, it's usually already too late.

The third has to do with how local variables are managed for functions defined at runtime - while you could rewrite code with macros to pass an implicit closure value array to the function at all points of use, it would be quite a bit less awkward to have it part of the language proper. Heck, it's not that unusual to have C functions that take a void * argument for "userdata"; the function/userdata combination is one way of representing closures when compiling to C.


Yeah I was waiting for someone to call me on this. I thought it might not be possible but wasn't sure. Mostly it was cool to see a "unique features of Lisp" blog post which didn't even mention macros.


I guess in theory you could do it using macros, but if the underlying language lacks certain features you end up with compiler-in-a-macro-style macros. I'm not sure you are really extending the language at that point...


Elisp does this; it has no language-level lexical scope, but you can get it using the lexical-let macro. From a functionality perspective it works surprisingly well, but if you look at the macroexpansion you'll go blind.


But isn't the definition of global at #1 a local thread closure captured variable?

I don't see why this is global...


Modifications to the global variable have local scope: "as long as we're within the body of this function (including function calls), modify stdout to this file buffer instead". It's distinct from global and lexical variables.


intersting, more so than most lisp is better articles.

like others here commented #1 sounds meh

#2 sounds awesome

#3 re Python (and intial mention that languages arent written in themselves) just sounds like author is not up to date with modern python.


#1 is definitely not meh, it's actually really practical if you're doing real work in Lisp. It seems meh if you're doing academic "basic CS concepts" exercises in Lisp, though.

> #3 re Python (and intial mention that languages arent written in themselves) just sounds like author is not up to date with modern python.

    Python 2.7.1 (r271:86832, Nov 27 2010, 18:30:46) [MSC v.1500 32 bit (Intel)] on win32
    Type "copyright", "credits" or "license()" for more information.
    
    >>> def foo():
    	x = 0
    	def bar(y):
    		x += y
    		return x
    	return bar
    
    >>> baz = foo()
    >>> baz(3)
    
    Traceback (most recent call last):
      File "<pyshell#10>", line 1, in <module>
        baz(3)
      File "<pyshell#7>", line 4, in bar
        x += y
    UnboundLocalError: local variable 'x' referenced before assignment
Is 2.7.1 not "modern" enough? I was under the impression that a lot of people are still on 2.7 rather than switching to Python 3. (I switched to Lua for scripting; it has working closures, lambdas, tail recursion, etc.)


the way original was worded it didn't sound like author was describing that particular wart, but, rather believing closures didn't exist. Generators, yield.

PyPy




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: