Emacs Lisp as a general purpose language?

I live in Emacs

As you may already know if you read this blog (yes, you, the only one who reads this blog …) I live inside Emacs.

I have been an Emacs user for more than 12 years now and I do nearly everything with it:

  • org-mode for organization and document writing (thanks to the export functionalities),
  • gnus for e-mail (I used to use VM before),
  • ERC and identica-mode for instant messaging and micro-blogging (things that I don’t do much anymore),
  • this very post is written in Emacs and automatically uploaded to my blog using the amazing org2blog package,
  • dired as a file manager,
  • and of course all my programming tasks.

This has been a progressive shift from just using Emacs as a text editor to using it nearly as an operating system. An operating system with a very decent editor, despite of what some may claim!.

It is not a matter of totalitarism or aesthetics, but a matter of efficiency: the keyboard is the most efficient user input method for me, so being able to use the same keyboard shortcuts for programming, writing e-mails, etc. is very useful.



Looking for a Lisp

In a previous post I wrote about my thoughts on choosing a programming language for a particular project at work. My main criteria for the choice were:

  • availability of a REPL;
  • easy OTB access;
  • integrated concurrent programming constructs.

I finally chose Clojure, a language that had been on my radar for nearly a year by then. I had already started to learn the language and made some tests to access OTB Java bindings and I was satisfied with it.

Since sometimes life happens, and in between there is real work too, I haven’t really had the time to produce lots of code with it, and by now, I have only used it for things I would had done in Python before. That is, some glue code to access OTB, a PostGIS data base, and things like that.

I also feel some kind of psychological barrier to commit to Clojure as a main language (more on this below). Added to that, has also been the recent evolution of the OTB application framework, which is very easy to use from the command line.

In the last days of my vacation time, the thoughts about the choice of Clojure came back to me. I think that everything was triggered by a conversation with Tisham Dhar about his recent use of Scheme. It happened during IGARSS 2012 in Munich, and despite (or maybe thanks to?) the Bavarian beer I was analyzing pros and cons again.

Actually, before starting to look at Clojure, when I was re-training myself about production systems, I had thought about GNU Guile (a particular implementation of Scheme) as an extension language for OTB applications, but finally gave up, since I thought that going the other way around was better (having the rule-based system around the image processing bits).

When I thought that I was again settled with my choice of Clojure, I stumbled upon the announcement of elnode‘s latest version. Elnode is a web server written in Emacs Lisp.

I then spent some hours reading Nic Ferrier’s blog (the author of Elnode) and I saw that he advocates for the use of Emacs Lisp as a general purpose programming language. Elnode itself is a demonstration of this, of course.

Therefore, I am forced to consider Emacs Lisp as an alternative.



But what are the real needs?

  1. Availability of a REPL: well, Emacs not only has a REPL (M-x ielm), but any Emacs lisp code in any buffer can be evaluated on the fly (C-j on elisp buffers or C-x C-e in any other buffer). Actually, you can see Emacs, not as an editor, but as some sort of Smalltalk-like environment for Lisp which gets to be enriched and modified on the fly. So this need is more than fulfilled.
  2. Easy access to OTB: well, there are no bindings for Emacs lisp, but given the fact that the OTB applications are available on the command line, they can be easily wrapped with system calls which can even be asynchronous. The use of the querying facilities of the OTB application engine, should make easy this wrapping. So the solution is not so straightforward as using the Java bindings on Clojure, but seems feasible without a huge effort.
  3. Concurrency: this one hurts, since Emacs lisp does not have threads, so I don’t think it is possible to do concurrency at the Lisp level. However, since much of the heavy processing is done at the OTB level, OTB multi-threading capabilities allow to cope with this.

For those who think that Emacs Lisp is only useful for text manipulation, a look at the Common Lisp extensions and to the object oriented CLOS-compatible library EIEIO may be useful.



So why not Emacs lisp then?

The main problem I see for the use of Emacs Lisp is that I have been unable to find examples of relatively large programs using it for other things than extending Emacs.

There are of course huge tools like org-mode and Gnus, but they have been designed to run interactively inside Emacs.

There are other more technical limitations of Emacs Lisp which can make the development of large applications difficult, as for instance the lack of namespaces, and other little things like that.

However, there are very cool things that could be done to make OTB development easier using Emacs.

For instance, one thing which can be tedious when testing OTB applications on the command line is setting the appropriate parameters or even remembering the list of the available ones. It would nice to have an Emacs mode able to query the application registry and auto-complete or suggest the available options.

This could consist of a set of Emacs Lisp functions using the comint mode to interact with the shell and the OTB application framework. This would also allow to parse the documentation strings for the different applications and present them to the user in a different buffer.

Once we have that, it should be possible to build a set of macros to set-up a pipeline of applications and generate an Emacs Lisp source file to be run later on (within another instance of Emacs or as an asynchronous process).

This would build a development environment for application users as opposed to a development environment for application developers.

And since Lisp code is data, the generates Emacs Lisp code, could be easily read again and used as a staring point for other programs, but better than that, a graphical representation of the pipeline could be generated (think of Tikz or GraphViz).

Yes I know, talk is cheap, show me the code, etc. I agree.




Clojure has a thriving community and new libraries appear every week to address very difficult problems (core.logic, mimir, etc.). Added to that, all the Java libraries out there are at your fingertips. And this is very difficult to beat.

I hear the criticisms about Clojure: it’s just a hype, it’s mainly promoted by a company, etc.

Yes, I would like to be really cool and use Scheme or Haskell, but pragmatism counts. And, anyway, the programming language you are thinking of is not good enough either.