Why two namespaces in Lisp ¶

(Dated: 26 Sept 2023)

Emacs Lisp has a separate namespace for functions and variables, like Common Lisp, but different from Scheme which has a single namespace. By convention, programmers say Emacs Lisp is a Lisp-2, whilst Scheme is a Lisp-1.

Richard Gabriel and Kent Pitman wrote a paper, Technical Issues of Separation in Function Cells and Value Cells, 2001, which is interesting. (https://dreamsongs.com/Separation.html). Here I summarise the idea.

They have the following quote:-

There are two ways to look at the arguments regarding macros and namespaces. The first is that a single namespace is of fundamental importance, and therefore macros are problematic. The second is that macros are fundamental, and therefore a single namespace is problematic.

There is also an interesting related post by Kent Pitman in 12 Mar 2002 (https://groups.google.com/g/comp.lang.lisp/c/Bj8Hx6mZEYI.

I think the question of what makes a Lisp is taken historically by examining the torch passing. Lisp 1.5 was of course a Lisp because it was passed the torch by other Lisps. Emacs Lisp and Common Lisp are both Lisps because they both ascended from Maclisp with a substantial (even if divergent) desire to be true to the original Maclisp concepts and user base. But at design meetings of Scheme I have repeatedly seen the designers say that compatibility was not relevant. That, to me, says there is no allegiance to existing codebases and userbases and there is more desire to build a new community. That, to me, is the very essence of the launch of a new language–the breaking of essential links with the past. They did it voluntarily, the people outside their community did not do it to them.

Back to the discussion of two namespaces. There are two namespaces, call them the value namespace and the function namespace.

Macros and name collisions: A macro expands into its lexical environment. There is the potential for name collisions if the expansion of a name results in a different interpretation.

Here is an example of macro expansion that works (Source: Emacs subr.el, simplified):-

(defmacro when (cond &rest body)
  "If COND yields non-nil, do BODY, else return nil.
When COND yields non-nil, eval BODY forms sequentially and return
value of last one, or nil if there are none."
  (list 'if cond (cons 'progn body)))

An example use of this macro is below:-

(when t
  (print "hello"))

I can check the macro expansion the following way. This is created using the function macrostep-expand (in the package macrostep), bound for example to C-c e e.

(if t
    (progn
      (print "hello")))

Why does the macro expansion typically work? Why do the names generated by the macro expansion not collide with the local lexical environment? How does an example arise of problematic expansion where name conflicts can happen? One explanation is related to local names. There is a counterpart to let, but for functions instead of values.

For example, cl-flet is like let but for functions:

(cl-flet ((sq (x)
	    (* x x)))
  (sq 4))

This means that if you write a macro that expands to a given identifier, potentially that identifier could be a function that is defined by flet as opposed to a general function as you may expect.

For example, if you write a macro expanding into a function list you can rely on the fact that it really is the built-in list as opposed to something redefined by the use of flet.

Historically, in MacLisp, the facilities for flet, labels, macrolet did not exist. These are the ones that create the risk of conflict. If you minimise the use of such constructs, you run into less risk of conflict. Because Emacs Lisp is a descendant of MacLisp, such facilities are used less.

Because there are two namespaces, the names of built-in functions are thus preserved.

If there was a single namespace, the macro might expand into a symbol f that, as a value could stand for something else that a given default.

Programmers are more careful about naming functions than naming values. Hence the function names are more predictable than value names.

Example expansion:

;; macro with predictable expansion
(defmacro my-macro (x)
  "Example expansion into lexical environment."
  `(list ,x y))

Use it like this:

(my-macro 2)

This expands to the following; list almost certainly refers to the function, whilst y could be something else in that context

(list 2 y)

How to use flet for debugging. There is actually a use case for flet. For example, redefine a function only locally. This can be useful for debugging or experimenting. This is essentially a form of hook:

(cl-flet ((message (x)
	    (* x x)))
  (message 2))

Conclusion: Fundamentally, there are 2 namespaces because Lisp programmers typically create local values with let, and infrequently local functions with flet or cl-flet. Thus one of the namespaces, usually the functions, is thus reliable for macro writers to design macro expansions for. (The same argument with infrequently creating local values is impractical)

Hence the philosophy that ’macros are fundamental’ naturally leads to the requirement of having two namespaces.