I suppose I *could* whip up Yet Another Blogging System using Gauche, but yeah… you can only write the same software so many times. ;-)

I was also trying to hack on something else in Gauche, but life kept interfering…

(( Gauche Scheme is seriously underrated, by the way. Although it’s an interpreter, it’s pretty fast; in an informal benchmark that I did just for the hell of it, it beat Chicken, Gambit, Scheme48 and Racket. Oh, and Python. ;-) I noticed this before, when it processed SXML much faster than *compiled* Chicken code. Also, it comes with a bunch of libraries baked in, and has a decent Windows version. What’s not to like? Well I do have a list of ideas for improvement actually, maybe someday I’ll blog about it… ))

Anyway, the plan is now, to raise ~$2000 so I can buy an iPad and a Mac that can actually be used to develop iOS software. Hence the renewed selling of stuff… As usual, I am also available for odd jobs and part-time work.

As always, comments and corrections are welcome. ^_^

IMHO, Chicken and Gauche are the two most desirable Scheme implementations at the moment. They both have a pragmatic approach that isn’t always present in other implementations. (Although I haven’t tried every single one of them…)

• It’s cross-platform (Unixoid systems and Windows)
• Builds out of the box on Mac OS X, even on my old G3
• Supports both hygienic and “dirty” macros
• Comes with an object system
• Comes with a bunch of libraries out of the box (like a HTTP library, which is one of the things I needed)
• Works with SLIB; in fact, if SLIB is already installed, it will find it and integrate it
• Supports a large number of SRFIs

Chicken has most of this too, except it lacks Windows binaries, and it recently got rid of unhygienic macros. Libraries can be installed as eggs, of course, so it might not need SLIB.

Anyway, said “pet project” involves making HTTP GET calls, and requires a powerful object system. (More about this later, if it ever becomes usable.) I wanted to use a Lisp/Scheme dialect for flexibility, and I wanted it to be cross-platform if at all possible, so users can run it on Windows as well. It turned out that it was remarkably hard to find a Scheme that matched all these requirements. I will not go into all the problems that I encountered, but suffice to say that Gambit, Scheme48, PLT and Petite Chez Scheme all had their own issues. (That is not to say that I won’t use them for other projects…)

That said, I might still change my mind and use Common Lisp instead… :-)

Then again, from what I have seen, this attitude is less prevalent in the Scheme community…

> (+ 1 (call/cc (lambda (k) (+ 3 (k 2)))))
=> 3

Continuations are hairy like Fuzzyman. ;-) So if you don’t understand the example above, don’t worry, they take a while to wrap your head around. Someday I’ll write a “continuations in 60 seconds” post (or maybe a bit more time will be needed :-); until then, this explanation might help.

It looks pretty good; it’s certainly more advanced than my attempt at a Scheme interpreter in Python (which is, in turn, more advanced than Psyche, and will be released eventually, once I backport some of the dollop code to it). Apparently it supports (among other things) hygienic macros and continuations, features that haven’t yet made it into my interpreter.

On a side note, it’s interesting to see how different the Ruby code looks compared to Python code. Five years ago I would have said (and did say :-) that Python and Ruby are much more alike than unalike. They still are, but they surely encourage (and discourage) different programming practices and idioms.

• To evaluate an expression, call eval on it.
• If the expression is atomic, this is just a matter of a simple name lookup, or the expression evaluates to itself (in the case of literals).
• If the expression is composite (i.e. a list), it is considered to be a function call, where the first element of the list is a function, and the other elements are the arguments passed to that function. We evaluate all of these elements (again by calling eval on them), then call apply to do the actual function application.
• If said function is built-in, we just call it, get a result back, and be done. If it’s a user-defined function, then we need to evaluate the expressions in that function’s body, using the procedure described here.

This is a simplified version — it doesn’t take into account special forms or macros, for example — but it does capture the essence of the eval-apply cycle.

Anyway, since expressions can be nested arbitrarily deep, it should be clear that eval and apply can and often do call themselves and each other.

Now, when writing a Scheme interpreter in a language like Python, it is tempting to use a straightforward translation of this principle. You write functions (or methods, if you wish) eval() and apply(), which may call themselves and each other. This works rather well — until you blow the stack. The implementation language’s stack, that is… because every call to eval or apply in the Scheme interpreter, also uses the implementation language’s call stack.

This is a problem, especially in Scheme, which relies on tail recursion to define “iterative” processes. (These still use recursion, but because of tail call optimization, they operate in constant stack space. See SICP 1.2.1.)

We can simulate tail recursion by using exceptions (as demonstrated by this Cookbook recipe, for example). This is the approach I followed in most of my projects. While it does work, sooner or later you’ll run into other problems, because the interpreter is still piggybacking on Python’s call stack, rather than having one of its own. So, implementing continuations, for example, becomes very difficult, as the notion of “the calling expression” is implicit.

So, I decided to use a different approach. And now we’re finally getting to the point of this blog post. :-)

This time I started with a call stack. The expression we want to evaluate is pushed onto it. For example,

(+ 1 a)

Then, I wrote a method run() which does one “evaluation step” using this call stack. If it’s done evaluating, it returns the result value, otherwise it returns None, as an indication that there’s still more work to do. (So, in order to evaluate an expression, you call it multiple times until you get a result back.)

How does such an evaluation step work? We take the topmost expression on the call stack, and start evaluating it. In this case, it’s a composite expression, so we just start at the beginning. We want to evaluate the symbol +. To do so, we push + onto the stack, and put a placeholder in its original position, indicated by “$$”.

;; call stack now looks like:
($$ 1 a) +

We’re now done with this step. In the next step, we again look at the topmost expression on the stack. It’s a symbol; we look it up, get a function back (represented by <lambda:+>, and stick it back into the original expression. We then move on to the next element of the list, which is 1, and push it.

(<lambda:+> $$ a) 1

Another step, and we get:

(<lambda:+> 1 $$) a

Let’s say the variable a has been defined and its value is 4. We then get:

(<lambda:+> 1 4)

Now we’re done evaluating the elements of this expression, and we can do the actual function application. Finally, we get the result 5.

This gets more complex when we use nested expressions, but the principle is the same.

(+ (* a (- c d)))
($$ (* a (- c d))) +
(<lambda:+> $$) (* a (- c d))
(<lambda:+> $$) ($$ a (- c d)) *
(<lambda:+> $$) (<lambda:*> $$ (- c d)) a
(<lambda:+> $$) (<lambda:*> 4 $$) (- c d)
;; ...etc...

It becomes more interesting when we introduce special forms into the mix. But since we have an explicit call stack now, we can clearly see what is going on with it, and tail recursion starts making much more sense. For example, if our topmost expression is

(if condition (do-this) (do-that))

and during evaluation we determine that condition evaluates to true, we can at that point replace the whole expression by

(do-this)

rather than having to push (do-this) and then sticking the result back into the if expression. Ditto for the last expression inside a begin block, and so on.

The special forms require a bit of hackery, since we don’t evaluate all of their arguments… but there’s only a few of them.

The proof-of-concept implementation that uses this concept is called dollop and is available at github. (Requires Python 3.0.) The name is because it’s only a “dollop of Lisp” (or rather, Scheme); it only supports a few special forms (begin, define, if, lambda), and a few functions for example programs (+, -, *, =, list). It cuts corners in other ways as well, as my goal was to get a working proof-of-concept out, not to write a complete Scheme interpreter.

Anyway, comments, bug fixes, improvements, etc, welcome.

(I will also release two other Lisp-y interpreters, but those will get blog posts and project pages of their own…)

Name: Hal Abelson
[...]

The voter indicated that he would like to submit the first chapter of Structure and Interpretation of Computer Programs as his statement. The Steering Committee agrees that this does fulfill the requirement, given that the voter did write it, and it is more that 75 words long. Unfortunately we don’t have the space to reproduce it here, but it can be found online at http://mitpress.mit.edu/sicp/.

(For non-Schemers: To register as a voter, people must submit a statement indicating their interest in Scheme and the standardization process. Most people’s submissions look like, “I have been using Scheme for X years blah blah blah…” I guess if you wrote SICP, you can afford to do it differently. :-)

*ponders*