Drinkable Chicken

In other news, there is a new documentation server for Chicken Scheme online. It’s called Chickadee and it looks pretty damn useful. You can look up built-in functions, SRFIs and eggs. I’m currently working on a little project that uses Chicken, so I guess I’ll be using this a lot. ^_^

IMHO, Chicken and Gauche are the two most desirable Scheme implementations at the moment. They both have a pragmatic approach that isn’t always present in other implementations. (Although I haven’t tried every single one of them…)

I did some long overdue Chicken hackery yesterday, and by accident I found out that Chicken’s (or rather, SRFI-26‘s) cut/cute macros are not the same as Arc’s [ ] syntax after all.

Scheme/Arc aficionados already knew this, of course, but to me it was news since I’ve never really used cut much. Here’s a short explanation.

Scheme’s cut produces an anonymous function that takes as many arguments as there are <> symbols at the “top level”. E.g.

> (cut + 1 <>)
#<procedure (? g3)>
; has one argument

> (cut + 1 <> <>)
#<procedure (? g4 g5)>
; has two arguments

; etc...

However, any <> that is found in a nested expression, is not considered as a parameter. Therefore:

;this works...
> (map (cut + 1 <>) '(1 2 3))
(2 3 4)

; but this does not:
> (map (cut + 1 (* 2 <>)) '(1 2 3))
Error: bad argument count - received 1 but expected 0: #<procedure (?)>

By contrast, Arc’s [ ] syntax produces an anonymous function that expects one and only one argument, which is represented by a single underscore. As a result, it’s simultaneously more and less limited than cut:

arc> (map [+ 1 _] '(1 2 3))
(2 3 4)
arc> (map [+ 1 (* 2 _)] '(1 2 3))
(3 5 7)
; no problem

; but two parameters doesn't work:
arc> (map [cons _ _] '(1 2 3) '(4 5 6))
Error: "#<procedure>: expects 1 argument, given 2: 1 4"

Of course, for anonymous functions that are more complex than this, it’s probably better to just use lambda… :-/ Or a named function.

This started out as a Chicken blog, where I would report my adventures with Chicken Scheme. But these days it’s more like a Python blog again. (Which isn’t too surprising, given my experience with the language, and the fact that I use it for work and most of my personal projects.)

Heck, it has even been added to Planet Python again, which wouldn’t have been the case if I wrote about Scheme all the time.

But really, I want to learn more about Scheme as well, *and* be able to use it in a pragmatic way. I just need a bit of focus. Right now I’m wondering what to look at next. Maybe object systems?

I’ve been hacking on and off on s1, my vaporware Awk-like tool that uses Scheme for scripting. The “language” defined by s1 includes a few macros. Strictly speaking, these are redundant, but they do make things shorter, which is important for writing one-liners.

Anyway, I am still kind of new to macros… so I’m wondering if the code I wrote is sound. I’m offering the definitions here for review. (Comments from experienced Schemers are welcome.)

1. inc! is used to easily increase variables. It can take any number of arguments; if none are given, 1 is added by default. For example:

• (inc! x) adds 1 to x
• (inc! x 33) adds 33 to x
• (inc! x 1 2 3 4 5) adds (+ 1 2 3 4 5) to x

As the exclamation mark indicates, inc! changes the variable in-place. Here’s its current definition:

(define-macro (inc! name . args)
  (let ((total (gensym)))
    `(let ((,total
             (if (null? (list ,@args))
                 1
                 (apply + (list ,@args)))))
       (set! ,name (+ ,name ,total)))))

2. Defining multiple variables does not mix well with one-liners. So I added a macro def that allows one to define them quickly and concisely. Like inc!, def takes any number of arguments. If an argument is a list (name value), then a variable is created with the given name and value. If an argument is a symbol, then a variable is created with that name and a value of 0. (Awk is often used to add numbers, so zero seems the most sensible default, IMHO.)

Examples:

• (def x) — same as (define x 0)
• (def x y) — same as (define x 0) (define y 0)
• (def (a 1) (b “hello”)) — same as (define a 1) (define b “hello”)
• (def q (w 3)) — same as (define q 0) (define w 3)

Here’s the current definition:

(define-macro (def . args)
  (if (null? args)
      #f
      (let* ((a (car args))
             (rest-args (cdr args))
             (name (if (list? a) (car a) a))
             (value (if (list? a) (cadr a) 0)))
        `(begin
           (define ,name ,value)
           (def ,@rest-args)))))

(I’m not sure about the #f; it’s not supposed to return a value anyway.)

In any case, s1‘s auxiliary functions and macros allow for concise code. (Some of it is sloppy, but useful for “scripting”, especially one-liners. Naturally, it’s always possible to write longer scripts using “cleaner” code.)

For example, here’s a one-liner that takes the last words on the given lines and adds them up (assuming they are numbers):

s1 '(B (def s)) (inc! s &$nf) (A (out s))'

(I’m not sure about the & syntax yet; it’s used here as a shortcut for the as-number function, which attempts to convert a string to a number, returning 0 by default.)

B and A are shorthands for BEFORE and AFTER, blocks that are executed before and after the main code (which is executed for each line in the given text). The actual order in which these appear doesn’t matter, but it’s probably more intuitive to do before-main-after.

Print the number of lines, words and characters (like wc):

s1 '(B (def c w)) (inc! w nf) (inc! c (len $0) 1) (A (out nl w c))'

Print names starting with “Ga”:

s1 '(if (~ #/^Ga/) (print $0))' /usr/share/dict/propernames

(I’m using regex literals, and ~ is the same as string-search, except it matches against $0 (the whole line) by default.)

These are just teasers. Actual code is subject to change. I will release this when the “API” is somewhat stable. It’s still mostly a toy, though… :-)

Looking for eggs that handle regular expressions, I found scsh-regexp. It implements most of the regexp API, as described in “Proposed SRE regular-expression notation“.

Now, I’m not particularly interested in the API, as Chicken already supports regular expressions out of the box (see the regex unit). What I find much more interesting is the part of the proposal that *isn’t* implemented. It also defines a syntax for describing regular expressions. This part isn’t available in the egg; according to scsh-regexp‘s documentation, it was only implemented in SCSH.

Many questions arise. Why wasn’t this implemented elsewhere? Would it be useful (i.e. compared to usual regex notation)? Would it be hard to port to Chicken? Would it be hard to write from scratch, going by the documentation only?

Most likely, I’m not up to the task, but that doesn’t stop me from dreaming about things like a sed-like stream editor using this kind of syntax for regular expressions. It would be super verbose, but also more readable… at least, that is the assumption.

More efforts to make s1 code shorter…

I added/changed the following:

• slice function (this will come in handy for string manipulation, although (slice s a b) is still longer than s[a:b]
• nf variable (indicates the number of fields in a line)
• $ now accepts any expression, not just a number literal (so we can say $nf or $(- x 1) or whatever)
• -f command line option to quickly set the field separator
• created an alias s1 for ‘csi -ss /path/to/s1.scm’ (OK, this is not a change to the tool proper, but it helps… nobody wants to type csi -ss with the full path, all the time)

None of this is particularly original (as it’s all borrowed from Awk and Python), but it does help to make code much shorter. I can now write a one-liner like this:

find "${1-.}" -type d | s1 -f/ '(print "   |" (make-string nf #\-) $nf)'

…which produces almost the same tree as described here. Almost, because it uses one “-” for each directory rather than two. We can fix that by making the example somewhat longer (and less readable to non-Schemers, although admittedly the Awk example isn’t very readable to non-Awkers either):

find "${1-.}" -type d | s1 -f/ '(print "   |" (make-string (* (- nf 1) 2) #\-) $nf)'

Next up: regular expressions…

Brainstorming a bit here…

So I saw this cool one-liner script. It displays a directory tree:

ls -R | grep ":$" | sed -e 's/:$//' -e 's/[^-][^\/]*\//--/g' -e 's/^/   /' -e 's/-/|/'

I have been hacking (on and off) on an Awk-like tool that uses Chicken for scripting. Its working title is s1, and so far it supports begin/main/end blocks like Awk, and $N-notation for quick access to fields.

The real problem is not, adding Awk-like functionality as Scheme functions, but rather, making the Scheme concise enough that it can be used for one-liners. For example, the following works, but is still too verbose:

ls -l | csi -ss s1.scm '(before (set! total 0)) (set! total (+ total (as-number $5))) (after (print total))'

(It shows the total size of files in the current directory, in case it wasn’t obvious. :-)

It gets even worse when we try to emulate the ls/grep/sed script mentioned above. Writing this as a one-line just isn’t feasible at this point; rather, I have to use a script:

;; tree.s1
;; Call with: ls -aR | csi -ss s1.scm -s tree.s1

(define (branch parts)
  (let* ((parts (reverse parts))
         (dirname (string-trim-right (first parts) #\:))
         (line (apply conc (map (lambda (s) "--") (cdr parts)))))
    (conc "|" line dirname)))

(when (string-suffix? ":" $0)
  (let* ((parts (string-split $0 "/")))
    (print "   " (branch parts))))

Much of the work goes into processing strings and lists, but even without that the code probably still would not fit on one line.

Granted, it’s hard to beat the conciseness of regular expressions (as used by grep and sed), but if I want s1 to be useful, it needs to support (much) shorter code. (Maybe I should rewrite it in Arc? ;-)

Here are some thoughts about making this shorter:

1. I dearly miss slicing. In Python, I would have used s[:-1] to chop off the trailing colon. Adding a slice function to the toolbox would probably be useful.

2. I could leverage some of the Awk-like features. This Reddit comment does just that:

find "${1-.}" -type d | awk -F/ '{printf "  |%*s%s\n",(NF-1)*2,"",$NF}'

It takes advantage of the following features:

• using “/” as a field separator (effectively splitting the path with no extra code)
• using NF (indicating the number of fields in a line) to compute the indentation level
• using $NF to display the last field (i.e. the directory name we want to see)

As it happens, s1 can do all these things as well, but it won’t be quite as concise. This is something to work on. Maybe there could be a special variable (or symbol) that translates to (length *fields*). I would have to hack the current $-literal to accept any expression, not just a number literal.

The tricky part is to make things concise while still keeping the spirit of Scheme. :-/ s1 will never automagically coerce numbers to strings (or vice versa), for example. I can, however, make it easy to convert them explicitly. (This is also the Pythoneer in me speaking… explicit is better than implicit, and all that good stuff.)

Anyway, there was really no point to this post, except letting the world know that I am tinkering with a new toy. :-) If it ever actually becomes useful, I’ll release it… Until then, I might talk more about it here.

Well, I managed to build SDL (and a number of related libraries) on Mac OS X… it was a bit of work, but eventually everything compiled. This page was a great help. (It’s aimed at Ruby users, but most of it still applies.)

I also built and installed the sdl egg. So far, so good.

The next question is: what do I do with it? :-) I’ve never used SDL before, and the official docs and tutorials all use C, which is different enough that I don’t know how to use it from Chicken.

If anybody has tips, or better yet, some simple example code to get me started, that would be very welcome. ^_^

So, I’ve been learning (Chicken) Scheme for over a month now. Let’s look back.

• I’m currently hacking on two projects in Chicken; one is a translation of an existing Python program, the other is something written from scratch. Neither will be very large. If they become somewhat useful/usable, I will probably make them available.

• I’m using vim, but looking for an editor that gets along better with Scheme (in terms of autoindent etc… I don’t like scheme.vim too much). Of course Emacs would be an obvious candidate, but I’m reluctant because of several reasons. (Less than stellar OS X support, unwillingness to learn many new keybindings and/or beat the editor into doing what I want, unwillingness to learn elisp at this point.)

• Even though succinctness is power, so far my Scheme code is almost twice as long as equivalent Python code. ¹) I’m not yet drawing any conclusions based on this; there are many possible explanations, like the fact that I am still learning and therefore tend to use more verbose code (for clarity, or simply because I don’t know a shorter idiom), while my Python code does not suffer from the same problem. So, we’ll just have to see if this changes over time.

• On the other hand, my Scheme code seems to be cleaner than the Python I had. Scheme really encourages rigorously splitting up code into smaller functions. (This is probably true for most, if not all, functional languages, by the way.)

• Chicken has been a joy to program in. It has a lot of useful functions out of the box. I do find that you don’t get very far if you only have vague ideas about what your code should do. :-) Functional languages seem to have this feature (?) more than, for example, Python or Ruby.

• I am impressed with the availability of Chicken libraries (eggs) so far. Granted, I haven’t needed very complex stuff so far (testing framework, command line argument parsing, etc), but what I did need was there, often in several flavors. Installation is stupid easy too.

When my initial projects are done, I plan to look at stuff that is a little more “real world”, like game development (using SDL?), OS X hacking, maybe web programming. (I also would like to contribute an egg at some point — but I need to write useful code first! :-)

¹) I’m just looking at file sizes here. Maybe I’ll do a more meaningful comparison later, when I have more code to compare.

More useless reader macros ahead. This time, let’s implement Arc’s [ ... _ ... ] lambda shortcut. In Chicken Scheme, of course. ^_^

I will be using { } rather than [ ]. Incidentally, Chicken already supports { } as an alternative for ( ):

> '{1 2 3}
1 2 3)
> {+ 1 2}
3

This reader macro will override that. Anyway, the code follows. It’s clumsy (although not as clumsy as my first version), and there’s probably a better/shorter way to do it (bit like installing the latest O2 UK Mobile Broadband after struggling through the 90s with 56k), that I’m not aware of yet. In any case, it’s naive; for example, this version does not allow nesting the { } constructs. Also, the replace-underscore function won’t replace underscores in nested lists. ¹) And so on. Remember that this is just for demonstration purposes before pointing out the zillions of flaws. :-)

(define (read-until-next-curly port)
  "clumsy way to read up until the first }."
  (read-token (lambda (c) (not (char=? c #\}))) port))

(define (replace-underscore exprlist)
  (define (replace-underscore-item y)
    (if (equal? y '_) 'x y))
  (map replace-underscore-item exprlist))

(define (make-lambda-form exprlist)
  `(lambda (x) (,@exprlist)))

(define (eval-string s)
  (with-input-from-string s read))

(define (transform-into-lambda s)
  (let* ((expr (eval-string s))
         (expr2 (replace-underscore expr)))
    (make-lambda-form expr2)))

(set-read-syntax! #\{
  (lambda (port)
    (let* ((s (read-until-next-curly port))
           (t (string-append "(" s ")")))
      (read-byte port) ; read trailing '}'
      (transform-into-lambda t))))

Testing…

(print { + _ 1 })     ; => #<procedure (? x)
(print ({+ _ 1} 44))  ; => 45

(print (map { * _ 2 } '(1 2 3 4 5 6)))
; => (2 4 6 8 10 12)

Cool. Of course, if you use a lot of small lambdas, then it’s possible to write a macro that shortens things for you, rather than having to resort to syntax changes. For example:

> (define-macro (fn . body)
>   `(lambda (_) ,body))
> (fn + _ 1)
#<procedure (? _)>
> ((fn + _ 1) 44)
45

;; compare:
> ((lambda (x) (+ x 1)) 44)
45

Anyway, much like the previous post, this was just an excuse to explore reader macros a bit more. So don’t shoot me…

¹) On a side note, why doesn’t SRFI-1 have a function to replace items in a list?