Drinkable Chicken

(I decided to limit myself to one rant a year. This is it.)

Just saw this blog post (via Hacker News): Dear Python. In it, the author lists a number of complaints he has about Python.

There is so much wrong with this article that it’s like shooting fish in a barrel.

First of all, the post seems to imply that Python started out great, but then added inferior copies of features found in other languages:

But then you got jealous of the big players. You saw that others had things you didn’t have and you wanted to compete in a game that you were never destined to play. And who could blame you? The megaliths in the crowd like Java and c++ were beginning to overshadow the small guys and were even starting to overtake some of the oldest and wisest players: c, Fortran, lisp. What did they have in common? Objects.

Objects have been around since the very first public release of Python (0.9.1). They were not added because of “object envy”. Hell, Java didn’t even exist when Python first came out.

But you weren’t ecstatic about objects. You didn’t really like them and you didn’t trust them. So you decided to only give them a little. You cut corners. You made objects but you made them hard to use. That was a mistake.

That’s another odd argument… IMHO, Python’s objects are *much* easier to use than, say, Java’s, or C++’s, where you have to micromanage access (public? private? protected?), write (often superfluous) getters and setters, and deal with such niceties as as “final” classes, checked exceptions, abstract classes, interfaces, etc. In Python, I can just write the class, stick in it whatever I want, *use* it however I want, and move on.

There is no encapsulation. There are three tiers to OO. One of them is encapsulation. Having OO without encapsulation is about as nice as sawing a leg off a three legged stool. I can hear you yelling, “That third leg is completely unnecessary. We’re all consenting adults here. Just balance, like this…” *crash*.

I am assuming that by “encapsulation” the author means “data hiding”. Python’s philosophy here is that in order to separate interface from implementation, data hiding is not necessary. That’s where the “consenting adults” idea comes from; you are *supposed* to talk to the object using its interface (methods or properties, usually), but if you really have to, you can easily access any attribute. There are ways to make this harder, but (again, IMHO) they are not very Pythonic.

Also, the notion of “three tiers to OO” really depend on which definition you go by. Java, C++, C# and their ilk are neither the first word in OO, nor the last. Languages like Smalltalk, Self and Io have (sometimes radically) different ideas about what objects are and how they are handled.

Then, libraries:

However, these libraries are inherently static. This means that if the library doesn’t do what you want it to, you are screwed. There is no diagnosing or fixing errors in the library. If they happen, pack up and try a different language.

I’m not sure in what sense Python’s standard library is “static”… there are a few built-in modules written in C, but most of them are in pure Python, and the source code is readily available. You could change them if you must, but a better approach would be to subclass them and replace the offending features/bugs. (Note that there are no “final” classes and private attributes to get in your way here. :-)

(The httplib problem mentioned looks like it can be solved by setting a proper timeout value, by the way. I’ve written several web crawlers in Python, and never encountered this issue.)

Then there’s the tired old “explicit self” argument again:

[...] But sadly, this is exactly how python is. It doesn’t really take care of anything for you. You have to have “this” as the first argument to every function in a class. Once in a function, you have to use “this” to access any variables or other functions.

I’ve written about this before (in 2003!) so I’ll just links to those posts here, rather than repeating it:

• The importance of being selfish
• The importance of being selfish, part 2

And here’s what Guido himself has to say about it: Why explicit self has to stay.

The Java way is better. The implied this is easy to understand. Some IDE’s even highlight the class variables with a different color than the local variables, allowing an easy distinction. In the end you save tons of characters by having an implied “this”.

The implied this (in a method’s argument list) may be easier, but Java gets away with this because it doesn’t have actual functions. Python, as a multi-paradigm (god I hate that word :) language, does, and in fact methods are really just functions that you stick in a class.

Also, the “tons of characters” you save don’t mean much, when a typical Java method definition starts with stuff like “public static final void …”!

To add insult to injury, python has no explicit support for member definition. A class’s members are created by setting them equal to something. There is no way to explicitly define them in the class definition. Anybody, anywhere, can add (or remove) members variables from a given object. Classes in Python are more like dictionary objects with functions attached than classes.

Um, yes, the lack of declarations is one of the factors that makes Python code shorter and more flexible. And indeed a class (or an object) is much like a dictionary with some special rules added. Hint: the same is true for languages like C++ and Java, except they hide that dictionary behind a ton of cruft, so you won’t notice it as much.

I agree that removing attributes from a class or object generally isn’t a good idea, but then again this doesn’t happen a lot, unless in special cases.

After much frustration at using your broken OO, I decided to give it a rest. Maybe python would make more sense if I just treated it as a flat procedural language. Then I noticed lambdas. [...] They can only be one line.

Agreed, Python’s lambda sucks compared to functional languages (and even Ruby). However, it was intended as a shorthand for simple functions, not as a full-fledged alternative to def.

Python, you are just like the little kid Peter. You see something others have, decide you must have it, and then you just stick it somewhere. To become a better language, you must outgrow this little kid copy cat phase.

If you want objects, implement objects all the way. This includes things like encapsulation and variable definitions. But if you don’t want objects, then don’t implement them at all. It is no help for anybody when you implement something that you don’t want people to use.

This really makes no sense. “Implement OO like Java, or don’t implement it at all”. Gee, go tell that to the authors of Smalltalk, Io, Self, CLOS, Simula, OCaml, even Ruby. Again, what “object oriented” means isn’t written in stone, and Java’s implementation of it certainly isn’t the gold standard. I don’t think the original inventors of OO intended the language to get in your way constantly, for starters.

~

Granted, Python has its problems. Over the years, these have become more clear to me. I am mostly infatuated with ultra-dynamic and/or functional languages at the moment (but that is a different story…). I don’t like the join method on strings, the fact that we have classmethods/staticmethods/etc now and all that drivel, and I would have liked the language to be more flexible, for example. But all of these criticisms are fairly minor. Python is still the language I am most comfortable with, and the language I reach for first in many cases.

So, yeah, I can understand people’s frustrations with the language, but this is just completely wrong. And what is this !@#$ about Python being a copycat language?! Yes, some features were borrowed from other languages — list comprehensions come to mind — but there aren’t that many examples of it, and it’s hardly the only language that does this. In fact, if I’m not mistaken, Java, C++ and C# all have grown features borrowed from dynamic languages… except such features tend to have limited use in a environment that is statically typed.

Well, I’ve started to sell some stuff on eBay… I will post the listings here, so you, my faithful readers, will get to see them first. :-)

I am actually selling these items so I can pay Dreamhost for another year of hosting, so it’s for a good cause. (I hope…)

Anyway, here are the first listings:

• Lisp in Small Pieces:
  eBay listing *SOLD*
  Amazon listing (sells for $79 used, $83 new)

• Programming Python, 2nd edition:
  eBay listing
  Amazon listing

• Python Cookbook, first edition:
  eBay listing
  Amazon listing

• Python and Tkinter Programming:
  eBay listing
  Amazon listing ($188 new, $70 used)

Programming Python, and the Cookbook, are a bit dated, of course… but that’s why they start out cheap! ;-)

The other two seem to be collectors items, more or less; my buy-it-now price is still lower than what you pay for the used version on Amazon, though.

More books coming up… mostly programming/computer science, and maybe some gaming stuff… if you are looking for something specific, and you suspect I might have it, drop me a note. :-)

For the past 2-3 months or so, I’ve been tinkering with a Lisp dialect called Liquid. Current prototype is in Python, but the idea is that I will eventually port this to a different platform; maybe the JVM, .NET, or Actionscript. (I should probably write a separate post discussing these choices.)

The prototype works, to some extent, and can be played with, for those who like this kind of stuff. (Just run the liquid script to get a REPL… error handling etc isn’t too good, but this should be better in future versions.)

Constructive criticism and ideas welcome (but keep in mind that this is just a prototype and that I am not an old Lisp hand :-).

Yesterday I stumbled upon this: Heist, a Scheme interpreter in Ruby.

It looks pretty good; it’s certainly more advanced than my attempt at a Scheme interpreter in Python (which is, in turn, more advanced than Psyche, and will be released eventually, once I backport some of the dollop code to it). Apparently it supports (among other things) hygienic macros and continuations, features that haven’t yet made it into my interpreter.

On a side note, it’s interesting to see how different the Ruby code looks compared to Python code. Five years ago I would have said (and did say :-) that Python and Ruby are much more alike than unalike. They still are, but they surely encourage (and discourage) different programming practices and idioms.

I’ve written a few (toy) Lisp interpreters over the years. Who hasn’t right? :-) At the core of such interpreters, there’s the eval-apply cycle. It basically works like this:

• To evaluate an expression, call eval on it.
• If the expression is atomic, this is just a matter of a simple name lookup, or the expression evaluates to itself (in the case of literals).
• If the expression is composite (i.e. a list), it is considered to be a function call, where the first element of the list is a function, and the other elements are the arguments passed to that function. We evaluate all of these elements (again by calling eval on them), then call apply to do the actual function application.
• If said function is built-in, we just call it, get a result back, and be done. If it’s a user-defined function, then we need to evaluate the expressions in that function’s body, using the procedure described here.

This is a simplified version — it doesn’t take into account special forms or macros, for example — but it does capture the essence of the eval-apply cycle.

Anyway, since expressions can be nested arbitrarily deep, it should be clear that eval and apply can and often do call themselves and each other.

Now, when writing a Scheme interpreter in a language like Python, it is tempting to use a straightforward translation of this principle. You write functions (or methods, if you wish) eval() and apply(), which may call themselves and each other. This works rather well — until you blow the stack. The implementation language’s stack, that is… because every call to eval or apply in the Scheme interpreter, also uses the implementation language’s call stack.

This is a problem, especially in Scheme, which relies on tail recursion to define “iterative” processes. (These still use recursion, but because of tail call optimization, they operate in constant stack space. See SICP 1.2.1.)

We can simulate tail recursion by using exceptions (as demonstrated by this Cookbook recipe, for example). This is the approach I followed in most of my projects. While it does work, sooner or later you’ll run into other problems, because the interpreter is still piggybacking on Python’s call stack, rather than having one of its own. So, implementing continuations, for example, becomes very difficult, as the notion of “the calling expression” is implicit.

So, I decided to use a different approach. And now we’re finally getting to the point of this blog post. :-)

This time I started with a call stack. The expression we want to evaluate is pushed onto it. For example,

(+ 1 a)

Then, I wrote a method run() which does one “evaluation step” using this call stack. If it’s done evaluating, it returns the result value, otherwise it returns None, as an indication that there’s still more work to do. (So, in order to evaluate an expression, you call it multiple times until you get a result back.)

How does such an evaluation step work? We take the topmost expression on the call stack, and start evaluating it. In this case, it’s a composite expression, so we just start at the beginning. We want to evaluate the symbol +. To do so, we push + onto the stack, and put a placeholder in its original position, indicated by “$$”.

;; call stack now looks like:
($$ 1 a) +

We’re now done with this step. In the next step, we again look at the topmost expression on the stack. It’s a symbol; we look it up, get a function back (represented by <lambda:+>, and stick it back into the original expression. We then move on to the next element of the list, which is 1, and push it.

(<lambda:+> $$ a) 1

Another step, and we get:

(<lambda:+> 1 $$) a

Let’s say the variable a has been defined and its value is 4. We then get:

(<lambda:+> 1 4)

Now we’re done evaluating the elements of this expression, and we can do the actual function application. Finally, we get the result 5.

This gets more complex when we use nested expressions, but the principle is the same.

(+ (* a (- c d)))
($$ (* a (- c d))) +
(<lambda:+> $$) (* a (- c d))
(<lambda:+> $$) ($$ a (- c d)) *
(<lambda:+> $$) (<lambda:*> $$ (- c d)) a
(<lambda:+> $$) (<lambda:*> 4 $$) (- c d)
;; ...etc...

It becomes more interesting when we introduce special forms into the mix. But since we have an explicit call stack now, we can clearly see what is going on with it, and tail recursion starts making much more sense. For example, if our topmost expression is

(if condition (do-this) (do-that))

and during evaluation we determine that condition evaluates to true, we can at that point replace the whole expression by

(do-this)

rather than having to push (do-this) and then sticking the result back into the if expression. Ditto for the last expression inside a begin block, and so on.

The special forms require a bit of hackery, since we don’t evaluate all of their arguments… but there’s only a few of them.

The proof-of-concept implementation that uses this concept is called dollop and is available at github. (Requires Python 3.0.) The name is because it’s only a “dollop of Lisp” (or rather, Scheme); it only supports a few special forms (begin, define, if, lambda), and a few functions for example programs (+, -, *, =, list). It cuts corners in other ways as well, as my goal was to get a working proof-of-concept out, not to write a complete Scheme interpreter.

Anyway, comments, bug fixes, improvements, etc, welcome.

(I will also release two other Lisp-y interpreters, but those will get blog posts and project pages of their own…)

Remember that old CCG Magi-Nation? Some kind soul has been working on a searchable card database (kind of like Gatherer for MtG) for this game… Magi-Nation Search. The site is still under construction, and a bit sluggish, but usable… and it uses Python as the query language! How cool is that? :-)

Update: As it turns out, that version was dog slow, due to the initial loading of cards which took 14s on my machine. >.< A new version, which takes 0.5s, is available now.

Starting immediately, I am available for part-time or full-time work! Yay! ;-) I’ll repeat last December’s post here, since it still applies. (Resume has been updated since then.)

I’m still getting the occasional inquiry about the status of Wax. I wonder if it would be worth reviving the project?

This is great stuff: A Challenge To Break Python Security.

The challenge is simple:

• Open a fresh Python interpreter and do:

  >>> from safelite import FileReader

• You can use FileReader to read files on your filesystem
• Now find a way to write to the filesystem from your interpreter

This has been discussed extensively for the last few days on python-dev. It’s funny how code seems to be pretty secure at first glance, then someone comes up with another loophole.

It especially piqued my interest since I am working on yet another searchable card database, this time using Google App Engine. Kind of like Gatherer, but for a different CCG than Magic, and (hopefully) more flexible. What does this have to do with security? Simple: the most flexible way to search cards is to store them as Python objects, then search them using a Python expression, e.g.

card.red and card.black and (card.creature or card.instant) and card.cost > 2

…or, a more convoluted query:

(card.red or card.black) and not card.multicolor \
and card.type == 'Dragon' and card.set.year > 2006

Now, executing an arbitary Python expression entered on a web page, is of course very unsafe. So I need to find ways to make it more secure, while still preserving the flexibility of a Python-based search. Although I’m not sure how much it matters in this particular case, because:

• According to the App Engine docs, an application cannot write to the system, so no overwriting of files (?).

• I’m not using the data store at all, and there is no user registration, so there’s no sensitive data to be accessed or manipulated.

• Projects like Try Python and Try Ruby seem to fare pretty well without imposing many restrictions on the user.

That said, there might be other ways to mess with the site. Personally I don’t care if a user manages to screw up their own session due to malicious hackery, as long as it doesn’t affect other users. :-)

Anyway, the site isn’t ready yet, I still need to add more cards and flesh out the API. If you want to try it (locally), drop me a note, and I’ll send you the code.

What are good places nowadays to look for work, if you’re a somewhat experienced Python developer? I mean, aside from the obvious places to look, like the Python Job Board.

Are sites like Monster and Dice still good ways to find work? Are there new players on the field? I haven’t had to look for work in a while, so I’m kind of out of touch.

Any suggestions are welcome… ^_^