Partisan Hackery

I don't feel like working on computer stuff just right now, so here is a 'blog entry. The laptop -- she is'a doing'a great! X Windows is not as 3-D accelerated as it could be, but it is accelerated, which is more than I could say for my old SiS. Also, I have to use Xine instead of all that fancy new GNOME Gstreamer stuff, because Gstreamer is... how you say. Not so good yet.

STEWART: See, the thing is, we need your help. Right now, you're helping the politicians and the corporations. And we're left out there to mow our lawns.

BEGALA: By beating up on them? You just said we're too rough on them when they make mistakes.

STEWART: No, no, no, you're not too rough on them. You're part of their strategies. You are partisan, what do you call it, hacks.

What is that I just quoted, right there? If you haven't seen it, I encourage you to click here. It is delightful. I'll quote some more of it in a bit. Go here, too: Rude Pundit. I recommend this one and this one and this one.

One computer thing I've been doing is working on this GNU system called Guile, which is an embeddable Scheme interpreter. I use Guile as the workhorse for gzochi's game-design language -- actually, Guile Scheme is gzochi's game-design language, and I embed Guile to make the games run. One problem, though, that I've had since day one, is that Guile's thread support is a bit shaky. Wait a second, have I explained threads in the blog before? Sometimes I like to explain some of this technical stuff so that if people like Devlin, for example, who are somewhat technically naive, but who have some vested interest in understanding stuff can know what I'm talking about.

So: Many times you will be writing a program, and you will want to start doing two things at once -- say, you want to listen for keys getting typed at the keyboard but you also want to be printing out messages you're getting from an Internet connection and you also want to be playing some music or something. Well, given that a program (as conceived of by most programming languages) is a sequential list of things to be done, it is hard to make more than one thing appear to happen at once. I mean, more than one thing cannot be done at once, but if you switch back and forth very quickly, you can give the user the impression that these things are happening at once. And if you want to do this yourself, you would have to say, in your program, "Okay, read a little bit from the keyboard, and now write some stuff to the screen, wait, go back to the keyboard, okay play a few notes of music, back to the screen, etc." That's a pain. So the operating system will very often give you a hand with this, in the form of "threads." You tell the operating system, via your program, that you have several disparate lists of instructions you want it to execute, several "mini-programs," say, called threads, and the system will start executing them all and will handle all of the switching between them for you. You don't have to worry about it, for the most part, until they have to interact with each other in any way, and then you have to worry a lot. But that's what threads are, anyway. Here's some more Jon Stewart:

BEGALA: Let me get this straight. If the indictment is -- if the indictment is -- and I have seen you say this -- that...

STEWART: Yes.

BEGALA: And that CROSSFIRE reduces everything, as I said in the intro, to left, right, black, white.

STEWART: Yes.

BEGALA: Well, it's because, see, we're a debate show.

STEWART: No, no, no, no, that would be great.

BEGALA: It's like saying The Weather Channel reduces everything to a storm front.

STEWART: I would love to see a debate show.

BEGALA: We're 30 minutes in a 24-hour day where we have each side on, as best we can get them, and have them fight it out.

STEWART: No, no, no, no, that would be great. To do a debate would be great. But that's like saying pro wrestling is a show about athletic competition.

Right, so Guile, when I started investigating it, supported a form of threading, which I needed, because the games being hosted by the gzochi server would need to be able to evaluate Scheme code concurrently with each other, but it was a weird, custom kind that wouldn't interact well with the operating system's normal threads. But the Guile people are in the middle of writing a new version and they've fixed that aspect of their thread stuff, and I'm excited about it. However, it looked like they still hadn't fixed this other thing, which is that their thread support didn't entail letting you cancel running threads.

See, sometimes one thread will be running and another thread will discover that there's no point in having the first thread keep going and that it should be shut down; for native Unix (and Windows) threads, this second thread can do just that, by telling the system to cancel the first thread. It gets a little bit complicated, though, because this first thread might be in possession of some resources that can only be held by one thread at a time -- for example, maybe a thread has exclusive rights to write to a particular file. If the thread is cancelled, the file stays inaccessible to everyone else. To get around this, most thread implementations allow you to install what're called cleanup handlers for threads, which are sections of code that get run when a thread is cancelled. So, for example, before you "lock" a file to write to it in a particular thread, you install a cleanup handler that unlocks the file, so that if the thread is cancelled in the middle of writing, the file gets unlocked for later use. More Stewart:

STEWART: But the thing is that this -- you're doing theater, when you should be doing debate, which would be great.

BEGALA: We do, do...

STEWART: It's not honest. What you do is not honest. What you do is partisan hackery. And I will tell you why I know it.

CARLSON: You had John Kerry on your show and you sniff his throne and you're accusing us of partisan hackery?

STEWART: Absolutely.

CARLSON: You've got to be kidding me. He comes on and you...

STEWART: You're on CNN. The show that leads into me is puppets making crank phone calls.

So Guile didn't have thread cancellation, which is something I needed, so that threads of Scheme evaluation that took too long wouldn't get the game "stuck." Well, I figured since the new version was in active development, I'd take a shot at it myself. Here's how I did it:

Every Scheme thread created by Guile has some information associated with it, such as a value to return to other threads that are waiting for it to finish, etc. I take that information, and add a little bit more to it: A list of expressions that need to be evaluated when the thread receives a cancellation signal. See, Scheme's what's known as a functional language, so expressions (instead of instructions in a so-called imperative language) are the basic unit of currency. So before you put a lock on some resource in your new Guile thread, you add a cleanup handler to my list of handlers -- so that you can clean up if you get cancelled while you're using the resource -- and then when you're done, you uninstall the handler. It wasn't super hard to do.

Here are the problems so far: Guile does this fancy dynamic library thing when it loads which makes it rather difficult to debug, so it took me a while even to begin to make progress debugging things. Also, threads that aren't created by Guile are not straightforward to cover with the cancellation cleanup policy I described above -- think of it like this: You embed Guile as an interpreter for Scheme code in your C program. You have a thread of C code initialize Guile and start intepreting Scheme for you. Your Scheme-interpreting-C-thread represents, in a way, Scheme-thread #1. Let's say the Scheme code you're interpreting launches another thread of Scheme code -- this thread is Scheme-thread #2, and will also, at it's core, be C code interpreting Scheme code, but this thread is covered by the cancellation policy, whereas Scheme-thread #1 is not. I think that's a feature, not a bug.

We'll see if they like it.

More computer shit: So Raymond Chen just posted this thing about how application developers chronically misuse the Windows API -- e.g., storing application data in like, undisplayed components of the widget set -- and then Microsoft had to go out of their way to work around this so their program would still work in the next version of Windows; he says (used without permission):

The moral of the story: Even if you change something that nobody should be relying on, there's a decent chance that somebody is relying on it.

(I'm sure there will be the usual chorus of people who will say, "You should've just broken them." What if I told you that one of the programs that does this is a widly-used system administration tool? Eh, that probably wouldn't change your mind.)

Some guy posted the following comment, which I think is a nice counterpoint (again, used without permission):

Although it really is amusing to read all those stories about "bad guys" who did something wrong and nowadays we have a few megs of code only to catch them, what is the real moral in that?

The only lesson I have learned is that I can use any dirty trick I want, if my app would be important enough that MS would test it (and then make sure it won't break). Why should I use ACT if MS will do that for me (and "repair" Windows accordingly)? (And if my app is not important enough, I'll just post some info about the trick to a newsgroup, someone else's important app will use it.)

Raymond didn't have anything to say on that one, last time I checked. Imagine how good Windows could be if Microsoft didn't take this retarded tack when it came to developing it!