The reason this is interesting is because it exposes an interesting quirk as to how WebKit interprets HTML entities. WebKit properly converts entities that represent characters outside of plane 0 into a surrogate pair, such as 𝍧 (𝍧). This gets converted into 0xD834DF67. The quirk is if you give it the surrogate pair codepoints directly, it doesn’t realize they’re not real characters individually and passes them through unscathed, so that same character can be written as �� (��). Now this doesn’t seem particularly harmful, except if you only write the first of these entities, WebKit will then get very confused. It will end up throwing away the entire rest of the line of rendered text. Interestingly, it starts displaying text again after a line break, even if it’s just an implicit line break.

The ideal behavior here is WebKit should just silently ignore any entities which reference a codepoint that’s part of a surrogate pair. The fact that it doesn’t really doesn’t hurt anything, but I thought it was worth documenting.

Update: A question was raised on twitter about how surrogate pairs affect indexing into a UTF-16 string. I didn’t know the answer, and strangely, I couldn’t find information on how to handle it with google either, so I tested empirically. NSString uses UTF-16 internally, so it was a great way to test. And what I found was that each half of a surrogate pair is counted as a separate character. The -length of the NSString is increased by 2 when you add a surrogate pair, and -substringFromIndex: will happily split up the surrogate pair for you. Of course, if you do split a surrogate pair, then attempting to convert the NSString into another encoding, even with the simple -UTF8String, will return NULL as such a conversion is illegal (when you generate a unicode stream it has to be well-formed, and so you cannot generate a stream with half of a surrogate pair - and half of a surrogate pair in UTF-16 will be converted into a single invalid 3-byte UTF-8 sequence).

By far the most powerful aspect of Twitterific is the SMS support. I have free SMS with my Sidekick data plan, but I’ve never had a reason to use it before. But now I can receive tweets on my phone all day long, and send my own no matter where I am. For example, I sent this tweet while in class.

If you want to keep track of what I’m doing, just take a gander at my profile. And if you sign up for your own account, you can add me as a friend to receive my tweets. If you add me and I recognize your name, I’ll probably add you back too.

We migrated off of Planet Argon and onto DreamHost, where we should have more control. We also upgraded to Trac 0.10.3 and turned off anonymous editing (users now have to register to file a ticket). This should (hopefully) prevent the issue that lead to Typosphere dying in the first place.

The issue, as near as I can tell, is Typosphere started getting spammed massively. At this time none of the developers (and that includes me) was really paying attention to Typo, as we were busy with other things. So for about a month Typosphere Trac got so full of spam that, well, it was more spam in one location than I’ve ever seen in the rest of my life. This managed to trip a bug in Trac that caused it to start sucking CPU and RAM, and so Planet Argon turned off Trac for our account.

Some time later, the other developers and I started trying to resurrect Typosphere. Unfortunately, at about this time the systems administrator for Planet Argon was preparing to leave the company, so any attempts at contacting him to resolve the issue went unanswered. I eventually called Planet Argon (which is how I learned that the systems administrator had, in fact, left that very day) and spoke to the new systems administrator. He agreed to try and fix Trac for us, but after hearing nothing for a few days, I decided it would be better to seek hosting elsewhere.

Luckily, I had access to a DreamHost account with plenty of spare bandwidth and disk space, so we decided to move there. For the most part the migration went smoothly, until I started up Trac and discovered exactly how much spam was in there.

This problem stumped me for about 2 weeks. I spent several hours trying to clean it by hand one day, and after those several hours I couldn’t tell the difference. So, yesterday, I finally sat down to try and solve the problem.

With the help of the fine folks on the #trac IRC channel, especially coderanger, I wrote a script which deleted every single ticket change after a certain timestamp (corresponding to the first spam comment). Unfortunately, there were probably a handful of legitimate changes lost, but there really was no other alternative. In any case, this script worked flawlessly, and Trac was de-spammed. To prevent this from happening in the future, I turned off anonymous editing and installed a plugin which allows users to register for an account. Hopefully the requirement of registration will block most spam.

There was one interesting aspect to this that puzzled me until yesterday. The vast majority of the spam I saw contained words that I had placed into the blacklist ages ago. I couldn’t figure out why the spam protection wasn’t working. And then yesterday I discovered the reason. The blacklist is kept on a page called BadContent. The first code block on that page consists of regular expressions, one per line, that each match a blacklisted expression. Unfortunately, I forgot to mark this page read-only. So what happened was one of the random spam attempts happened to target this page. The spammer replaced the content with his own code block containing a vast number of <a href> tags linking to spammy websites. This had the effect of replacing the entire blacklist with a bunch of regular expressions matching <a href> tags. This meant that all of the stuff that was previously blacklisted was no longer being blocked, opening the floodgates for all sorts of spam.

I think I’m going to convert Typo to use Haml for all its templates. I already did Azure’s layout file and it was pretty simple.

Update: Comments disabled because I’m tired of all the inane comments I’ve been receiving. There’s really no discussion to be had here, so no point in having comments.

Anyway, I wanted a way to do JavaScript cross-site AJAX tricks, like what Greasemonkey enables. Unfortunately, because MouseHole is a pre-processor, not a JavaScript extension, it’s not possible. So I spent a few hours and wrote a way for MouseHole scripts to provide content that doesn’t actually exist (as opposed to mutating existing content). This way JavaScript can do an AJAX call within the same domain (so it doesn’t hit the security limitations) but that call is intercepted by your script, so you could pull info from other sites and send it back, or whatever you wish.

I haven’t actually done anything with this new capability yet aside from a rather silly example script which simply adds a button to Google that asks MouseHole for a random number. I’d really like to extend Google Maps, but that will require delving into how it works, and it’s far too late to do that now, as I need to get to bed.

Anyhow, I put together a patch and sent it to the MouseHole Scripters mailing list. If you check out MouseHole from the CVS repository, you can apply my patch and test it. Or you can just wait to see if it gets added to MouseHole.

The only problem was, the logic matrix for the puzzle doesn’t fit on a single piece of paper, and I don’t have Excel installed. So what did I do? I wrote a CGI script in Ruby that generated a logic matrix using a table of checkboxes, and I wrote some accompanying JavaScript to make it easier to use. Go try it!

If you don’t want to solve the puzzle yourself, feel free to check out some of my steps. Just please, don’t submit this as your own answer. If you’re going to submit an answer, do the work yourself. So anyway, here are some of my steps:

1. Step
2. Step
3. Step
4. Step
5. Step
6. Step
7. Step
8. Step
9. Step

And here’s the final solution.