The reason this is interesting is because it exposes an interesting quirk as to how WebKit interprets HTML entities. WebKit properly converts entities that represent characters outside of plane 0 into a surrogate pair, such as 𝍧 (𝍧). This gets converted into 0xD834DF67. The quirk is if you give it the surrogate pair codepoints directly, it doesn’t realize they’re not real characters individually and passes them through unscathed, so that same character can be written as �� (��). Now this doesn’t seem particularly harmful, except if you only write the first of these entities, WebKit will then get very confused. It will end up throwing away the entire rest of the line of rendered text. Interestingly, it starts displaying text again after a line break, even if it’s just an implicit line break.

The ideal behavior here is WebKit should just silently ignore any entities which reference a codepoint that’s part of a surrogate pair. The fact that it doesn’t really doesn’t hurt anything, but I thought it was worth documenting.

Update: A question was raised on twitter about how surrogate pairs affect indexing into a UTF-16 string. I didn’t know the answer, and strangely, I couldn’t find information on how to handle it with google either, so I tested empirically. NSString uses UTF-16 internally, so it was a great way to test. And what I found was that each half of a surrogate pair is counted as a separate character. The -length of the NSString is increased by 2 when you add a surrogate pair, and -substringFromIndex: will happily split up the surrogate pair for you. Of course, if you do split a surrogate pair, then attempting to convert the NSString into another encoding, even with the simple -UTF8String, will return NULL as such a conversion is illegal (when you generate a unicode stream it has to be well-formed, and so you cannot generate a stream with half of a surrogate pair - and half of a surrogate pair in UTF-16 will be converted into a single invalid 3-byte UTF-8 sequence).

For those of you who are like me and listen to full albums and lament the lack of a Compilations preference for the iPhone, learn to love the CoverFlow view. Since this view is album-centric it handles compilations properly. The only real flaw with this view is it’s ugly if you don’t have album art.

I ended up camping out at Best Buy starting at 6PM on saturday. If I wanted I probably could have shown up around 4 AM or so and still gotten a unit, but I figured if I was going to camp, I’d do it right. I hung out with a group of guys from WPI (guys I hadn’t met before), and it was a lot of fun. Unfortunately, it was also freezing cold. From about 2AM to 5AM everybody was huddled under blankets trying to stay warm. Some people actually managed to sleep, but I didn’t. Still, I think it was worth it.

pmTool, the process run by Activity Monitor to actually collect stats, appears to leak memory. If I leave Activity Monitor running for a good period of time, when I check up on it pmTool is often using over 100MiB of Real Memory.

I just checked my laptop, pmTool was using over 100MiB of Real Memory. Right now on my desktop it’s using 41MiB of Real Memory, but I don’t remember how long it’s been running for. I also believe a good deal of memory is currently paged out.

After checking up on it, pmTool on my desktop has a Private Memory size of 91MiB.

If I run leaks on pmTool on my desktop, I see 27931 leaks for 85799936 total leaked bytes, which works out to about 81.8MiB of leaked memory. All but 2 of these leaks appear to be blocks of memory of size 3072 that start with '?\0'.

In any case, this seems to be a pretty significant leak in pmTool.

A highly non-scientific test shows it to be adding one new leak every time the UI in Activity Monitor updates (default is every 2 seconds). So if you haven’t changed the update frequency then pmTool will leak 3KiB of memory every 2 seconds.

That’s a lot.

I haven’t noticed pmTool taking more than about 100MiB of Real Memory so far, but I never thought to check Private Memory until now so it may have simply been paging out a lot of leaks. If my calculations are correct, every 24 hours pmTool will have leaked 126.6MiB.

If you keep Activity Monitor running constantly like I do, you may want to check what your current memory usage is. If you see anything close to what I am, you may want to re-evaluate the importance of Activity Monitor (or set up a cron job to periodically restart Activity Monitor). On the other hand if you’re not seeing a leak problem I’d be interested to hear the details of your system (I’m currently running OS X 10.4.8). I’m not entirely certain whether this leak is Apple’s fault or whether one of the myriad customizations to my system is somehow causing this. As far as I know none of the hacks I use that inject into other code processes affect pmTool (seeing as how it’s not an AppKit application, so it doesn’t load Input Managers, and it’s run as root so it won’t be affected by anything that tries code injection), so this seems likely to be Apple’s fault, but if so I’m really surprised that they haven’t fixed it.

Update: I just checked the current memory usage of pmTool. I haven’t restarted it in at least a day, and yet it was listing a reasonable 1.2MiB of Real memory, and roughly the same Private Memory. This is the memory usage I’d want it to have, and yet completely different than the observed behaviour I documented in this post. I double-checked and the memory usage on my laptop was up to 55MiB. I wish I had thought of running leaks on pmTool when I saw this, but instead I quit and relaunched Activity Monitor, and immediately the leaky behaviour came back.

Update 2: I feel silly now. I completely forgot I’d set up a root crontab entry with @hourly killall pmTool to minimize the leak. That’s why it was showing such a small memory usage.

Update 3: I just got a brand new 2GHz Core 2 Duo MacBook, running 10.4.8 with the latest security update (i.e. the same OS as my iMac), and quite mysteriously, this leak isn’t showing itself. pmTool has a respectable 960KiB of Real memory usage, and no leaks. I can’t explain it.

Not only is that game a lot of fun, it’s also a really visually appealing game (surprisingly beautiful for an Xbox game - I can only wonder at how it would look if it took advantage of the Xbox 360). All the moves look great, all the attacks have a lot of style and flair, and Ryu (the main character) is an incredible badass.

Usually when games are really hard and I can’t seem to progress past a certain point, I get discouraged. But ever since my success with that first boss, the seeming impasses in this game only drive me to try harder. There are points in this game where you have to do the same segment of the level over and over, and over, and over, ad infinitum. Again, such repetition would normally discourage me, but instead I use it as the chance to practice and hone my fighting skills until I can breeze past the previously-impossible segments with nary a scratch.

If you have never tried Ninja Gaiden, and you have a chance to (say, you or a friend own an Xbox), you owe it to yourself to purchase Ninja Gaiden Black and give it a try.

After the event I managed to get a few pictures of Tom with my cameraphone. Click on the picture below to view the set.