Update: I played around for a few more hours and got something much better. It’s still fragile, but not nearly so much as before, and it restores the fade effect for all the google content. Same download link as before.

Update 2: Looks like Google fixed their homepage sometime this morning, so this extension is now officially obsolete. I will continue to host the download if anybody is interested in the code.

Read more to find out how 1Password gets around these limitations.

A bit of history

When Cocoa was introduced, one of the behaviors that every Cocoa application automatically acquired was the loading of Input Managers. These Input Managers were intended to allow developers to extend the text input system of OS X in ways that the system did not provide by default. However, these Input Managers were really nothing more than Cocoa bundles that got loaded by every single Cocoa app at launch. This means that it was very quickly abused to become a general plugin mechanism for applications that do not natively support plugins (such as Safari). In recent OS updates, Apple has been deprecating this mechanism, and now in Snow Leopard it’s completely gone for 64-bit apps.

How can I use Safari plugins?

The simplest way is to simply right-click Safari in the Finder, select Get Info, and check the Open in 32-bit mode checkbox. However, this has the downside of removing all of the performance benefits of 64-bit apps. Besides the normal benefits of an improved instruction set and a more modern Objective-C runtime, Safari’s JavaScript engine particularly benefits from running in 64-bit mode. So while this is a stopgap measure, it isn’t suitable for long-term use.

Luckily, the smart folks who make 1Password came up with a solution for their upcoming 1Password 3.0 (which is in public beta right now). I have only done some minimal poking around, but I believe I’ve figured out the mechanism they use to load their plugin into Safari in the absence of Input Managers.

Sow how does it work already?

First, an introduction to AppleScript and Scripting Additions. AppleScript is a rather old technology, first introduced in System 7. It is a human-readable scripting language that can control any application that implements support for it, along with a slew of system functions and, more recently, generic control of UI elements[^1]. Under the hood, it sends Apple events to actually talk to each process. Scripting additions are bundles that provide additional functionality to AppleScript, generally by installing Apple event handlers or doing Apple event data coercion. Several scripting additions are bundled with the system, such as the Keychain Scripting addition which provides access to the Keychain from within AppleScripts.

What does this have to do with 1Password?

I’m getting there. The thing about scripting additions is that they will be potentially loaded by any process on the system. Generally, they get loaded into a process that attempts to use an AppleEvent that the scripting addition handles. Here it gets a little fuzzy, as the documentation does not specify whether the sending process or the receiving process loads the scripting addition. In general, it doesn’t matter, and so I suspect the sending process is usually the one that handles the event[^2]. In fact, up through Mac OS X 10.5, the “context” for a scripting addition was always assumed to be “Machine”, which means the AppleEvent can be handled by any process on the machine it was sent to[^3]. New in 10.6 is the ability to specify exactly which context the scripting addition handler must execute in, including “Any”, “Machine”, “User”, or “Process”. That last context is the important one, as it means the scripting addition handler must be executed in the Apple event’s target process.

There is one caveat here which is that if a process has not initialized AppleScript, it will not have loaded any scripting additions. You can get around this by sending the kASAppleScriptSuite/kGetAEUT (ascr/gdut) Apple event to a process (according to QA1070 the system automatically installs the handler for kASAppleScriptSuite/kGetAEUT on Mac OS X 10.2 or later). This event causes the “system handler table” for that process to be initialized, which loads the scripting additions.

So we can load Scripting Additions. What now?

The ability to load a scripting addition into a target process simply by sending it an Apple event is the key mechanism that allows us to restore the old Input Manager functionality. And this is exactly what 1Password does. 1Password includes a scripting addition that handles the ONEP/Load Apple event with a context of “Process”. This handler takes a single argument, the path to a given bundle, and it loads that specified bundle into the target process. The last component is a background daemon called 1PasswordAgent. This daemon sends the ONEP/Load Apple event to Safari immediately after Safari is launched[^4], causing Safari to load the 1Password WebKit plugin. You can see this in the following AppleEvent log:

~~~ { 1 } ‘aevt’: ONEP/Load (i386){ return id: 284 (0x11c) transaction id: 0 (0x0) interaction level: 64 (0x40) reply required: 0 (0x0) remote: 0 (0x0) for recording: 0 (0x0) reply port: 37635 (0x9303) target: { 1 } ‘psn ‘: 8 bytes { { 0x0, 0x1a01a } (1PasswordAgent) } fEventSourcePSN: { 0x0,0x1a01a } (1PasswordAgent) optional attributes: < empty record > event data: { 1 } ‘aevt’: - 1 items { key ‘——’ - { 1 } ‘TEXT’: 70 bytes { “/Applications/1Password.app/Contents/Extensions/WebKitExtension.bundle” } } } ~~~

Is that it?

Yep, that’s it. Through the use of the Scripting Additions mechanism and a daemon, 1Password is able to load its bundle into 64-bit processes. This is actually fairly similar to how applications that use mach_inject work, except instead of injecting code at the mach level, it leverages existing higher-level interprocess communication mechanisms, which is likely to be far safer.

There is one more step. Safari sends an AppleEvent back to 1PasswordAgent (1Pwd/UNLK) after the bundle has been loaded. I am not sure what the purpose of this is, but I suspect UNLK stands for “Unlock” and is used for some sort of synchronization between the 1Password extension and 1PasswordAgent. It should not be necessary for most plugins.

How do I use this in my own Safari plugin?

At the moment, the only way is to do all the work that the 1Password guys did, and run your own daemon in the background to load your plugin into Safari. Perhaps if enough people ask, the 1Password guys will generalize their solution into something like SIMBL, e.g. a generic application plugin loader. Or maybe someone who reads this will be inspired to do it themselves.

One last thing: Apple clearly does not like bundles loading themselves into arbitrary processes on the system. And for a good reason. A lot of crashes and instability in the system are caused by poorly-written Input Managers or APE Haxies. This is why they removed the Input Manager system entirely. Unfortunately, as long as Safari continues to have no plugin API, such hacks are necessary to be able to use products like 1Password. If you choose to use this mechanism yourself, just be aware that Apple will most certainly frown upon it, and may try and take steps to eliminate this hole in future versions of the OS. And please, make sure your plugin is bug-free and future-compatible[^5]. The absolute worst thing you can do is cause users to experience crashes in applications.

[^1]: Controlling UI elements requires turning on the “Enable access for assistive devices” checkbox in the Universal Access preference pane. [^2]: At least, on Snow Leopard. It seems Leopard always loaded the scripting addition into the target process, as it did not have the concept of a context. [^3]: AppleEvents can be sent between machines if Remote Apple Events is enabled in the Sharing preference pane. [^4]: As I suggested earlier, it actually sends the kASAppleScriptSuite/kGetAEUT (ascr/gdut) Apple event to Safari first, to ensure the scripting addition has been loaded. [^5]: One of the big wins of SIMBL was that it made it so easy to specify a maximum supported bundle version for your plugin, and it strongly encouraged that you always set this to the current bundle version. In other words, whenever Safari got updated, you would be forced to test your plugin against the latest version of Safari and re-certify it as correct, or it would simply disable itself.

Update: Every now and then people keep coming to me and asking me about Zynga. To clarify, I don’t work there any more. I left at the end of 2009.

DNS-SD Browser is the iPhone version of my popular desktop Bonjour Browser software. It enables you to view all of the Bonjour services on your local network as well as on wide-area Bonjour domains.

The reason this is interesting is because it exposes an interesting quirk as to how WebKit interprets HTML entities. WebKit properly converts entities that represent characters outside of plane 0 into a surrogate pair, such as 𝍧 (𝍧). This gets converted into 0xD834DF67. The quirk is if you give it the surrogate pair codepoints directly, it doesn’t realize they’re not real characters individually and passes them through unscathed, so that same character can be written as �� (��). Now this doesn’t seem particularly harmful, except if you only write the first of these entities, WebKit will then get very confused. It will end up throwing away the entire rest of the line of rendered text. Interestingly, it starts displaying text again after a line break, even if it’s just an implicit line break.

The ideal behavior here is WebKit should just silently ignore any entities which reference a codepoint that’s part of a surrogate pair. The fact that it doesn’t really doesn’t hurt anything, but I thought it was worth documenting.

Update: A question was raised on twitter about how surrogate pairs affect indexing into a UTF-16 string. I didn’t know the answer, and strangely, I couldn’t find information on how to handle it with google either, so I tested empirically. NSString uses UTF-16 internally, so it was a great way to test. And what I found was that each half of a surrogate pair is counted as a separate character. The -length of the NSString is increased by 2 when you add a surrogate pair, and -substringFromIndex: will happily split up the surrogate pair for you. Of course, if you do split a surrogate pair, then attempting to convert the NSString into another encoding, even with the simple -UTF8String, will return NULL as such a conversion is illegal (when you generate a unicode stream it has to be well-formed, and so you cannot generate a stream with half of a surrogate pair - and half of a surrogate pair in UTF-16 will be converted into a single invalid 3-byte UTF-8 sequence).

Disclaimer: I have no personal stake in Nullriver or MediaLink, I am simply a happy customer.