Dan Wood: The Eponymous Weblog (Archives)

Occasionally I want to find what package a particular Mac OS X tool was installed with. I don't want to make the mistake of calling a library or tool (using NSTask) that is only installed with developer tools.

For instance, you wouldn't want to have a non-developer Cocoa application invoke /usr/bin/atos because that is only installed as part of the developer tools, not part of the standard Mac OS X installation. On the other hand, /usr/bin/curl is standard on all Macs (at least in recent years).

How do you find out for sure? One way is to have a mac laying around without developer tools installed. Sorry, no-go for me! A better way is to look inside your receipts to see what package the tool in question was installed from. If it's in Essentials.pkg or BSD.pkg or BaseSystem.pkg, you can probably assume safely that all your users (at least on the version of the OS you are testing) have that tool available. If it's in some other package, you might have to ensure that the package in question (such as developer tools or a specific Apple-created application) is part of the system requirements for your program.

I have been trying to get comfortable in bourne shell scripting lately; here is my little script to find a string inside all of your computer's installation receipts (found in /Library/Receipts). I call it bomfind; use it like bomfind pattern.

#!/bin/sh

# Go to main receipts directory.  Does not search in home dir.
cd /Library/Receipts

for PKG in *.pkg
do
    pushd "$PKG/Contents" > /dev/null
    CONTENTS=`lsbom -s Archive.bom | grep $1`
    if [ ! -z "$CONTENTS" ]; then
        echo "$PKG:"
        echo "$CONTENTS"
        echo
    fi
    popd > /dev/null
done

Occasionally the developer may want to store a preference in User Defaults which is only applicable to a particular machine, meaning that if the application is launched on a different machine, that preference will not be in effect. Imagine, for instance, a preference to show animation or not that is turned off on a slow machine, but then when the user upgrades to a fast machine and transfers her home directory, the old preference would no longer apply.

There is actually a way to do it somewhat elegantly in Cocoa, using [ScreenSaverDefaults defaultsForModuleWithName:], even if you're not writing a screen saver. The defaults are stored per-machine in ~/Library/Preferences/ByHost/, with the file name including the machine's MAC (Ethernet) address as part of the file name so that preferences are unique to each machine.

We tried using this approach in Sandvox, but found it tricky to store some defaults there and others in [NSUserDefaults standardUserDefaults]. So instead we came up with a new, simple approach: use the MAC address as part of the defaults key.

For instance, we have a preference to remember if Core Image is accelerated on the particular machine. (The test for this is a bit of a lenghty process, so it's best to perform it once when needed and store the result in user defaults for subsequent launches.) Rather than writing the preference under the key of "CoreImageAccelerated", we construct the key like this:

NSString *key = [NSString stringWithFormat:@"CoreImageAccelerated %@",
    [[KTUtilities MACAddress] base64Encoding]];

As with the screen saver defaults, we use the MAC address. Alas, there is no easy way to get the MAC address of your Mac, but Apple has provided some decent sample code to get started on that method; I'll leave that as an exercise to the reader.

We happen to have some methods to encode and decode in Base 64; it's a nice compact way to represent binary data as an ASCII string. So we take the NSData returned by MacAddress and convert it into a string. I won't go into the implementation details, but it's just a few lines of code wrapped around some library functions in curl, which every Mac has. Base 64 encoding, for example, uses the function Curl_base64_encode.

Each time the application is launched from the same machine, the key will remain constant. If the user defaults are tranferred to another machine, then the value will be reset for that machine. (The old key will remain in the defaults file, which might be useful if you have a shared home directory, where the same home directory and defaults file is used but from different machines.)

Third past today! Since I just posted a "gripe" entry, I thought I would also post something constructive. I wanted to talk about how cool class posing is for debugging.

We have used class posing quite a few times just to insert some extra warnings for our debug builds. A lot of times, you can catch a nasty condition before it happens and provide some information about what's happening. For instance, if you are only supposed to call a particular method from the main thread, a posed subclass could define your entry points to log an error message if it's being called from a background thread.

For example, I wanted to log some information whenever an NSThread got detached, so I could see where all my application's threads were coming from. So I created a subclass of NSThread called, unimaginatively, MyThread:

@interface MyThread : NSThread
@end

This class has two methods (in an @implementation block, of course). The first is used to force the subclass to be used in place of its parent class:

+ (void) load
{
    [self poseAsClass:[NSThread class]];
}

The second is a redefinition of +[NSThread detachNewThreadSelector: toTarget: withObject:]. It logs some useful information and then calls the "super" implementation to actually do the work. I've simplified my code a bit here but you should get the idea:

+ (void)detachNewThreadSelector:(SEL)selector
                       toTarget:(id)target
                     withObject:(id)argument;
{
    LOG((@"Detach -[%@ %@%@]", target, NSStringFromSelector(selector),
        argument));
    [super detachNewThreadSelector:selector
                          toTarget:target
                        withObject:argument];
}

Generally I don't need this logging to happen, so the code is normally commented out with #ifdef 0. But when I need to know what's going on with threads, this is very helpful.

I've also use this technique to debug my bindings and log each instance of -[NSObject bind:toObject: withKeyPath: options:] and -[NSObject unbind:], all the initialization methods on QTMovie to make sure that the movies are created on the main thread, and so forth.

I'm going to channel Andy Rooney for this post. The difference is, you as a reader get to comment, so if I've been living in my debugging cave for so long that I've failed to learn the appropriate debugging techniques, feel free to enlighten me!

Gripe #1: For some reason, Apple's crash catcher isn't noticing crashes in the app I'm working on, if I launch it normally (without XCode). If you don't get the "unexpectedly quit" dialog, then you don't get a crash report. And yet other applications get the crash report and log just fine — Safari crashed on me yesterday and I got the dialog and the log.

Gripe #2:Some crash logs come back with an application version of '???'. They must be launched improperly so that the info.plist is not readable. It's hard to know what version I'm examining in that case!

Gripe #3:Exceptions that were helpfully caught for me without giving me enough context information are really annoying. for instance, check out this thing in a log:

*** NSRunLoop ignoring exception '*** -[NSCFDictionary setObject:
forKey:]: attempt to insert nil value' that raised during posting of
delayed perform with target 181d2ce0 and selector 'processTasks'

.... or ...

Exception raised during posting of notification.  Ignored.  exception:
*** -[NSCFDictionary setHidden:]: selector not recognized
[self = 0x6b95f90]

Yeah, I see that it's somewhere downstream from my method 'processTasks'. But I don't know the dictionary or where it got set.

Gripe #4:Sometimes, symbols in a backtrace are returned to us, and when we look them up on our symbols build with atos, the resulting strings make absolutely no sense. Sometimes.

OK, I'm done griping for now. Back to happy, shiny thoughts of Christmas cheer!

One thing that drives me crazy about debugging in XCode is how the default print-object command invokes debugDescription on an object. Normally this is the same as description, but for certain data objects, it's much less readable than plain old description. OK, it has a bit more information, but it's generally not what I'm looking for and it's formatted so poorly.

What I do in our application is to redefine debugDescription in terms of description for debug builds. (I bracket this code within and #ifdef so it's not built for the release version.) Here's NSDictionary:

@implementation NSDictionary ( OverrideDebug )

- (NSString *)debugDescription
{
    return [self description];
}

@end

I do something similar for NSSet.

Easy enough, right? For NSArray, I've gone a step further by only printing the first 20 or so items in the array. That way an array with thousands of items won't fill up my terminal!

@implementation NSArray ( OverrideDebug )

- (NSString *)debugDescription
{
    #define MAXARRAYSHOW 20
    if ([self count] > MAXARRAYSHOW)
    {
        NSArray *subArray = [self subarrayWithRange:
            NSMakeRange(0,MAXARRAYSHOW)];
        return [NSString stringWithFormat:@"%@ [... %d items]",
                    [subArray description], [self count]];
    }
    else
    {
        return [self description];
    }
}
@end

For NSDictionary, I've gone all out and made a much more useable description. This implementation shows the data in a nicely formatted Hex/ASCII dump that's much more scannable. I also cut it off at 1024 bytes since I don't really want to see much more than this, and I hate to have to wait for a megabyte-long chunk of data to finish displaying.

Note that I've also gone ahead and re-implemented description, not debugDescription, so I always get this new style of display.

@implementation NSData ( description )

- (NSString *)description
{
#define MAXDATABYTES 1024
    unsigned char *bytes = (unsigned char *)[self bytes];
    unsigned length = [self length];
    NSMutableString *buf = [NSMutableString stringWithFormat:
        @"NSData %d bytes:\n", length];
    int i, j;

    for ( i = 0 ; i < length ; i += 16 )
    {
        if (i > MAXDATABYTES)       // don't print too much!
        {
            [buf appendString:@"\n...\n"];
            break;
        }
        for ( j = 0 ; j < 16 ; j++ )    // Show the row in Hex
        {
            int offset = i+j;
            if (offset < length)
            {
                [buf appendFormat:@"%02X ",bytes[offset]];
            }
            else
            {
                [buf appendFormat:@"   "];
            }
        }
        [buf appendString:@"| "];   // now show in ASCII
        for ( j = 0 ; j < 16 ; j++ )
        {
            int offset = i+j;
            if (offset < length)
            {
                unsigned char theChar = bytes[offset];
                if (theChar < 32 || theChar > 127)
                {
                    theChar ='.';
                }
                [buf appendFormat:@"%c", theChar];
            }
        }
        [buf appendString:@"\n"];
    }
    [buf deleteCharactersInRange:NSMakeRange([buf length]-1, 1)];
    return buf;
}
@end

Can anybody think of more improvements to the standard ways out describing objects?

Is this a bug in iTunes 7? Anybody else having this happen?

I've noticed that many albums in my library have lost their "Artist" value, being replaced with "Various Artists". So what used to be albums of songs now are split into multiple albums, or in the case here, just two songs of an album have lost their artist:

One of the cool new features of Sandvox 1.1 is that we now have an Apple Help book.

Before, choosing "Sandvox Help" from the Help menu would just direct the user to view our online wiki. (Some people have wondered why we didn't chose Sandvox to build our help. The answer is that although it is technically possible, Sandvox just isn't the right tool for that kind of content.) We had originally chosen a wiki (MediaWiki) as the authoring system for our help because of the way that it is good at managing content; just by creating a link to a new page, it shows up on a list of orphan pages, so there's an automatic to-do list as you start authoring pages. It's pretty easy to create a site without links to nonexistant pages this way. We were also able to, during the early stages, rely on some user contributions to the site. As we started dedicating some resources to building the help, we were able to edit the site "live" so that the help got better and better as time went by.

At some point, we decided that we were ready to start documenting features that were not part of the current 1.0.x release. To do that, we made a clone of the wiki and put it on a new subdomain, where only the official authors of the site (Myself, Terrence, and Mike) could view and edit. A private wiki seems kind of strange, but we were documenting unreleased software, and I didn't want to confuse users of the released version.

Many users were probably more confused by being directed to a wiki than helped by it. There is a lot of extraneous information on a mediawiki page that really wasn't needed. So I decided to work on exporting the wiki to a simpler page look.

I built up some shell scripts to automate the process. The first step is to download the wiki, as rendered as HTML, onto the local computer. The second step, the bulk of the work, is to clean up the HTML so that only the essential content remains. The third step just merges the cleaned-up HTML into our source tree so it will be part of the application.

Getting the HTML, the first step, is fairly brute-force. There might be some cleverer way to extract information from the mediawiki database, but I wanted something simple. It boils down to a single line of wget: (Simplified here just a bit)

wget --domains=private_domain --level=2
	--no-parent --convert-links --html-extension
	--recursive --reject "*\?*"
	http://private_domain/Special:Allpages

wget is a great little utility; it's too bad it's not included with Mac OS X by default. One thing that I find annoying is that it seems to match the patterns after downloading a URL, rather than before. So it uses up a lot of time and bandwidth for the many special links on a mediawiki page that I don't actually want. The download takes a while, but at least it works for putting a static copy of the main pages of the wiki onto my local system, ready for processing.

The second step is just a big shell script that operates on the files. It performs the following tasks:

1. Copy the downloaded wiki into a new directory for editing the files (so that I don't have to re-download the original files if my script isn't quite right)
2. Remove certain pages and directories that I don't want for the user documentation (developer & designer pages, mediawiki "meta" pages, the top-level pages that are replaced in the Apple Help pages, etc.)
3. Loop through all the pages and build up an index page, properly taking redirect pages into account
4. Repair all links to "redirect" pages to point to the proper target pages
5. Using the sips utility, try converting the PNG files from the wiki to JPEG 2000 format. Only the files that are actually smaller are kept as JP2. (We also have a hand-maintained blacklist of files to exclude from this process because some of the images look terrible when converted to JP2.) Overall, this technique shrinks the images from 8.4 megabytes to 4.3 megabytes!
6. The remaining PNG files are run through optipng to shrink them down as much as possible. This shrinks the files down just a bit more.
7. Using perl -pi -e 's/source/destination/g' ..., do a bunch of substitutions on the .html files to remove the junk we don't want, move keywords that we have explicitly defined at the bottom of each page into the <meta> tags, etc. There are actually a lot of sub-steps here that I won't get into, but I will note that in order for this to really work, I first changed \n to \r so that it would essentially treat the file as a single line of text. I just couldn't get perl to do its substitution across \n line breaks.
8. Run tidy on the pages so the HTML is readable
9. Do a check for dead links.

The final step carefully merges the edited files into our source tree, careful not to clobber subversion tags; it also leaves alone the hand-built, static pages (such as the initial page and "Discover Sandvox"). With the CSS there, the final web site looks and behaves a lot like many of the other help books that come from Apple.

I've replicated this process on our own server as well, so that the website docs.karelia.com contains our help pages as well. There are a few differences with the home page, and we don't remove the special categories such as the Developer and Designer pages. This allows us to link to specific pages on the web when communicating with Sandvox users who need help. (Now if Google would just index docs.karelia.com, that website will be searchable as well!)

On the pages for Apple help, the trickiest part was getting everything just right so that Apple Help would do the right things: list Sandvox in its list of applications, show the correct initial page, be searchable, and so forth. The documentation for Apple Help is sorely lacking, (Actually, this "preliminary" document from 2004 is probably the most useful) but their mailing list fills in the gaps.

OK, this post had nothing to do with Cocoa. Sosumi.

I had to share this with the world. My mom recently bought this water pitcher with the most bizarre label on it:

Product this peculiar overlengthy groove is it design to surface, can stretch reach flowering shrubs preventing rivers from overflow effective when watering flowers.

Full size scan @ flickr: The Luxurious Flower Shedding

Today I kept reading blog posts and notice iChat status messages of people I know so excited about the Nintendo Wii (pronounced "why" — I don't care for computer gaming) ...

... But this story was certainly a sobering take on the phenomenon.

"Oh, you're buying those Nintendo tapes? You've got those ones that are all war.. I've been there. I've seen it." A few people were perplexed, but still curious about what the guy had to say.

...

"You remember that, when you're playing your little tapes," the man said as he gestured playing around with a controller, "you remember that there are people really doing that. They're shooting and they're getting shot at."

...

Not only was it a moment of intense frustration, but also introspection - I was sitting in line for a f[...] Nintendo while there are people dying for no reason. I'm programmed to buy the latest crap just because it's the latest crap, and play games that mock the reality of the horrifying environment of war. You can try your best to change the status quo sonny, but it's not gonna work - so fire up your Sony NintendoBox 2000 and shut the hell up.

Read the whole story about the early-morning encounter at the mall.

This is all over the blogsphere, but I thought I'd mention it here. MacSanta is a grassroots website that popped up yesterday. Within the period of a day or so, dozens of independent software companies have added their wares to this site. All of the products listed below are available for a 20% discount using a coupon code of MACSANTA.

Naturally, Sandvox is there. But some of my other favorite companies and products are listed there as well. There are dozens more applications that I haven't heard of than I could have imagined. This is a great starting point for all your Mac Geek Christmas shopping!

Scott Stevenson was nice enough to blog about my presentation at the Silicon Valley CocoaHeads meeting, both before and after the fact.

It was really fun. I have gone to a meeting here and there; we even presented Sandvox at a special meeting at the Apple Store in San Francisco during last year's Macworld Expo. But this gave me a chance to get really technical, talking about some of the design decisions we made for Sandvox, our use of open source, some client/server techniques we've used for bug reporting and payment, and so forth.

The meeting really inspired me to start blogging about some of this here. I've been in hiding for the last few months while we've hunkered down for this last release of Sandvox, but I don't want to stay hunkered down forever! Maybe if I post some interesting stuff here, Scott will list this blog, or at least the Cocoa section of it, on CocoaBlogs!