Per-object ordered relationships using Core Data

Posted on August 8, 2008 by Brian Webster
Filed Under Cocoa, Development, Programming | 5 Comments

Core Data is a great technology to allow easy creation of complex data models, saving you from writing a lot of boilerplate code. One of the limitations of Core Data, however, is that when one entity has a to-many relationship with another entity, the objects in that relationship are unordered. The only order that can be imposed on such a relationship is by sorting based on one or more attributes of the child objects, and there is no “inherent” ordering to the objects otherwise.

Several different solutions (one example here) have been created for this problem, most of which use some variation of adding an “index” attribute to the child objects which specifies what order the objects come in (i.e. the first object has an index of 0, the next and index of 1, and so on). The tricky part come in keeping the index attributes up to date, but there are several solutions for that which all work pretty well.

The major disadvantage to this approach comes when you have child objects which can be contained by more than one parent object. The classic example of this is a program like iTunes, where you can have multiple playlists, each of which can contain multiple tracks, and the user can arrange the tracks in any order they wish in each playlist. The problem is that there is only one “index” attribute for each track, but each track will have a different index depending on what playlist we’re looking at.

I decided to take a stab at this problem using a new approach, which stores the ordering information for each container object in the container itself, and not in the children. I also wanted this solution to be generic and reusable, so you don’t have to reimplement everything for each relatonship you want to give a custom ordering to.

The solution I came up with is a custom subclass of NSManagedObject called BWOrderedManagedObject, which provides support for imposing an ordering on any to-many relationship the object has. At the end of this post, you’ll find a download link containing the source code for this class, as well as a corresponding BWOrderedArrayController class which provides support for sorting a table view by this custom ordering, and a simple demo showing how to hook everything up.

BWOrderedManagedObject

In order to keep track of what order the objects in a particular relationship are in, we need to store a separate piece of data that provides that ordering. The way BWOrderedManagedObject handles this is to store the ordering in a separate attribute. By default, the key used for this attribute is simply the relationship key with the string “Ordering” appended to it, so for example, if you have a “tracks” relationship, the ordering info for your tracks will be stored under the “tracksOrdering” key.

The ordering attribute should be defined in your Core Data model as a transformable property attribute with the appropriate name. The attribute will actually consist of an NSArray holding NSURL objects, with each URL object containing the URIRepresentation of the NSManagedObject it represents. The URIRepresentation is obtained from the NSManagedObjectID for a given object, and is a simple, persistent identifier that can be used to identify the object and won’t change even when saved to disk and read back again (with one important exception – see below). The order of the URLs in the array is what defines the order of the objects themselves in the relationship.

To actually specify an ordering for your objects, NSManagedObject provides a set of methods that allow you to access, insert, delete, and move objects within the array (objectInOrderedValueForKey:, insertObject:inOrderedValueForKey:atIndex:, etc.). It also provides a mutableOrderedValueForKey: method (similar to Cocoa’s mutableArrayValueForKey: method) which returns an NSMutableArray proxy object which you can use to rearrange objects in the relationship as though it were a plain array. To actually access the ordered values, just use the orderedValueForKey: method, which returns an NSArray containing the objects in the correct order.

BWOrderedArrayController

In a typical NSTableView + NSArrayController setup, sorting is accomplished when the user clicks one of the table view’s column headers. Each column has an NSSortDescriptor assigned to it, which are then passed to NSArrayController, which sorts its contents based on the sort descriptors. However, since sort descriptors function by comparing attributes of the objects themselves, this method of sorting won’t work for BWOrderedManagedObject, because the sort order is stored in the container object, and not the objects being sorted

BWOrderedArrayController fills this gap by handling the case where the user has clicked a column which represents the “natural” ordering of the table’s content. To use BWOrderedArrayController, first bind its “contentSet” binding to your object’s content just like you would with a normal array controller. Then, in your table view, select the column you want the user to be able to click to sort the content by their custom ordering.

Instead of binding this column’s “value” binding like your other table columns, instead switch to the attributes inspector pane and enter the name of the ordering key that your content’s ordering is stored under in the “Sort Key” field. For example, if you’re displaying a list of tracks that’s stored in the “tracks” key of the selected playlist, you would enter “tracksOrdering” in the Sort Key field.

If everything is hooked up correctly, clicking that table column will sort the content of the table based on the ordering specified by the user. Performing actual rearrangement of items via drag and drop is not implemented in BWOrderedArrayController itself, but in the project download, an example of how to implement reordering is given in the MusicLibraryDocument class

Inner workings

Most of the nitty gritty of how the code works is documented in comments in the code itself. The tricky parts were:

Using the code

The code is freely available under the MIT license and can be downloaded from this link. The download consists of an example project which demonstrates a very very basic data model with two entities: Playlist, and Track. The Playlist entity has a to-many “tracks” relationship and a “tracksOrdering” property to contain the ordering information. The demo is a document based app that lets you create new playlists, drag files from the Finder into the track list to add them, sort the tracks in order, and rearrange the tracks in the list via drag and drop. The idea is loosely modeled after iTunes, but the demo doesn’t actually filter out non-music files, or play music, or anything else useful. It is a demo, after all. :)

The BWManagedObject class can be used as-is and has no dependencies, and BWOrderedArrayController only depends on BWManagedObject, so you should be able to just add them to your project and compile them (do make sure to link to CoreData.framework, of course).

I have not yet tested this code extensively, so use at your own risk. In particular, I have no idea yet how it performs with larger data sets or other variations in configuration. I welcome all feedback and bug reports at bwebster@fatcatsoftware.com. If I get enough interest, I may even put the code up in a public VCS of some sort, for now I’ll stick with the zip file.

The code will only work on OS X 10.5 and later, the primary reason being that transformable attributes are only supported on Leopard, which is what the ordering attribute must be defined as. This is possible to do under Tiger as well, but requires a lot more code, which I didn’t feel was worth the time to write. Apple outlines what needs to be done in their documentation though, so it should be possible to backport if you need it to work under Tiger. Most of the other code should be mostly Tiger compatible (it doesn’t use new ObjC 2.0 features, for example), but may contain other Leopard gotchas.

Fixing Applescript

Posted on June 3, 2008 by Brian Webster
Filed Under Development, Programming | 2 Comments

After reading Daniel Jalkut’s post about replacing/supplementing Applescript on the Mac with Javascript (or perhaps another scripting language such a Ruby or Python), it got me to thinking about what exactly it is about Applescript that tends to trip people up. Would it be possible to figure out what the problem areas are, and just fix Applescript in those areas? There would be problems to this approach, the most prominent one being backwards compatibility. Some of the problems are fundamental enough that, if you fixed them, it would cause existing scripts to break. I believe Apple actually had plans a few years back of making a “new” Applescript that mostly acted like current Applescript, but was in fact a separate language and made an explicit break with previous versions of the language. Kind of like Carbon, where 80% of the stuff would work fine, but that problematic 20% would need rewriting under the new system.

So, what exactly are the problems that vex Applescripters so? There are quite a few, but here are some of the ones I can think of.

Lack of basic language features


Many modern scripting languages come with a fairly hefty set of built in data types and functions (or classes and methods for OO languages) that support a wide array of built-in functionality. Applescript has its own set of functionality, but for many standard tasks, it frequently either lacks the ability to do so at all or supports things but in a very hard to use or unintuitive way. A few examples include:

This is just a small sample, I’m sure there are many other things that are pretty standard in other languages that are absent from Applescript, or hidden away in the Standard Additions scripting dictionary somewhere.

File references are the devil



There are at least 5 different ways I can think of to refer to a file using Applescript (alias, POSIX file (a.k.a. file URL), Finder style object specifier, POSIX path, Carbon path). Different applications use different data types in different places, and will often barf back errors if you don’t use just the right kind of file reference. Figuring out how to translate from one kind to another is often maddening. Here’s an example from one of my own scripts, which uses three of the five types in a single line:


set helpFolderPath to POSIX path of ((folder "Help" of folder "en.lproj" of ptFolderAlias) as alias)

Files really need to be treated as first class citizens, with support built in to the language, and without needing to rely on the Finder for all file system access. Or wait, am I supposed to rely on System Events now instead? Or maybe a “do shell script” call to ls on the command line? Oh, the pain.

Scripting dictionaries



This is one of the biggest hurdles that a lot of beginning scripters have with Applescript, is understanding how scripting dictionaries work. Tools like Script Debugger are extremely helpful when it comes to exploring an application’s dictionary (I hardly look at the dictionary anymore these days, and just go right to the explorer pane and start drilling down), but while $199 is a fair price for a developer, most people aren’t going to want to drop that sort of change just to learn Applescript. Script Editor did get a pretty good upgrade in Tiger, but I think it needs more work to help beginning scripters understand how to explore and use scripting dictionaries.

The death of recordability

Another invaluable tool for beginning scripters which is all but dead is application recordability. Back in the OS 9 days, you could hit the “Record” button in Script Editor and go perform operations in other applications, and the equivalent script commands would magically appear in Script Editor. This was a great way to quickly learn the commands to use to do certain things you already knew how to do using the GUI of the application. Since OS X, recording has basically gone away. The only applications I know of that are still recordable are the Finder and BBEdit/TextWrangler. Cocoa has zero support for recordability. Theoretically, you can use the Carbon APIs to do this still. I actually tried once, and gave up after a couple days of banging my head against the wall. I’d love to see recordability come back, with a whole new approach if necessary, with full Cocoa support.

Applescript does have a lot going for it, but is hamstrung by a lot of these types of issues and prevented from reaching its full potential. Replacing it with another language is one way to go, but fixing Applescript also has a lot of potential. This above list is just a sample of some of Applescript’s shortcomings (if you have your own pet peeves, feel free to express them in the comments). The idea of redesigning Applescript from the ground up, with nearly 20 years of experience to build on, is very appealing, but who knows if Apple will ever have resources to devote to a project like that.

Optimization for Dummies

Posted on November 23, 2007 by Brian Webster
Filed Under Cocoa, Development, Programming | 2 Comments

A somewhat inconspicuous looking article on optimizing third party code has stirred up quite a conversation/flame war among many in the Mac developer community. The comment thread on the article is already quite long, and it seems to me that the people with opposing viewpoints are just talking right past each other at this point, so I thought I’d add in my own perspective here.

The jist of the article is that the author, Ankur, needed to draw some gradients in one of his applications, downloaded the source code for CTGradient, an open source library that provides a class to draw various types of gradients, and decided that it had way more functionality than he needed in his own program. So, he went through the code and basically removed everything he didn’t need for this single application. That’s all fine and good, and the article is actually an interesting look at performing various refactorings and dead code removal.

The problem arises in that he implies that if a developer decides to use some open source code and doesn’t go through and strip out every ounce of functionality that they’re not immediately using, that means that they’re encouraging “code bloat” and that their code is not “optimized”. Several commenters asked what kind of performance/memory gain he actually saw, and the only numbers he provided were from the “Real Memory” column in Activity Monitor, which is a pretty crude measure. The conversation went downhill from there.

I think one main point of miscommunication here is over the terminology the original author uses for some of the things he’s talking about. When you talk about “optimizing” code, I, and most other developers I know of, think of making the code run faster. The author reinforces this by stating “…you can optimize this thing till it runs like a Ferrari”. Certainly sounds like he’s talking about making the code faster, but the vast majority of what he’s doing is simply stripping out code that he’s not going to be using. This has pretty little effect in and of itself – it saves a few KB in disk space, and if the code truly is unused, it probably won’t even get paged into memory in the first place. There are a few places where the code probably runs faster, but the gains look pretty minimal in the big picture. However, arguing the nitty gritty details about his particular optimizations is missing the bigger point…

Engineering is all about tradeoffs, and in computer software, this typically means choosing between things such as memory usage, disk usage, CPU usage, and so on. Every one of these, however, inevitably comes up against the restraint of development time. You can spend days, weeks, or months optimizing your code in various ways, but it’s all for naught if you don’t eventually ship your application. This means that you can’t do everything, and have to pick and choose what areas of your code to work on, whether to add new features or shore up existing ones, and how much time to spend optimizing performance and memory usage.

What is conspicuously missing from the article is any sort of evidence that CTGradient was actually causing any sort of performance or memory problem in his application in the first place. Now, may more have gone on that he didn’t include in his write-up, but it sounds like he simply looked at the code, decided that it was obviously too big and bloated, and set to work spending quite a bit of time hacking it down to the bare minimum he needed.

Finding and fixing performance and memory problems in real applications, however, is rarely so simple. It’s rare that you can glance at a piece of code and immediately deduce that it’s going to be problematic for your application, causing slowdowns or whatnot. Most such bottlenecks are discovered as a result of rigorous testing, using tools provided by Apple such as Sampler, Shark, and Instruments (among others) to dig into the details of what your app is actually doing and where it’s spending its time. Upon discovering such a problem, you can then go in and spend your precious time fixing what most needs to be fixed. It’s not that the modifications he makes don’t actually reduce code size and memory footprint (they do) or increase performance (still not really sure about this without empirical data), but with testing first to find what needs fixing, the time spent doing all this could very well have been better spent fixing something that actually needs fixing.

I actually use CTGradient in a couple of my projects, and I use the code basically untouched. This is not because I’m “lazy” and would rather count my customers’ money while cackling evilly than spend the time to strip out everything I don’t use, but rather because I have plenty of other things to spend time on, optimizing my app in ways that make a difference, and adding features that people want and need. None of my tests of my drawing code have ever shown any performance problems arising from using CTGradient as-is, so my motto is, if it ain’t broke, don’t fix it. The critical flaw in Ankur’s argument in his post is that he never showed any evidence that anything was broken in the first place.

Miscellaneous Leopard development gotchas

Posted on November 1, 2007 by Brian Webster
Filed Under Cocoa, Development, Programming, Tips & Tricks | Comments Off

I’ve been using Xcode 3.0 under Leopard to do my development since Leopard came out last week, and thought I’d share a few changes that tripped me up.

Well, that’s what I’ve got for now, I may add more to the list later as I come across them. Hopefully this will help somebody out there (or maybe myself in a couple months when I forget and have to relearn this stuff over again).

Subclassing NSSortDescriptor for fun and profit

Posted on September 7, 2007 by Brian Webster
Filed Under Cocoa, Development, Programming | 1 Comment

In the project I’m currently working on, I’m using a pretty standard NSTableView + NSArrayController setup using the default sorting functionality. However, I found myself wanting to do some custom sorting, i.e. not just the usual sorting provided by @selector(compare:). This is made a little more complex by the fact that the columns of the table can be dynamically added and removed, so it’s not quite as simple as just changing the sort selector in Interface Builder. On top of that, I later found that I needed more customization still, because I needed to do special handling of nil values that was different from NSSortDescriptor’s behavior.

So, I set off to make a custom subclass of NSSortDescriptor. I can just override compareObject:toObject:, easy as pie! Right? Er… not quite. There really isn’t any information at all in the Cocoa docs about subclassing NSSortDescriptor, and there are a few gotchas that aren’t necessarily obvious when implementing the subclass, especially if you’re using it with an NSTableView.

To begin, I made my subclass (ITSortDescriptor) and overrode the compareObject:toObject: method. This worked fine to begin with, but I discovered that if I clicked on a table column to change the sort order of the table, it would soon revert to the default sort behavior. After some debugging, I discovered that the reason for this was that NSTableView was making copies of the columns’ sort descriptors using the NSCopying protocol. NSSortDescriptor implements NSCopying, but apparently its implementation will always return an instance of NSSortDescriptor, even the object being copied is an NSSortDescriptor subclass. So, I implemented copyWithZone: in ITSortDescriptor to return an instance of the subclass rather than a plain NSSortDescriptor. The implementation is pretty straightforward:

- (id)copyWithZone:(NSZone*)zone
{
return [[ITSortDescriptor alloc] initWithKey:[self key] ascending:[self ascending] selector:[self selector]];
}

This solved the problem for some cases, but there were still times when the column would revert to its old sorting behavior. I narrowed it down to when the table was already sorted by the column in question, and then you clicked on that column again to reverse the sort order. Apparently, NSSortDescriptor’s implementation of reversedSortDescriptor suffers from the same problem as its implementation of copyWithZone:. Overriding that in a similar manner then fixed that problem:

- (id)reversedSortDescriptor
{
return [[[ITSortDescriptor alloc] initWithKey:[self key] ascending:![self ascending] selector:[self selector]] autorelease];
}

To be really cool, instead of having [ITSortDescriptor alloc], that could instead be [[self class] alloc], but that would only really be useful if that’s what NSSortDescriptor did itself.

So with those modifications, I now have my table sorting just the way I want it. I thought I’d blog this so that anyone else who came across this stuff and was banging their heads against the wall might find it and save themselves some pain.

Feed