Big Behind-the-Scenes changes for SDL 1.2.14 on Mac OS X (Snow Leopard)


SDL logo

At least from my perspective, there has been a surprising amount of work and effort put into the (upcoming) maintenance release of SDL 1.2.14 for the Mac. Many of the changes have been focused on modifying the Mac code base to support Snow Leopard, particularly 64-bit. As I understand it, Ryan Gordon has spent the time cleaning up the codebase with help from user patch submissions in the SDL Bugzilla to modernize the codebase. (Thank you to all of you that have submitted patches.)


Meanwhile, I have been focusing on modernizing the Xcode projects, Xcode application templates, doing testing, updating documentation, and working on the official binary distribution.


So there are some fairly substantial behind the scenes changes Mac developers should be aware of which I will try to consolidate here. (I have also documented all this through out our SDL documentation, but it is scattered about in different places.) 


Table of Contents


64-bit Universal Binaries

As described, this is probably the biggest change to SDL for Mac. The backend has been changed to no longer use deprecated APIs not available in 64-bit. And for the binary release, we have a 3-way Universal Binary (Intel 64-bit, Intel 32-bit, PowerPC 32-bit).

There is at least one API SDL depends on (relating to power management APIs) that Apple did not finish porting to 64-bit until Snow Leopard. This means it is impossible to use SDL as 64-bit on (non-Snow) Leopard or earlier. (You will also need Snow Leopard or later to compile your app as 64-bit.)  Note that since Snow Leopard does not support PowerPC, we do not build a 64-bit PowerPC binary.

There is a corner case you need to be aware of. Because SDL 64-bit will not work successfully on anything prior to Snow Leopard, you need to take some precautions to prevent the 64-bit architecture from being launched on pre-Snow Leopard. Since (regular) Leopard had 64-bit support built into the OS, if you deploy a SDL Intel 64-bit binary (compiled in Snow Leopard) and somebody tries to run it on an Intel running (non-Snow) Leopard, the 64-bit architecture will be detected and will be launched by default which will likely result in some kind of failure. (Also note that even Tiger had preliminary 64-bit support.)


To prevent this you have several options:

1) Don't deploy a 64-bit binary

2) Tell 64-bit Mac Intel users that they must be running Snow Leopard

3) Force the 32-bit architecture to be the default using a Launch Services key (requiring the user to manually change this themselves if they want 64-bit)

4) Set a Launch Services key that instructs the computer to launch the 64-bit architecture on Snow Leopard or greater.


The last option is the best option in my opinion. To do this, you must add some lines to your application's Info.plist.

<key>LSMinimumSystemVersionByArchitecture</key>
<dict>
<key>x86_64</key>
<string>10.6.0</string>
<key>i386</key>
<string>10.4.0</string>
<key>ppc</key>
<string>10.4.0</string>
</dict>


If you are using Xcode to edit the Info.plist, the category name is "Minimum system versions, per-architecture".

Minimum system versions per-architecture

If you use our new SDL/Xcode Application Templates for Snow Leopard (and also Leopard), we set this up for you so you don't need to worry about it.


10.4 (Tiger) as the new minimum OS requirement

Yes, we've heard all the complaints about this one, but we've made the decision to move on. There are multiple reasons for this switch. The first reason is a lot of old APIs we used prior to the 64-bit cleanup have been deprecated for a long time and didn't make it over to 64-bit. Most of the APIs started stabilizing around Tiger and they were fairly decent APIs. Furthermore, it was during Tiger that the Intel Macs were introduced and Intel Macs now seem to constitute maybe 85% of the Mac market.

Second, Apple has removed the 10.3 SDK and gcc 3.3 from Snow Leopard's Xcode so it is extremely difficult to compile for anything less than 10.4. If you recall during SDL's transition to Universal Binaries for 32-bit Intel back in the Tiger days, you may recall a lot of fussing and ugly things we did to the Xcode projects to maintain 10.2 compatibility. One of the major issues was that gcc 4.0 started dynamically linking to a core library (libgcc_s.dylib) and this library did not exist prior to 10.3.9. This forced us to build PowerPC with gcc 3.3, but compile Intel with gcc 4.0 (as 3.3 would not work on Intel). With the removal of gcc 3.3 from the tools, we can no longer compile for pre-10.3.9. In addition, with the removal of the 10.2 and 10.3 SDKs, we can no longer safely compile for older versions of the OS. Yes, we could play with the -mmin-macosx-version flag while using the 10.4 SDK, but this removes a compiler check that would tell us if we are accidentally using a newer API not available before 10.4. Since we just switched over all the code, and moving forward, we won't be thinking about these issues, (not to mention that none of us are testing on these older versions of the OS), it seems haphazard to to set this flag.

Third, on a related note, SDL_image has been switched over to use a new ImageIO based backend on Mac. ImageIO requires 10.4.

If you really need legacy pre-10.4 support, you are free to either try recompiling SDL yourself, or you can grab the prior release of SDL and rip out the PowerPC architecture of the binary and lipo it into the new binary. However, be aware that there are new bug fixes that you will not get on PowerPC if you do this. And of course, you always have the option of not upgrading to the new version of SDL.


Updated Xcode Project Templates

The templates have been overhauled and separated into Xcode/Mac OS X versions. The template substitution format changed since Tiger's Xcode, so we needed to modernize. In addition, only Snow Leopard can successfully build 64-bit Universal Binaries for SDL as described earlier so our Snow Leopard template is the only one that sets up 64-bit Universal Binaries.

For Snow Leopard, our template is setup to use the 10.6 SDK and gcc 4.2 for 64-bit, and the 10.4u SDK and gcc 4.0 for 32-bit. If you don't need backwards compatibility, you might just use the 10.6 SDK exclusively. Also, feel free to experiment with llvm-gcc and clang. If these work for you, they often generate faster code (and in less time).

SnowLeopardXcodeTemplateSettings

Note that we needed to remove the SDL Custom Cocoa Application template. It had a NSQuickDraw dependency which was an API  that was deprecated since the beginning of Mac OS X. This was the template that allowed you to render SDL in an NSView which could be placed anywhere within the context of a larger native Cocoa app. It may be possible to resurrect this feature, but nobody to my knowledge is actively working on it. So user contributions are greatly welcome.

We also removed all the Project Builder templates.

The Leopard and Snow Leopard templates define the LSMinimumSystemVersionByArchitecture key as described earlier. Note that the Leopard template does not build 64-bit as we require 10.6, so this key won't do anything for you, but I did it as a precautionary step in case people inadvertently use the Leopard template in Snow Leopard. However, the Tiger templates are almost unchanged from our prior template releases as I did not want to risk breaking things, so these do not set the Launch Services key.


For Leopard and Snow Leopard (Xcode 2.5, 3+), we recommend you install the templates to:

/Library/Application Support/Developer/Shared/Xcode/Project Templates/Application


For Xcode 1.0 to 2.4:

/Library/Application Support/Apple/Developer Tools/Project Templates/Appllcation 


Also note you may place it in per-user locations, e.g.

~/Library/Application Support/Developer/Shared/Xcode/Project Templates/Application


And for advanced users who have multiple versions of Xcode installed on a single system, you may put each set in a directory with the Xcode version number instead of using "Shared", e.g.

/Library/Application Support/Developer/2.5/Xcode/Project Templates/Application

/Library/Application Support/Developer/3.1/Xcode/Project Templates/Application

/Library/Application Support/Developer/3.2/Xcode/Project Templates/Application



Copy each of the SDL/Xcode template directories into the correct location (e.g. "SDL OpenGL Application"). Do not copy our enclosing folder into the location (e.g. TemplatesForXcodeSnowLeopard).

So for example, in:

/Library/Application Support/Developer/Shared/Xcode/Project Templates/Application

you should have the 3 folders:

SDL Application
SDL Cocoa Application
SDL OpenGL Application


After doing this, when doing a File->New Project, you will see the projects under the Application category. (Newer versions of Xcode have a separate section for User Templates and it will appear in the Application category of the User Templates section.)


Xcode 2.4 on Tiger

Xcode Tiger Templates
Xcode Snow Leopard Templates

Xcode 3.2 on Snow Leopard


Native Xcode Documentation Integration (DocSet)

Work is currently being done in SDL to support Doxygen, spearheaded by Kenneth Bull. And about a year and a half ago, Doxygen introduced initial support for a new documentation format for Xcode 3.0 or greater called DocSets (Documentation Sets). So now that we have the two foundational pieces, we can finally generate documentation that integrates nicely with Xcode.


In the SDL Developer Extras DMG (where our Xcode Templates also are packaged), we add the DocSet to the Documentation folder, alongside the (handcrafted) HTML and man pages. You will need to drill down into the XcodeDocSet directory and find the org.libsdl.sdl.docset bundle. We recommend you copy this to:

/Library/Developer/Shared/Documentation/DocSets


Again, this follows all the standard Xcode patterns you've seen with the templates. You may need to create the directories if they don't already exist. You may install it on a per-user basis. And you may target specific versions of Xcode in lieu of using the "Shared" directory.


To use, it is quite simple. Just bring up the Xcode Documentation Browser window (can be activated through the Xcode Help Menu) and start searching for something. 

Xcode DocSet


If nothing is found on a legitimate search, verify that the SDL documentation is enabled by opening up the DocSet popup box below the toolbar in Snow Leopard. (In Leopard, the DocSets appear in the left-side panel.) 

DocSetEnable


Another handy trick is to use the mouse and Option-Double-Click on a function or keyword to bring up documentation on the selected item. Prior to Xcode 3.2 (Snow Leopard), this would jump you to the entry in the Xcode Documentation Browser.

However, in Xcode 3.2 (Snow Leopard), this behavior has been altered and you are now given a hovering connected popup box on the selected item (called Quick Help). Unfortunately, the Doxygen generated DocSet doesn't currently provide Quick Help information. You can either follow a link to the main Documentation Browser from the Quick Help, or alternatively, you can bypass Quick Help by using Command-Option-Double-Click which imitates the pre-Xcode-3.2 behavior of Option-Double-Click. (Please file feedback with both Doxygen and Apple to improve Quick Help integration.)


QuickHelp

Quick Help



For those that want to tweak the documentation output, you can find my Doxyfile in the XcodeDocSet directory in the Xcode section of the SDL source code base. (You can also find a zipped pre-generated copy of the doc set bundle.)

One of the most significant options is "Separate Member Pages" which I disable. When disabled, the documentation is about 6MB. When enabled, the documentation is closer to 1.6GB (yes, gigabytes). Obviously, distribution will be really hard with sizes that huge so I disable the option.

I also disabled Dot because there didn't seem to be much benefit of generating graphs for public C functions.

One thing I would like to see is a CSS file that makes the Doxygen DocSet look more like the native Apple documentation style. Style sheets are outside my expertise so I am asking for contributions on this one. Meanwhile, I also request you send feedback to Doxygen and Apple about this issue too.


Finally for convenience, I have added a new shell script target to the Xcode project that builds SDL that refers to my Doxyfile and generate the DocSet we distribute.


Update: 2009-10-04

After more inquiries, I was reminded that I needed to set the auto-brief options in the Doxyfile for JavaDoc and QT. This at least allows the brief descriptions to be seen in Quick Help. However, parameter, return values, etc. still are not seen. The auto-brief changes are now checked in.


SDL Satellites

Some of the SDL Satellite projects (my nickname for SDL_image, SDL_mixer, SDL_ttf, and SDL_net) have also been affected by our recent changes.


SDL_image: New ImageIO Backend

We are changing the backend of SDL_image to use Apple's ImageIO API (when possible) instead of depending on 3rd party libraries directly (e.g. libpng, libjpeg). (This was a patch I submitted a while back.) Ultimately, ImageIO wraps libjpeg and libpng but Apple doesn't provide direct access to these libraries. Prior to this, we have been statically linking our own built 3rd party libraries into SDL_image, but this required a lot of effort on our part as we had to manage the 3rd party libraries ourselves to build Universal Binaries. Not to mention that this left us responsible for dealing with any security problems that might be discovered in the libraries.

In addition to freeing us from these extra responsibilities, ImageIO automatically gives the Mac backend access to more image formats such as JPEG 2000. And our binary size has shrunk by about 10X.

The catch is that ImageIO requires 10.4 Tiger or greater, but since we have decided this will be the new minimum requirement for SDL anyway, it works out.


SDL_mixer: No MIDI support

Because SDL_mixer depended on legacy/deprecated Quicktime APIs for MIDI support, we had to remove MIDI support from SDL_mixer. (Recall that we removed MP3 support quite awhile back during the Tiger/Intel transition as we never got SMPEG fulling working and embeddable into SDL_mixer using static linkage.) This could be fixed by adding a native Core Audio backend to SDL_mixer.

Update (2009-10-18): In the 11th hour, Ryan Gordon has implemented a Core Audio backend for MIDI. So MIDI is back. 

Also, we got took a stab at getting SMPEG support working so MP3 support is back as well. Thanks to our new 10.4 minimum requirement, I've leveraged @loader_path and created an umbrella framework so we can dynamically link and transparently hide SMPEG inside the SDL_mixer framework which avoids our previous static linking problems and also avoids breaking everybody's existing build system.

Also, due to SDL's new dual license scheme, we needed to dynamically link mikmod for tracker format support (e.g. MOD, S3M, etc.). The code base was also changed a bit as the libmikmod source code exists in an self-contained zip file. For distribution, I apply the same umbrella framework techniques I used for SMPEG. In the future, I may also apply these changes to Ogg Vorbis for consistency.

Update (2009-11-11): 

There is a bug in Xcode 2.5's (Tiger) linker/dyld. It will cause you to see unresolved symbol problems during linking for Mikmod and SMPEG symbols.
http://lists.apple.com/archives/Xcode-users/2007/Nov/msg00402.html

This problem only affects building/linking with the Xcode 2.5 toolchain. So for example, building on Leopard or Snow Leopard with Xcode 3+ or building with Xcode 2.4 on Tiger and then deploying the resulting binary on Tiger should not have any problems. It is only the act of building/linking with the Xcode 2.5 toolchain that causes problems.

If you cannot build on Leopard or Snow Leopard, or downgrade to Xcode 2.4, then add the following magic incantation to your link flags.

-dylib_file @loader_path/Frameworks/mikmod.framework/Versions/A/mikmod:/Library/Frameworks/SDL_mixer.framework/Versions/A/Frameworks/mikmod.framework/Versions/A/mikmod

-dylib_file @loader_path/Frameworks/smpeg.framework/Versions/A/smpeg:/Library/Frameworks/SDL_mixer.framework/Versions/A/Frameworks/smpeg.framework/Versions/A/smpeg

(This assumes you the SDL_mixer.framework is in /Library/Frameworks.)

Tech Note on -dylib_file here:
http://developer.apple.com/mac/library/qa/qa2007/qa1567.html


SDL_ttf: Dynamic linking to FreeType

Prior to this release, we have been statically linking libfreetype.a into SDL_ttf. The libfreetype.a came from Apple itself as we simply copied the file from where they installed it on Macs. We statically linked it because older versions of Mac OS X did not install the X11 binaries by default. But since Leopard, Apple stopped shipping a static version of FreeType, so we don't have a 64-bit version. Since we have made the decision to make 10.4 the new minimum requirement, we assume libfreetype.dylib is available and now dynamically link to that (also resulting in a smaller SDL_ttf binary size). If you use SDL_ttf as a static library, you will need to explicitly link your application against FreeType or you will probably get unresolved symbol errors during link.


Other Random Items

Here are a few items I wanted to mention because I was personally curious about these issues.

In my SDL_Clipboard proposal, I uncovered a bug in the BMP image loader in SDL. It would not handle negative height values which represents an inverted image. My patch has been accepted and merged into this release. 

In addition, we tested and discovered that this same bug existed in SDL_image as well and the patch has been adapted and applied for SDL_image too. 

Interestingly, since we are switching to ImageIO for the Mac backend in SDL_image, this bug would not be present with the ImageIO backend as Apple already handles this case correctly. But all the other platforms that use the common generic SDL_image backend will benefit from this fix now.


No Dual Mode (i.e. No Garbage Collection Support)

This was just a general curiosity thing for me. I was wondering if SDL would work compiled in garbage collection mode. SDL itself would not be garbage collected, but it would allow you to use SDL within a larger app that might be garbage collected. Since we used to have the SDL Custom Cocoa Application template that let you put SDL into an arbitrary NSView, this would not be an unreasonable thing to think of. 

But in my simple testing, gc mode leads to a pretty quick crash. So for now, gc mode does not work. If there are volunteers that would like to make this dual mode clean and bring back the NSView functionality we lost by moving to 64-bit, I would certainly like to see the contributions.


Please Test

If you reading this before the final SDL 1.2.14 release, please test! As described, there are a lot of changes, potentially breaking things (but hopefully not). Also, all these changes have been applied/duplicated to the SDL 1.3 branch, so test that as well.

As always, SDL can be found at:
http://www.libsdl.org/


I also want to remind people that I made several screencasts for OpenSceneGraph a few years ago dealing with Mac and Xcode-isms. They are based on the same concepts we use for SDL so I recommend you check them out if you haven't already. You can find them in my Screencasts section on this site.


Update (2009-10-19): SDL 1.2.14 has officially been released!


(Shameless Plug) My new book: Beginning iPhone Games Development 

I am co-authoring a book entitled Beginning iPhone Games Development, being published by Apress. We discuss APIs more native to iPhone and iPod Touch than SDL, though in my defense, there is more on OpenAL in this book than I think has ever been written before. (For those confused why I use OpenAL as a defense, it is because of the historical relationship it has with SDL.)

But regardless, I hope many of you might be interested in the book anyway. And at the very least, I hope the rest of you will indulge me as all of my contributions over the years to SDL have been unpaid and completely voluntary.

We are past the first-draft stage and expect the book to ship before the end of the year (hopefully much sooner). I will post more details on this site as details become finalized. Stay tuned!



Copyright © PlayControl Software, LLC / Eric Wing