The state of iOS Open Source – and what to do about it!

There is a vibrant community of open source projects for iOS. You need a calendar UI components or a JSON parser? No problem, the projects are out there. Most code out there is of very high quality. Unfortunately the distribution of the code is generally very crude, barely half a step away from sharing code on disks or uploading a ZIP-archive to a BBS.

The Current State of Affairs

Most Open Source projects for iOS are distributed as a source code repository with an Xcode project. But here the good news stop for most projects. The Xcode project usually only has one target, a sample project, mixing the reusable code in the same bucket as the sample/incubator code around it.

The usage instructions suggest you drag and drop a directory, or even worse a collection of files, into your own project. This is a very primitive solution, passable in the ’80s when all we had was disks and BBS:es, not good enough today. All history and connection back to the source is lost, any value added by source control systems is lost the second you decide to use an open source component this way. Any bug-fix or improvement you make will require a manual merge back to the original project. You also loose all tracking information the moment you copy a file, what version of project X did you use, some special branch?

This Need To Change

This affair also highly discourages re-use. Almost every open source project out there that involves network connectivity also uses Apple’s Reachability example code as a base for determining network status. Apple’s code is not complete for all use-cases, and not free of bugs, so almost every project has a more or less modified variant of the same code causing compiler errors. Ensuring that using several open source project in one application is bound to end up with conflicts. Leaving it up to you to determine which projects version of Reachability that is the master version, often you end up writing a third version of Reachability for your project. And none of these changes and improvements will ever end up in a re-usable way for the iOS community as a whole.

This is not an impossible problem to solve, in the Java world there is for example Maven that solves this very well. Maven is a huge tool, and re-inventing that wheel for iOS would be a mammoth task. Fortunately that is not needed; Xcode already has the basics that we need in order to create iOS projects that are highly extendable and re-usable.

There is a Solution Today

One Xcode project can have one or more targets, usually the target is your application, sometimes you have one target for iPhone and a separate target for iPad. Targets are not limited to applications, you can also have a Unit-test targets, and more importantly static library targets.

A static library target yield a static library that can be linked into any other target, so be applications or other static libraries. A static library is a cruder version of a Framework, like UIKit and Core Data that Apple provide us. Crude as they are, it is still good enough to use for sharing re-usable Open Source project components.

A Xcode project has the notion of sub-projects. A Xcode sub-project is a reference to another Xcode project. You can set the sub-project’s Targets as direct dependencies for your targets, and instruct the sub-projects products to be linked with your products. In essence say “Use what that other Xcode project creates to build my project”.

The idea of Xcode’s sub-projects has a logical one-on-one mapping to sub-modules in Git. Meaning that exposing re-usable iOS Open Source projects as static libraries on for example github is as easy as can be.

Eating Your Own Dogfood

This is actually what I have done for years, ever since I released HessianKit in 2008. Most of the code you write will be re-usable for the next project, having an easy way group your re-usable components into a growing library will save you a lot of work.

Firstly I never have to start with a blank canvas for any project. But more importantly, when working on many applications, any bug fixes and improvements I make to my shared components immediately trickles out to all other projects as well. If I add animated insert of cells in my column table view for project A, then all other projects is only one git pull away from benefiting from this new feature as well.

Jayway have since a few weeks begun the process of publishing our re-usable components for iOS. So far this has resulted in three projects on github:

  • CWFoundation – a collection of new classes and categories to add functionality that is commonly missing from Foundation framework.
  • CWUIKit – a collection of classes and categories that do the same for UIKit. CWUIKit requires CWFoundation.
  • CWCoreData – A collection of categories for Core Data that makes using Core Data in a multithreaded application a breeze.

These are living projects and will be updated with new functionally over time, and large enough to warrant a blog-post for each on their own. There are also a few Componentized forks of popular iOS open source projects at Jayway’s github page.

Tutorial for Using a Sub-Project from a Sub-Module

Working with an environment like this have many benefits, but also requires some knowledge of both Xcode and git. There are a few gotchas to be aware of, limitation of Xcode and the linker, best explained with a short tuturial.

CWUIKit has additions to UIAlertView that allows an alert to be created directly from a NSError instance, just like AppKit do on Mac OS X. Lets walk through how to create a new project that uses this functionality from CWUIKit to display the error message from a faulty URL request.

This tutorial is for Xcode 3.2, but works with minor modifications for Xcode 4. Changes for Xcode 4 are written in block quotes like this.

1. Create new “Window-based Application” from Xcode.

I chose a Window-based app because it contains the least amount of template code, and fits the purpose of an example well.

2. Initialize the project directory as a git repo.

We need to initialize the project directory as a proper git-repo in order to use git’s sub-modules.

Skip this step if project is created with git-repo in Xcode 4.

3. Add sub-module

This will create a sub directory named Components, this is where I by convention put all my sub-modules/sub-projects, it is not a hard requirement.

4. Update and initialize all submodules

A git-submodule is by default not initialized, this is to avoid downloading large amount of data for a sub-modules you do not need, such as a test project. This is why we must update and initialize the sub-projects to realize them when starting a new repo or cloning an old one.

5. Add sub-project


Locate CWUIKit.xcodeproj and drag and drop into your project.
Highlight the CWUIKit.xcodeproj item and ensure the “Target Membership” column, that has a bullet as symbol, is checked for the libCWUIKit.a product. This is the static library to link against.
Also add the CWUIKit target as a direct dependency to your applications target so that CWUIKit is automatically build before building your application.

It is not required to setup target membership in Xcode 4, only set Target as Target Dependency in Build Phases tab, and add the Product to be linked in the same tab.

6. Recursive Sub-Projects

Static libraries are by their nature linked into your binary, this means that if static lib A and static lib B both define the same global symbol C you will get a duplicate symbol ‘C’ error when linking you app.
CWUIKit uses functionality from CWFoundation, but because of this limitation of the linker it can not explicitly add this functionality to itself. Doing so would make it impossible to use CWUIKit side-by-side with other projects that also uses CWFoundation, such as CWCoreData.
The work around is that any project that includes CWUIKit must also include CWFoundation. So repeat step 5 with the CWFoundation.xcodeproj from /Components/CWUIKit/Components/CWFoundation.

Xcode 4 has full knowledge of recursive sub-projects, so this step can be skipped. Only add CWFoundation sub-projects product to be linked.

7. Add QuartzCore and OpenGLES frameworks to your target.

Static libraries can not enforce linking of frameworks on it’s own, so building the app in it’s current state would result in missing symbols errors.
Using CWUIKit requires the app to be linked against QuartzCore and OpenGLES frameworks for CoreAnimation and OpenGL additions.

8. Setup header search paths.

Unfortunately sub-projects headers are not automatically added to the search path of your projects. So you must add “Components/**” as a header search path for your project.

Xcode 4 considers header search baths to be an advanced setting, to change the build settings display for Basic to All if you can not find it.

9. Setup linker flags

Add “-all_load” to other linker flags.
The GCC and LLVM linker also has a limitation/bug for static libraries that will strip out any methods for categories on a class that is not defined in the static library. For example any method extending UIAlertView in libCWUIKit.a will be stripped out by this bug.

10. Add #import "CWUIKit.h" to you application delegate. And add this small code to the application:didFinishLaunchingWithOptions: method

11. There is no step 11.


But there is a screenshot of the running app.

Conclusions

This method of using Xcode sub-projects and git sub-modules for managing shared components works very well, and scales to be even better the more sub-projects and developers you have. The greatest benefit is without any doubt that bug-fixes and new features by default always trickles back to their source and become re-usable for all.

All the downsides are in the limitations of static libraries themselves, and can unfortunately not be overcome unless Apple introduces proper support for third party frameworks in iOS SDK. The two major gotchas are missing symbols caused by missing frameworks required by dependencies, and duplicate symbols caused by implicit inclusion of dependencies from dependencies. I have found that the best solution is to always have a README with your project where you explicitly name each Framework and Component that is required.

Using Xcode sub-project to manage your builds solves the problem of dependency resolution without the need to write a new system, works on the developer’s desktop as well as on a build server. Using git sub-modules to manage your dependencies solves the problem of including all recursive dependencies as well as versioning. Any sub-module can by locked at a particular stable version by commit or tag as required, with full history. But best of all; you as a developer is free to just code away knowing that you can later commit and push your changes back, independently of how or where the code originates.

16 Comments

  1. Really interesting, will definitely use this method in the future. I am just afraid that my projects gets cluttered up with sub projects. So that I can’t tell what project I am working on, But with the benefit of only having to do the change in one place definitely overrules the added complexity.

    Thanks for a nice post :)

    • The size of the sub-projects is a balance. We started out with many small single purpose sub-projects, but have now merged them into fewer more generic sub-projects.

      Many small sub-projects encourage a single responsibility, and reduces risk of breaking related functionality. But we quickly realized that the cost is that developers just can not be bothered to create new sub-projects, or even search for existing ones. And a system that is not used, is worse than no system at all.

      Developers are lazy, a fact you have to live with and even embrace. Our current rule of thumb is that anything smaller than an EPub reader or say ASIHTTPRequest goes into a generic sub-project. You are allowed to break the rules if you understand the rules :).

  2. Daniel Hjort

    Excellent! I’ve been using the approach myself a little and used some static libraries as git submodules from Github. Works great! Like the way you paint the big picture of what benefits it could bring for all of us.

  3. Erik Sundin

    Great blogpost! We have a similar setup but with Subversion, using svn externals to check out dependencies as part of an application project. It works ok, but I sure miss maven :-)

  4. Great post! What are your thoughts on Workspaces in XCode 4?

    • I have too little experience with Xcode 4 on larger project setups to say anything definitive. My initial thought is that workspaces in Xcode 4 are a more visible solution to the same problem that sub-projects already solves. There is a large overlap in functionality, especially in Xcode 4 where sub-projects are better integrated in the parent-project, shared indexes and all. What workspaces brings to the table is resources not directly part of the product, and cross project refactoring. For now I say workspaces allow for sub-projects v1.1.

  5. In my projects, I’ve found that setting User Search Paths (in XCode 4) causes auto complete to break. When set, I can’t autocomplete anything in the main project, disabled and the headers included it works fine. Every run into this?

  6. Magnus T

    Good walkthrough.

    I have sub-projects working well but I cannot figure out a nice and flexible way to make .png and .xib resources in the subproject available for the main project/app.

    Any suggestions? Have seen pointers to “add copy step in build phase” but no solution yet…

    /M

  7. thats a long post actually so i havent read it all but the topic attracted me so bookmarked it i’ll post the feedback after reading it till the end

  8. I had been wondering if your hosting is OK? Not that I am complaining, but sluggish loading instances times will sometimes affect your placement in google and can damage your quality score if advertising and marketing with Adwords.

  9. Excellent critique, I liked the electronics technician certification part

  10. Well I searched for this title and found this,
    never thought i’d find the answer

  11. This is really interesting, You are a very skilled blogger.

    I’ve joined your rss feed and look forward to seeking more of your fantastic
    post. Also, I have shared your web site in my social networks!

Trackbacks for this post

  1. Link dump for July 26th | The Queue Incorporated
  2. My daily readings 08/03/2011 « Strange Kite
  3. The state of iOS Open Source | Niklas Saers' blog

Leave a Reply