Building for Open Source

A Multiple Repository Architecture

It started with an email from Wink entitled "Help :)" - after joining Aletheia Ware's Open Source Community he had run into some confusion about all the repositories and dependencies.

Aletheia Ware uses multiple repositories to split code into many pieces each with a specific purpose. The alternative approach is a monolithic repository where all code related to a project is stored in a single place. Choosing between these two architectures has been a topic of hot debate among the software community.

The Perspective Android app is made up of 12 repositories:

Similarly, the S P A C E Android app is composed of 13 repositories:

Compartmentalizing and isolating code allows for better testing and code reuse; Common Libraries are shared between Perspective and S P A C E. Future games can share Joy while future blockchain apps can share BC, Alias, and Finance.

This architecture unfortunately creates a challenge for developers joining the open source project. Whereas my development machine already has all the repositories synced, scripts to perform the build steps, and symbolic links to chain the build artifacts; other developers would need to sync each of the 12 repositories separately, build each in turn, and make the output of each step available to the next.

Wink addressed these issues by;

• introducing a new repository, PerspectiveSuite, which uses Git Submodules [1][2] to pull all the relevant repositories together into a single monolithic repository. A submodule points to a specific commit in another repository and needs to be explicitly updated, which offers two advantages;
  • submodules can be 'locked' to a specific version.
  • changes across submodules can be committed to the suite atomically.
• creating a directory within PerspectiveSuite to house the build scripts which could then be executed by Gradle.
• removing the symbolic links in favour of relative links which worked both with and without the suite.

The developer workflow is now much simplier:

            # Clone
            git clone git@github.com:AletheiaWareLLC/PerspectiveSuite.git
            # Clean
            ./gradlew clean
            # Sync
            ./gradlew sync
            # Build
            ./gradlew build
            

To create such a suite for your project 'FooBar' first initialize a new repository, add the submodules, and push to a remote:

            # Initialize repository
            git init FooBarSuite
            # Add submodules
            git submodule add git@github.com:Username/Foo.git
            git submodule add git@github.com:Username/Bar.git
            # Add remote
            git remote add origin git@github.com:Username/FooBarSuite.git
            # Push
            git push -u origin master
            

Submodules give the best of both worlds - each repository is highly specific with a single purpose but multiple repositories are linked together into a single suite.