Building services with lots of code becomes a problematic issue when your application’s scale gets larger. Most of the projects suffer from difficulty of making changes to the codebase after some time, which causes instability of features. Therefore, your application components should be atomic and easily modifiable. There are several ways doing this and most of the big companies utilize these methods.

Typescript has lots of advantages on implementing the modularity, since most of the written code is usable in multiple projects. Additionally, it is quite easy to manage package dependencies with Typescript thanks to simple package managers, such as npm and yarn, which are well-known and pnpm that is relatively new.

Submodules

The concept behind submodules is simple: You have multiple repositories, which are referenced by and located inside a single main repository. This brings a nice level of abstraction of packages, and you are able use multiple packages in the same codebase, also referencing them in other places. The submodule repository becomes part of the main repository, similar to a folder that really exists inside.

You can run the command below inside your main repository to initiate the connection with submodule. Note that the examples are given using GitHub; however, this concept works for other popular git version control systems.

git submodule add https://github.com/asgarovf/submodule_name

The connection is controlled with .gitmodules file, which is automatically added to your main repository after you initiate a submodule. This file has the following structure:

[submodule "submodule_name"]
	path = submodule_name # Path to your submodule in main repo
	url = https://github.com/chaconinc/submodule_name # Remote url 

You will see that reference to submodule inside the main repository will be created as another folder, but the folder will have no files although your submodule has content inside. The content should be synced manually by running the command below:

 git submodule update --init --recursive

What we emphasized about the word similar above is a point where problems behind submodules arise. To reflect the changes for each submodule, you need to consistently update the repositories by running an update command shown above. Otherwise, your main repository will not take the changes from the submodules. This causes a huge problem when your submodules gets updated consistently by multiple people. Making a small change on submodules require a lot of effort to test.

Still, the submodules works fine for managing big repos, but requires some additional work to do for synchronization.

Monorepo

Monorepo steps up when the synchronization issue arises and easily solves it with the power of Javascript package managers. This structure is similar to submodules. However, with a monorepo, the packages of your repository is actually living inside as a directory and can be accessed directly without any effort.

Assume that you have package called X, which lives inside the monorepo. Your changes to package X will directly affect to all other packages which depends on package X.

You are able to use all the package managers to achieve a monorepo structure, but npm is quite simple and easier to manage than its alternatives.

Getting started with Monorepo

There are multiple ways to easily start with a monorepo structure. Multiple open-source tools are created to simplify the Typescript monorepo creation and maintenance. Most popular ones are Lerna, Turborepo, Rush and etc. We as Clave, are using Turborepo to manage the apps and packages and I will give examples and mention important points on managing them.

You can get started with Turborepo, using the documentation provided in the link below. It gives a broad introduction to setting up a new new monorepo, also allowing you to add it to existing repositories.

After setting up a monorepo, you can use shortcut commands provided by Turborepo to build, run and publish different packages from the same command line.

Some Disadvantages

• There might be various optimization issues because of the monorepo structure, since you collect all the packages and apps to same place. Although the tools like Turborepo is quite optimized, it is still a long process to install all the dependencies for monorepo, especially if your repo grows with lots of separate packages and apps.

• It is a bit problematic to maintain the versions for multiple packages, since the caching mechanism of the package managers can sometimes install different versions than the defined versions in package.json

• Difficult to manage commonjs modules, since import and export commands are not supported and whole monorepo mostly built with ESModules. You need to build the packages which are used in commonjs modules with tsc command if you want to import them.

Some Advice

While maintaining more than 10 packages and apps in Clave, we faced some issues related to monorepo, which are not exactly documented. Our services include packages for React Native, NestJS, NextJS, Create React App, Expo Modules and more. Since each of these packages have different Typescript configurations, it sometimes becomes difficult to manage.

• Put your package-lock.json in .gitignore, since it causes dependency inconsistency and merge conflicts, if you are working on same projects at the same time.

• Always try to use same versions for all third-party packages. It means that if your app X uses React version of 18.2.0, keep it exactly the same for app Y if it also depends on React.To make sure that each app/package of your monorepo has the same version, check the node_modules which are located inside each package. If the node_modules for each package is not created, it means that the versions match with other app/packages. If node_modules exists inside a package separately, it means that there are different versions for a package installed in monorepo. For example, in the image below, you can see that the typescript package is installed separately and have different version. You should try avoiding this behavior as much as possible to keep your monorepo structure clean.If this behavior keeps happening even if all package.json files have the same versions for a third-party package, try removing all node_modules inside your monorepo and package-lock.json, and installing the dependencies in root level again. You can run the command below in root of your monorepo to clean all the node_modules and package-lock.json.

find . -name 'node_modules' -type d -prune -exec rm -rf '{}' + && rm package-lock.json

• Do not forget to add third-party dependent packages to package.json of each app inside monorepo that utilizes that package. Once you install a third-party package first time on any app/package, it gets available for other app/packages too and can be imported everywhere. However, it might a problem while build the packages separately. You need to interact with each app/package of your monorepo as an individual service and make sure that it works without the dependencies which are installed by other app/packages.

Open Sourcing

Keeping everything in a single repository is great and increases your development speed a lot. However, when it comes to open-sourcing projects it is a bit different. You might want to open-source only single part of your app. This might be your business choice or need for documenting some parts in a better way before going open-source.

We recently started open-sourcing some packages that we built for Clave. We don’t want to keep the copy of the each package and maintain in a separate repository. It is difficult to accept open-source contributions to different packages if git history is diverged. Therefore, we should somehow keep the codes at the same place, also being able to push to different repositories.

Publishing an npm package is exactly the same as sharing a package outside of monorepo and you don’t need additional stuff to do. You can check articles about publishing npm packages.

We are using git subtree commands to achieve sharing codes to different repositories from Monorepo. You need to follow the steps below to push changes to another remote.

Let’s assume you are sharing the clave-corea separate Git repository, which is located in packages/clave-core path in the monorepo.

1. Commit the changes just like before using git commit -m 'message'

2. Make sure that remote is set correctly by running git remote -v

   1. You should be able to see a remote URL that is mapped to correct git repository

   2. If you can’t see the upstream, you can add by running git remote add upstream-{NAME} {GIT_REPOSITORY} where NAME is core in our case and GIT_REPOSITORY is url of your git remote.

   3. Push to a subtree using the command below:git subtree push --prefix=packages/clave-{NAME}/ upstream-{NAME} {BRANCH}

   4. You can pull the changes from repo with the command below:git subtree pull --prefix=packages/clave-{NAME}/ upstream-{NAME} {BRANCH}

Conclusion

The main goal of this article is introducing you with Monorepos and share some experience and difficulties we faced while developing Clave. You need to remember that the best approach in version control is always the one which is comfortable for you. We believe that it is one of the easiest ways to manage multiple apps/packages in one single Github repository with a tool like Turborepo.