Bundler: The best way to manage a Ruby application's gems

If you just want to know our recommended workflow, and don't care about the rationale, feel free to jump to the summary below.

Bundler's Purpose and Rationale

We designed bundler to make it easy to share your code across a number of development, staging and production machines. Of course, you know how to share your own application or gem: stick it on GitHub and clone it where you need it. Bundler makes it easy to make sure that your application has the dependencies it needs to start up and run without errors.

First, you declare these dependencies in a file at the root of your application, called Gemfile. It looks something like this:

source 'https://rubygems.org'

gem 'rails', '3.0.0.rc'
gem 'rack-cache'
gem 'nokogiri', '~> 1.4.2'

This Gemfile says a few things. First, it says that bundler should look for gems declared in the Gemfile at https://rubygems.org. You can declare multiple RubyGems sources, and bundler will look for gems in the order you declared the sources.

Next, you declare a few dependencies:

on version 3.0.0.rc of rails
on any version of rack-cache
on a version of nokogiri that is >= 1.4.2 but < 1.5.0

After declaring your first set of dependencies, you tell bundler to go get them:

$ bundle install    # <code>bundle</code> is a shortcut for <code>bundle install</code>

Bundler will connect to rubygems.org (and any other sources that you declared), and find a list of all of the required gems that meet the requirements you specified. Because all of the gems in your Gemfile have dependencies of their own (and some of those have their own dependencies), running bundle install on the Gemfile above will install quite a few gems.

$ bundle install
Fetching source index for http://gemcutter.org/
Using rake (0.8.7)
Using abstract (1.0.0)
Installing activesupport (3.0.0.rc)
Using builder (2.1.2)
Using i18n (0.4.1)
Installing activemodel (3.0.0.rc)
Using erubis (2.6.6)
Using rack (1.2.1)
Installing rack-mount (0.6.9)
Using rack-test (0.5.4)
Using tzinfo (0.3.22)
Installing actionpack (3.0.0.rc)
Using mime-types (1.16)
Using polyglot (0.3.1)
Using treetop (1.4.8)
Using mail (2.2.5)
Installing actionmailer (3.0.0.rc)
Using arel (0.4.0)
Installing activerecord (3.0.0.rc)
Installing activeresource (3.0.0.rc)
Using bundler (1.0.0.rc.3)
Installing nokogiri (1.4.3.1) with native extensions
Installing rack-cache (0.5.2)
Installing thor (0.14.0)
Installing railties (3.0.0.rc)
Installing rails (3.0.0.rc)
Your bundle is complete! Use `bundle show [gemname]` to see where a bundled gem is installed.

If any of the needed gems are already installed, Bundler will use them. After installing any needed gems to your system, bundler writes a snapshot of all of the gems and versions that it installed to Gemfile.lock.

Setting Up Your Application to Use Bundler

Bundler makes sure that Ruby can find all of the gems in the Gemfile (and all of their dependencies). If your app is a Rails 3 app, your default application already has the code necessary to invoke bundler. If it is a Rails 2.3 app, please see Setting up Bundler in Rails 2.3.

For another kind of application (such as a Sinatra application), you will need to set up bundler before trying to require any gems. At the top of the first file that your application loads (for Sinatra, the file that calls require 'sinatra'), put the following code:

require 'rubygems'
require 'bundler/setup'

This will automatically discover your Gemfile, and make all of the gems in your Gemfile available to Ruby (in technical terms, it puts the gems "on the load path"). You can think of it as an adding some extra powers to require 'rubygems'.

Now that your code is available to Ruby, you can require the gems that you need. For instance, you can require 'sinatra'. If you have a lot of dependencies, you might want to say "require all of the gems in my Gemfile". To do this, put the following code immediately following require 'bundler/setup':

Bundler.require(:default)

For our example Gemfile, this line is exactly equivalent to:

require 'rails'
require 'rack-cache'
require 'nokogiri'

Astute readers will notice that the correct way to require the rack-cache gem is require 'rack/cache', not require 'rack-cache'. To tell bundler to use require 'rack/cache', update your Gemfile:

source 'https://rubygems.org'

gem 'rails', '3.0.0.rc'
gem 'rack-cache', :require => 'rack/cache'
gem 'nokogiri', '~> 1.4.2'

For such a small Gemfile, we'd advise you to skip Bundler.require and just require the gems by hand (especially given the need to put in a :require directive in the Gemfile). For much larger Gemfiles, using Bundler.require allows you to skip repeating a large stack of requirements.

Grouping Your Dependencies

You'll sometimes have groups of gems that only make sense in particular environments. For instance, you might develop your app (at an early stage) using SQLite, but deploy it using mysql2 or pg. In this example, you might not have MySQL or Postgres installed on your development machine, and want bundler to skip it.

To do this, you can group your dependencies:

source 'https://rubygems.org'

gem 'rails', '3.2.2'
gem 'rack-cache', :require => 'rack/cache'
gem 'nokogiri', '~> 1.4.2'

group :development do
  gem 'sqlite3'
end

group :production do
  gem 'pg'
end

Now, in development, you can instruct bundler to skip the production group:

$ bundle install --without production

Bundler will remember that you installed the gems using --without production. For curious readers, bundler stores the flag in APP_ROOT/.bundle/config. You can see all of the settings that bundler saved there by running bundle config, which will also print out global settings (stored in ~/.bundle/config), and settings set via environment variables. For more information on configuring bundler, please see bundle config.

If you run bundle install later, without any flag, bundler will remember that you last called bundle install --without production, and use that flag again. When you require 'bundler/setup', bundler will ignore gems in these groups.

You can also specify which groups to automatically require through the parameters to Bundler.require. The :default group includes all gems not listed under any group. If you call Bundler.require(:default, :development), bundler will require all the gems in the :default group, as well as the gems in the :development group.

By default, a Rails generated app calls Bundler.require(:default, Rails.env) in your application.rb, which links the groups in your Gemfile to the Rails environment. If you use other groups (not linked to a Rails environment), you can add them to the call to Bundler.require, if you want them to be automatically required.

Remember that you can always leave groups of gems out of Bundler.require, and then require them manually using Ruby's require at the appropriate place in your app. You might do this because requiring a certain gem takes some time, and you don't need it every time you boot your application.

Checking Your Code into Version Control

After developing your application for a while, check in the application together with the Gemfile and Gemfile.lock snapshot. Now, your repository has a record of the exact versions of all of the gems that you used the last time you know for sure that the application worked. Keep in mind that while your Gemfile lists only three gems (with varying degrees of version strictness), your application depends on dozens of gems, once you take into consideration all of the implicit requirements of the gems you depend on.

This is important: the Gemfile.lock makes your application a single package of both your own code and the third-party code it ran the last time you know for sure that everything worked. Specifying exact versions of the third-party code you depend on in your Gemfile would not provide the same guarantee, because gems usually declare a range of versions for their dependencies.

The next time you run bundle install on the same machine, bundler will see that it already has all of the dependencies you need, and skip the installation process.

Do not check in the .bundle directory, or any of the files inside it. Those files are specific to each particular machine, and are used to persist installation options between runs of the bundle install command.

If you have run bundle pack, the gems (although not the git gems) required by your bundle will be downloaded into vendor/cache. Bundler can run without connecting to the internet (or the RubyGems server) if all the gems you need are present in that folder and checked in to your source control. This is an optional step, and not recommended, due to the increase in size of your source control repository.

When your co-developers (or you on another machine) check out your code, it will come with the exact versions of all the third-party code your application used on the machine that you last developed on (in the Gemfile.lock). When **they** run bundle install, bundler will find the Gemfile.lock and skip the dependency resolution step. Instead, it will install all of the same gems that you used on the original machine.

In other words, you don't have to guess which versions of the dependencies you should install. In the example we've been using, even though rack-cache declares a dependency on rack >= 0.4, we know for sure it works with rack 1.2.1. Even if the Rack team releases rack 1.2.2, bundler will always install 1.2.1, the exact version of the gem that we know works. This relieves a large maintenance burden from application developers, because all machines always run the exact same third-party code.

Updating a Dependency

Of course, at some point, you might want to update the version of a particular dependency your application relies on. For instance, you might want to update rails to 3.0.0 final. Importantly, just because you're updating one dependency, it doesn't mean you want to re-resolve all of your dependencies and use the latest version of everything. In our example, you only have three dependencies, but even in this case, updating everything can cause complications.

To illustrate, the rails 3.0.0.rc gem depends on actionpack 3.0.0.rc gem, which depends on rack ~> 1.2.1 (which means >= 1.2.1 and < 1.3.0). The rack-cache gem depends on rack >= 0.4. Let's assume that the rails 3.0.0 final gem also depends on rack ~> 1.2.1, and that since the release of rails 3.0.0, the Rack team released rack 1.2.2.

If we naïvely update all of our gems in order to update Rails, we'll get rack 1.2.2, which satisfies the requirements of both rails 3.0.0 and rack-cache. However, we didn't specifically ask to update rack-cache, which may not be compatible with rack 1.2.2 (for whatever reason). And while an update from rack 1.2.1 to rack 1.2.2 probably won't break anything, similar scenarios can happen that involve much larger jumps. (see [1] below for a larger discussion)

In order to avoid this problem, when you update a gem, bundler will not update a dependency of that gem if another gem still depends on it. In this example, since rack-cache still depends on rack, bundler will not update the rack gem. This ensures that updating rails doesn't inadvertently break rack-cache. Since rails 3.0.0's dependency actionpack 3.0.0 remains compatible with rack 1.2.1, bundler leaves it alone, and rack-cache continues to work even in the face of an incompatibility with rack 1.2.2.

Since you originally declared a dependency on rails 3.0.0.rc, if you want to update to rails 3.0.0, simply update your Gemfile to gem 'rails', '3.0.0' and run:

$ bundle install

As described above, the bundle install command always does a conservative update, refusing to update gems (or their dependencies) that you have not explicitly changed in the Gemfile. This means that if you do not modify rack-cache in your Gemfile, bundler will treat it **and its dependencies** (rack) as a single, unmodifiable unit. If rails 3.0.0 was incompatible with rack-cache, bundler will report a conflict between your snapshotted dependencies (Gemfile.lock) and your updated Gemfile.

If you update your Gemfile, and your system already has all of the needed dependencies, bundler will transparently update the Gemfile.lock when you boot your application. For instance, if you add mysql to your Gemfile, and have already installed it in your system, you can boot your application without running bundle install, and bundler will persist the "last known good" configuration to the Gemfile.lock snapshot.

This can come in handy when adding or updating gems with minimal dependencies (database drivers, wirble, ruby-debug). It will probably fail if you update gems with significant dependencies (rails), or that a lot of gems depend on (rack). If a transparent update fails, your application will fail to boot, and bundler will print out an error instructing you to run bundle install.

Updating a Gem Without Modifying the Gemfile

Sometimes, you want to update a dependency without modifying the Gemfile. For example, you might want to update to the latest version of rack-cache. Because you did not declare a specific version of rack-cache in the Gemfile, you might want to periodically get the latest version of rack-cache. To do this, you want to use the bundle update command:

$ bundle update rack-cache

This command will update rack-cache and its dependencies to the latest version allowed by the Gemfile (in this case, the latest version available). It will not modify any other dependencies.

It will, however, update dependencies of other gems if necessary. For instance, if the latest version of rack-cache specifies a dependency on rack >= 1.2.2, bundler will update rack to 1.2.2 even though you have not asked bundler to update rack. If bundler needs to update a gem that another gem depends on, it will let you know after the update has completed.

If you want to update every gem in the Gemfile to the latest possible versions, run:

$ bundle update

This will resolve dependencies from scratch, ignoring the Gemfile.lock. If you do this, keep git reset --hard and your test suite in your back pocket. Resolving all dependencies from scratch can have surprising results, especially if a number of the third-party packages you depend on have released new versions since you last did a full update.

Deploying Your Application

When you run bundle install, bundler will (by default), install your gems to your system repository of gems. This means that they will show up in gem list. Additionally, if you are developing a number of applications, you will not need to download and install gems in common for each application. This is nice for development, but somewhat problematic for deployment.

In a deployment scenario, the Unix user you deploy with may not have access to install gems to a system location. Even if the user does (or you use sudo), the user that boots the application may not have access to them. For instance, Passenger runs its Ruby subprocesses with the user nobody, a somewhat restricted user. The tradeoffs in a deployment environment lean more heavily in favor of isolation (even at the cost of a somewhat slower deploy-time bundle install when some third-party dependencies have changed).

As a result, bundler comes with a --deployment flag that encapsulates the best practices for using bundler in a deployment environment. These practices are based on significant feedback we have received during the development of bundler, as well as a number of bug reports that mostly reflected a misunderstanding of how to best configure bundler for deployment. The --deployment flags adds the following defaults:

Instead of installing gems to the system location, bundler will install gems to vendor/bundle inside your application. Bundler will transparently remember this location when you invoke it inside your application (with Bundler.setup and Bundler.require).
Bundler will not use gems already installed to your system, even if they exist.
If you have run bundle pack, checked in the vendor/cache directory, and do not have any git gems, Bundler will not contact the internet while installing your bundle.
Bundler will require a Gemfile.lock snapshot, and fail if you did not provide one.
Bundler will not transparently update your Gemfile.lock if it is out of date with your Gemfile

If you use Capistrano, you should symlink vendor/bundle to shared/vendor_bundle so that bundler will share your installed gems between deployments (making things zippy if you didn't make any changes), but still give you the benefits of isolation from other applications.

By defaulting the bundle directory to vendor/bundle, and installing your bundle as part of your deployment process, you can be sure that the same Unix user that checked out your application also installed the third-party code your application needs. This means that if Passenger (or Unicorn) can see your application, it can also see its dependencies.

The --deployment flag requires an up-to-date Gemfile.lock to ensure that the testing you have done (in development and staging) actually reflects the code you put into production. You can run bundle check before deploying your application to make sure that your Gemfile.lock is up-to-date. Note that it will always be up-to-date if you have run bundle install, successfully booted your application (or run your tests) since the last time you changed your Gemfile.

FAQ: Why Can't I Just Specify Only `=` Dependencies?

Q: I understand the value of locking my gems down to specific versions, but why can't I just specify = versions for all my dependencies in the Gemfile and forget about the Gemfile.lock?

A: Many of your gems will have their own dependencies, and they are unlikely to specify = dependencies. Moreover, it is probably unwise for gems to lock down all of *their* dependencies so strictly. The Gemfile.lock allows you to specify the versions of the dependencies that your application needs in the Gemfile, while remembering all of the exact versions of third-party code that your application used when it last worked correctly.

By specifying looser dependencies in your Gemfile (such as nokogiri ~> 1.4.2), you gain the ability to run bundle update nokogiri, and let bundler handle updating **only** nokogiri and its dependencies to the latest version that still satisfied the ~> 1.4.2 version requirement. This also allows you to say "I want to use the current version of nokogiri" (gem 'nokogiri' in your Gemfile) without having to look up the exact version number, while still getting the benefits of ensuring that your application always runs with exactly the same versions of all third-party code.

FAQ: Why Can't I Just Submodule Everything?

Q: I don't understand why I need bundler to manage my gems in this manner. Why can't I just get the gems I need and stick them in submodules, then put each of the submodules on the load path?

A: Unfortunately, that solution requires that you manually resolve all of the dependencies in your application, including dependencies of dependencies. And even once you do that successfully, you would need to redo that work if you wanted to update a particular gem. For instance, if you wanted to update the rails gem, you would need to find all of the gems that depended on dependencies of Rails (rack, erubis, i18n, tzinfo, etc.), and find new versions that satisfy the new versions of Rails' requirements.

Frankly, this is the sort of problem that computers are good at, and which you, a developer, should not need to spend time doing.

More concerningly, if you made a mistake in the manual dependency resolution process, you would not get any feedback about conflicts between different dependencies, resulting in subtle runtime errors. For instance, if you accidentally stuck the wrong version of rack in a submodule, it would likely break at runtime, when Rails or another dependency tried to rely on a method that was not present.

Bottom line: even though it might seem simpler at first glance, it is decidedly significantly more complex.

FAQ: Why Is Bundler Downloading Gems From `--without` Groups?

Q: I ran bundle install --without production and bundler is still downloading the gems in the :production group. Why?

A: Bundler's Gemfile.lock has to contain exact versions of all dependencies in your Gemfile, regardless of any options you pass in. If it did not, deploying your application to production might change all your dependencies, eliminating the benefit of Bundler. You could no longer be sure that your application uses the same gems in production that you used to develop and test with. Additionally, adding a dependency in production might result in an application that is impossible to deploy.

For instance, imagine you have a production-only gem (let's call it rack-debugging) that depends on rack =1.1. If we did not evaluate the production group when you ran bundle install --without production, you would deploy your application, only to receive an error that rack-debugging conflicted with rails (which depends on actionpack, which depends on rack ~> 1.2.1).

Another example: imagine a simple Rack application that has gem 'rack' in the Gemfile. Again, imagine that you put rack-debugging in the :production group. If we did not evaluate the :production group when you installed via bundle install --without production, your app would use rack 1.2.1 in development, and you would learn, at deployment time, that rack-debugging conflicts with the version of Rack that you tested with.

In contrast, by evaluating the gems in **all** groups when you call bundle install, regardless of the groups you actually want to use in that environment, we will discover the rack-debugger requirement, and install rack 1.1, which is also compatible with the gem 'rack' requirement in your Gemfile.

In short, by always evaluating all of the dependencies in your Gemfile, regardless of the dependencies you intend to use in a particular environment, you avoid nasty surprises when switching to a different set of groups in a different environment. And because we just download (but do not install) the gems, you won't have to worry about the possibility of a difficult **installation** process for a gem that you only use in production (or in development).

FAQ: I Have a C Extension That Requires Special Flags to Install

Q: I have a C extension gem, such as mysql, which requires special flags in order to compile and install. How can I pass these flags into the installation process for those gems?

A: First of all, this problem does not exist for the mysql2 gem, which is a drop-in replacement for the mysql gem. In general, modern C extensions properly discover the needed headers.

If you really need to pass flags to a C extension, you can use the bundle config command:

$ bundle config build.mysql --with-mysql-config=/usr/local/mysql/bin/mysql_config

Bundler will store this configuration in ~/.bundle/config, and bundler will use the configuration for any bundle install performed by the same user. As a result, once you specify the necessary build flags for a gem, you can successfully install that gem as many times as necessary.

Summary

A Simple Bundler Workflow

When you first create a Rails application, it already comes with a Gemfile. For another kind of application (such as Sinatra), run:
```
$ bundle init
```
The bundle init command creates a simple Gemfile which you can edit.
Next, add any gems that your application depends on. If you care which version of a particular gem that you need, be sure to include an appropriate version restriction:
```
source 'https://rubygems.org'

gem 'sinatra', '~> 0.9.0'
gem 'rack-cache'
gem 'rack-bug'
```
If you don't have the gems installed in your system yet, run:
```
$ bundle install
```

To update a gem's version requirements, first modify the Gemfile:

source 'https://rubygems.org'

gem 'sinatra', '~> 1.0.0'
gem 'rack-cache'
gem 'rack-bug'

and then run:

$ bundle install

If bundle install reports a conflict between your Gemfile and Gemfile.lock, run:
```
$ bundle update sinatra
```
This will update just the Sinatra gem, as well as any of its dependencies
To update all of the gems in your Gemfile to the latest possible versions, run:
```
$ bundle update
```
Whenever your Gemfile.lock changes, always check it in to version control. It keeps a history of the exact versions of all third-party code that you used to successfully run your application.
When deploying your code to a staging or production server, first run your tests (or boot your local development server), make sure you have checked in your Gemfile.lock to version control. On the remote server, run:
```
$ bundle install --deployment
```

Notes

[1] For instance, if rails 3.0.0 depended on rack 2.0, that gem would still satisfy the requirement of rack-cache, which declares >= 1.0 as a dependency. Of course, you could argue that rack-cache is silly for depending on open-ended versions, but these situations exist (extensively) in the wild, and projects often find themselves between a rock and a hard place when deciding what version to depend on. Constrain the dependency too much (rack =1.2.1) and you make it hard to use your project in other compatible projects. Constrain it too little (rack >= 1.0) and a new release of Rack may break your code. Using dependencies like rack ~> 1.2.1 and versioning code in a SemVer compliant way mostly solves this problem, but it assumes universal compliance. Since RubyGems has over 100,000 packages, this assumption simply doesn't hold in practice.