Magento 2 Deployment

In Magento 1, it was still possible to change the code of a live webshop on the fly. In Magento 2, this is no longer an option. A deployment step is needed. In this article I’ll tell you what deployment is, why it is necessary, and what our approach to it is at the moment.

Deployment is the process of moving new code from the safeness of the development environment to the hard reality of production. Since Magento is written in PHP, it is well suited to copy single files from development to production without any disruption of the live shop. However, as processes get more complex, and the demands for real-time performance more severe, things like pre-processed code and caching take the place of flexibility. Moving to production becomes a process by itself.

Lees dit document in het Nederlands

Standard Magento Deployment

Magento describes the steps to update a production server in this document.

php bin/magento maintenance:enable
composer require –no-update
composer update
php bin/magento setup:upgrade
php bin/magento setup:di:compile
php bin/magento setup:static-content:deploy
php bin/magento maintenance:disable

These are just the common steps. Specific shops may need additional ones. Together, these steps take several minutes, depending on the complexity of the shop. If this build process is done directly on the production shop, it needs to go in maintenance mode for several minutes. Complex shops with multiple locales, themes and extensions take more time. If nothing extra is done, updating a shop is a complex process that takes considerable time to complete. Time in which the shop is offline. For many shops this is unacceptable.

The holy grail of deployment is zero downtime: the shop is not down for a single moment. While this is not possible, one can try to keep it as low as a few seconds. Some Magento community members have published their approaches to this problem. Ourselves, we built a process from some of the existing techniques and made some changes needed for our specific use-case.

The Build Server

At BigBridge we maintain shops for various customers. These shops are hosted with different hosting providers that use both Apache 2 and Nginx as their webserver. All hosting providers (so far) use a Linux based OS. We store a complete shop in a single Git repository, but exclude (.gitignore) the parts that can be retrieved from other sources. We do not use the repository to control which modules are enabled; config.php is not checked in.

We wanted a build tool that performed all of the tasks needed to deploy a new release automatically, without human interference. One command, “deploy!”, should be enough to update a shop to its latest release.

It is possible to build a new release on the production server, independent of the live shop, but it takes system resources away from the shop, and requires the production server to have access to the code repositories. If Sass is used, Node.js needs to be installed on the server. For these reasons, it is easier to do the building away from the production environment, on a special build server. We use our development machine as our build server.

Our build environment resembles the production environment in that it also has a database, runs in production mode, and has all the same extensions installed. It has no content, no products and no customers.

There’s another reason why deployment is best done on a separate build server, or at least on a separate build database. When a new extension is added to the shop, it will be automatically added and enabled to config.php by Magento by setup:upgrade. This means, however, that setup:static-content:deploy can only be run after the database is upgraded. This takes minutes of downtime. By duplicating the complete build process on the build server, including the setup:upgrade step (requiring a build database) this problem can be avoided. Only after setup:upgrade and setup:static-content:deploy are complete, all code will be copied to production.

Tool: Deployer

We have written a custom deployment tool around Deployer. Deployer is a PHP tool that allows easy interaction with a production server. I will give some examples of the steps it takes to deploy, but we don’t release the source code because there’s no one-size fits all deployment script. Different companies working with Magento have different demands on a deployment tool. However, Deployer is easy to work with and can be tailored by any developer versed in PHP.

Our tool contains a directory with a config file for each shop. Changes to the deployment tool are easy to make.

Directories and public_html

The production server directory structure we crafted looks like the standard Deployer structure, with some differences:

shared is the directory that contains all files that are shared between releases: config files, images, feeds, sessions, logs. It is set up once and never touched again. Next to the database, the shared folder is the only essential part of the webshop that needs to be backed up.

The release directories (5, 6, 7, current) contain symlinks to subdirectories of this shared folder. The symlinks are preferably relative. Absolute symlinks may have more severe permission constraints on some servers.

public_html, which is normally a directory, is turned into a symlink that points to:

deployments/releases/current/pub

Hosting providers don’t always provide the possibility to set the document root to whatever directory you would like (e.g. pub). By changing public_html into a symlink, you take that power into your own hand.

In our setup, current is not a symlink pointing to the latest release dir, as is the case in standard Deployer. It’s a plain directory. We have made this change because symlinks are regularly cached (for example by webserver Apache) for some time and this is problematic. Notably, when an extension is updated to a new version, the version in the table “setup_module” must match the version in the code’s “module.xml”. When a new version is deployed, some long lived Apache children may still hold cached symlinks to previous release code for several minutes. Database version and code version do not match and Magento throws an error. Always pointing the symlink public_html to the same directory (current/pub) solves this problem.

When a new release is made, the deployment process creates a new release dir, say ‘8’.

In the switch stage, after maintenance mode is enabled, current and 8 switch names. This can be done with three ‘mv’ instructions.

After this the database is updated, caches are cleared (notably Redis) and maintenance mode is disabled. The shop is offline for just a few seconds.

Rollback

In case of an emergency, we can reactivate a previous release of the shop by just changing directory names.

This does not work if an extension has a new version, because the version of the code does not match the version in the database anymore. In such a case, change the version in the extension’s module.xml manually to prevent prolonged downtime.

Deployment stages

Whenever a new release is needed, we give a single command to the server to start the deployment process. At this point the deployment tool goes through five phases:

create a build version of the shop
update its code, its module versions and its database
prepare a new release directory on the production server
switch to the new directory
clean up old releases

These phases are described in more detail below.

Create the build shop

The first 2 phases take place on the build server.

In the first release, a complete “build shop” is created on the build server, programmatically. It consists of an empty database and a composer install of a Magento installation. There’s no need to copy the production database to the build server, because all we’re interested in is the table “setup_module”, presuming that our build process is the only process that adds extensions to the production server.

Update the build shop

The shop on the build server is updated to the new version. The code is pulled from the BitBucket Git repository, and the third party code is installed using Composer. The update is done on the previous state of the build server, so that “git pull” only downloads the changes since the previous release.

All these steps are performed on the offline build shop:

git pull
create a file pub/static/deployed_version.txt
composer install
apply a set of patch files for known Magento problems
chmod +x bin/magento
build vendor/snowdog/frontools/node_modules for Sass compilation
set the build shop to production mode
bin/magento setup:upgrade
bin/magento di:compile
bin/magento setup:static-content:deploy
Do ‘gulp styles’ to compile css files from sass files
git checkout pub/.htaccess (because it is overwritten by earlier steps)

Prepare a new release on production

Deployer takes care of creating a new directory.

rsync can be used to copy code to the production server very efficiently. The following command copies the folder $source to the folder $target on the server via SSH. It checks if the directory $current on the production server already has the needed files, so they don’t need to be transferred. Even better, it creates hard links to these existing files, and this saves both time and disk space. It excludes the .git directory.

rsync --link-dest {$current} --exclude ".git/" -e "ssh -p {$port}" -a {$source} {$target}

Finally Magento is set to production mode if it wasn’t already (first release only).

Switch

In this phase, the new release is activated.

bin/magento maintenance:disable
mv {$new} {$temp} && mv {$current} {$new} && mv {$temp} {$current}
bin/magento setup:upgrade –keep-generated
bin/magento cache:flush
bin/magento maintenance:enable

Cleanup

We keep up to 10 releases per shop. Older releases are removed in this phase.

Finally

A separate deployment process is a must have for a Magento 2 webshop. It ensures that changes to the shop are made in a controlled fashion that ensures minimal downtime. Problems in the upgrade process show up in the offline build environment where they can still be fixed. If the released code shows problems only after it has gone live, there’s also the possibility to quickly revert to the previous code. These features make that a deployment tool is not only a necessity, it can also be an improvement to maintenance quality.

A deployment tool needs to keep evolving to adapt to changing circumstances however. Magento 2.2 promises a near-zero downtime deployment model, so this may change everything again!

Make sure to consult the Magento deployment page if you’re planning to create a deployment process.