Drone is a Continuous Integration (CI) tool. Continuous Integration refers to the process of compiling and testing your code during all phases of development, and it’s a real good thing to do when you’re working on projects of a certain size.

A lot of tools exist to do that, such as Jenkins, Hudson or Travis. Basically they do the same thing: whenever you push a commit on a distant repo, a script is automatically fired to build your project, run some tests and warn you if something went wrong. Optionally you can bind that with a code reviewing tool such as Gerrit and use it to refuse commits which does not compile or pass your tests.

Drone makes use of the Docker technology, which you must have at least heard about if you were not living in a cave this two last years-or-so ¹.

First you have to put a small .drone.yml file at the root of your repo, which contains the Docker image you want to use and the build and test scripts you want to run. Then when you push a new commit in your repo, Drone will fetch the Docker image you specified, spawn a container from it, clone your repo inside, run the scripts and give you the results (and then destroy the used container). Pretty cool, huh ?

Finally, Drone integrates nicely with other tools, like Github, Bitbuckets and Gogs (of which I already spoke, and which is great, and you should use it).

Drone comes in two flavours: hosted or open-source (self hosted). I have no experience with using the hosted version, but from help videos I can assume it’s pretty much the same thing as the open source one, only with better UI and a way to replace your .drone.yml file with online configuration. Which should be really, really helpful, because you WON’T FIND ANY GOOD NOR COMPLETE DOCUMENTATION ON THIS GODDAMN YAML FILE ANYWHERE!. There, now I feel better. :]

Anyway.

The Open Source Edition is hosted on Github. The simplest way to use it, is by using Docker (you should have it anyway since it’s a dependency). An official image exist on the Docker Hub. Following the apparent Drone policy, there is no README whatsoever to tell you how the Docker image work. You will need to run the a command along the following to create a useful container from the image²:

Okay, that’s a lot of stuff for one command. Let’s review the important parts.

-d : to run a daemon, nothing unusual here

--link postgresql:db : by default docker uses a sqlite database, but it can work with mysql or postgresql databases too. I already had a postgresql instance, inside a container, so I decided to make use of it. The name you give it inside the container (here db) is not important. As you will see, the configuration to connect to the base will be passed next.

-e DRONE_DATABASE_DRIVER : with this you specify the database driver you want to use. Default is sqlite, here I put postgres because I want to make use of my postgresql instance.

OK, time to notice something: we pass a lot of environnment variables. According to the meager Drone documentation I was able to collect, this is the way it should be. To configurate Drone, you can use regular configuration files or use environment variables. When using a Docker container, environment variables are actually the best way to go. Let’s continue now.

-e DRONE_DATABASE_DATASOURCE : This variable must contain the connection string for PostgreSQL (or MySQL, or the filename if you’re rolling with sqlite). This connection string is not a Drone thing, but is specified by PostgreSQL. I only specified the most important variables here: the host (the name I used for the link), login, password, database name and no SSL since I’m running locally.

-e DRONE_GOGS_URL -e DRONE_GOGS_SECRET : you can link your drone instance with several different git frontends: Bitbucket, Github, Gitlab and Gogs. There are specific configuration settings for each of them. Here I use Gogs (did I mention it was great and that you should use it?), so I need to feed Drone my Gogs URL and a Gogs “secret” which I still did not figure what purpose it serves and filled randomly. Take care though that you will need to have the rights to create webhooks on your Gogs instance if you want the link to work smoothly. Otherwise it is still possible but you will have to write the hooks by hand (logging-in with a user with the right permissions)

-v /var/run/docker.sock:/var/run/docker.sock : the most important part: the socket to your Docker daemon. Keep reading.

As I told you earlier, Drone makes use of Docker to build and test your projects inside containers. To do so, it obviously needs a working Docker instance. But how do we provide it one when Drone is itself running inside a Docker container? We could embed the Docker software inside the Docker container, but although it’s definitely possible, it is not easy nor elegant: you have to run the “host container” with special privileges, and it makes the docker image size quite consequent. The best way to go is then to leverage the client/server architecture of Docker. The Docker clients (mainly the docker binary for console command) all connect to a running Docker daemon listening on a socket. By default, that socket is a unix socket attach to the file /var/run/docker.sock. And boom, there it is: if you link this file as a volume inside the Docker container you can use a Docker client inside without having to install the daemon.

We are now with our drone freshly installed and linked to our Gogs instance, it’s time to use it. First of all, we can log on our web interface (for me it’s http://drone.lertsene.com), using our Gogs credentials. Then we can sync the projects (Drone will fetch the public git repos we have), and select one to use with Drone. Next you should meet with a message indicating that you need a .drone.yml file in your project in order to build it with Drone.

The .drone.yml file is a Yaml file with Drone specific keywords used to describe how you want your project to be built, fetched, deployed, published, and how you want to be notified about it. Here is one of mine, that I use for my pelican blog.

Please note that the keywords you can use in this .drone.yml file are quite numerous, and the documentation listing them is quite sparse. Here I will only describe the keywords I used. The rest of the keywords will probably make the subject of another article, when I will be more familiar with them.

First comes the most important part of the file. The image keyword indicates the Docker image you want to use with your project. The syntax is the same that for the Docker hub: user/imagename:_tag_. Note that a lot of aliases have been defined for convenience that redirect to the creator of Drone project Docker repo: bradrydzewski. So when you type image: go1.2 you will actually be using the Docker image bradrydzewski/go:1.2. Take care that those “convenient” images are really fat (or were the last time I tried to use them) because they are built on top of the Ubuntu image. Consider using lighter images if your internet connection is not fast enough.

The script keyword is where you specify the commands for your build. You can put the command irectly in the Yaml file like I did, or create a build script and just launch it from here. This is bash scripting by the way, so the image you are using will need to have a bash binary.

With those two keywords, you have a basis for all the builds and tests you want to run. But then what happend when, like me, you want to automatically deploy the result of your build? This is what the deploy keyword is for. Whatever under this keyword section won’t be run unless the script section passed without error.

Under the deploy section, you can find different deployment methods: heroku, modulus, nodejitsu, tsuru (I have no idea what those are), git, ssh or bash.

With the ssh method for example, you specify a URL and a command to run. Drone will push (with SCP) the results of your build to the URL and then will run the command you specified remotely. git will do the same, using git instead of ssh, and finally bash will let you write your own bash script to do what you want.

Here I is where I had to stop and think. I only have one server, so I only wanted Drone to push the result of my build somewhere on the server. SSH or Git are a bit overkill for this, especially considering that this would make me create specific users and keys and therefore add an unneeded security risk.

The ideal way to do it would be to mount a host volume at the container creation and push my build there. But there does not seems to be a way to do that with Drone yet.

Upon reading the source looking for a solution, I noticed that the only host volumes mounted by Drone when creating a build container were used for caching build results. So that’s what I settled to use. Beware though: it’s kind of ugly and I hope that Drone will provide a better way to do that later.

The cache keyword let you specify a volume to use on the build container spawned by Drone. This container is mounted from your host in /tmp/drone. In other terms, it’s logically equivalent to using the Docker flag: -v /tmp/drone/mygogs.url.com/myuser/myreponame/path/to/my/volume:/path/to/my/volume when running your container. And that’s why I have a /public cache, and why my deploy bash script uses rsync to push the output of my build to /public ³.

Now that your .drone.yml file is ready, you can make a new commit in your project and push your repo. Drone should receive an alert from Gogs, fetch your project, parse your .drone.yml and start a build, all by itself!

OK, that was a simple and pretty straightforward setup of Drone. I have no doubt that you have specific needs push the boundaries of what is possible to do with what I just presented you. So here are some pointers:

Finally, when running into weird behaviour, I would recommend you to:

Before completing my .drone.yml file, I spent some time doing test inside my build script, at some point running find / commands to figure where everything was, if my volumes were mounted, etc.

This is an aspect I didn’t even considered, because I don’t build big projects. Therefore I make no use of parallel builds, (real) cache, etc. For my small blog project, a build from commit to result is approximately 15 minutes long. Sorry, I can’t get more informative than that, but please tell me if you have useful informations on the subject.

As always for comments, suggestions or insults, please use my twitter account.