Tutorial

How to Use a Remote Docker Server to Speed Up Your Workflow

Published on June 25, 2019
How to Use a Remote Docker Server to Speed Up Your Workflow

Introduction

Building CPU-intensive images and binaries is a very slow and time-consuming process that can turn your laptop into a space heater at times. Pushing Docker images on a slow connection takes a long time, too. Luckily, there’s an easy fix for these issues. Docker lets you offload all those tasks to a remote server so your local machine doesn’t have to do that hard work.

This feature was introduced in Docker 18.09. It brings support for connecting to a Docker host remotely via SSH. It requires very little configuration on the client, and only needs a regular Docker server without any special config running on a remote machine. Prior to Docker 18.09, you had to use Docker Machine to create a remote Docker server and then configure the local Docker environment to use it. This new method removes that additional complexity.

In this tutorial, you’ll create a Droplet to host the remote Docker server and configure the docker command on your local machine to use it.

Prerequisites

To follow this tutorial, you’ll need:

  • A DigitalOcean account. You can create an account if you don’t have one already.
  • Docker installed on your local machine or development server. If you are working with Ubuntu 18.04, follow Steps 1 and 2 of How To Install and Use Docker on Ubuntu 18.04; otherwise, follow the official documentation for information about installing on other operating systems. Be sure to add your non-root user to the docker group, as described in Step 2 of the linked tutorial.

Step 1 – Creating the Docker Host

To get started, spin up a Droplet with a decent amount of processing power. The CPU Optimized plans are perfect for this purpose, but Standard ones work just as well. If you will be compiling resource-intensive programs, the CPU Optimized plans provide dedicated CPU cores which allow for faster builds. Otherwise, the Standard plans offer a more balanced CPU to RAM ratio.

The Docker One-click image takes care of all of the setup for us. Follow this link to create a 16GB/8vCPU CPU-Optimized Droplet with Docker from the control panel.

Alternatively, you can use doctl to create the Droplet from your local command line. To install it, follow the instructions in the doctl README file on GitHub.

The following command creates a new 16GB/8vCPU CPU-Optimized Droplet in the FRA1 region based on the Docker One-click image:

  1. doctl compute droplet create docker-host \
  2. --image docker-18-04 \
  3. --region fra1 \
  4. --size c-8 \
  5. --wait \
  6. --ssh-keys $(doctl compute ssh-key list --format ID --no-header | sed 's/$/,/' | tr -d '\n' | sed 's/,$//')

The doctl command uses the ssh-keys value to specify which SSH keys it should apply to your new Droplet. We use a subshell to call doctl compute ssh-key-list to retrieve the SSH keys associated with your DigitalOcean account, and then parse the results using the sed and tr commands to format the data in the correct format. This command includes all of your account’s SSH keys, but you can replace the highlighted subcommand with the fingerprint of any key you have in your account.

Once the Droplet is created you’ll see its IP address among other details:

Output
ID Name Public IPv4 Private IPv4 Public IPv6 Memory VCPUs Disk Region Image Status Tags Features Volumes 148681562 docker-host your_server_ip 16384 8 100 fra1 Ubuntu Docker 5:18.09.6~3 on 18.04 active

You can learn more about using the doctl command in the tutorial How To Use doctl, the Official DigitalOcean Command-Line Client.

When the Droplet is created, you’ll have a ready to use Docker server. For security purposes, create a Linux user to use instead of root.

First, connect to the Droplet with SSH as the root user:

  1. ssh root@your_server_ip

Once connected, add a new user. This command adds one named sammy:

  1. adduser sammy

Then add the user to the docker group to give it permission to run commands on the Docker host.

  1. sudo usermod -aG docker sammy

Finally, exit from the remote server by typing exit.

Now that the server is ready, let’s configure the local docker command to use it.

Step 2 – Configuring Docker to Use the Remote Host

To use the remote host as your Docker host instead of your local machine, set the DOCKER_HOST environment variable to point to the remote host. This variable will instruct the Docker CLI client to connect to the remote server.

  1. export DOCKER_HOST=ssh://sammy@your_server_ip

Now any Docker command you run will be run on the Droplet. For example, if you start a web server container and expose a port, it will be run on the Droplet and will be accessible through the port you exposed on the Droplet’s IP address.

To verify that you’re accessing the Droplet as the Docker host, run docker info.

  1. docker info

You will see your Droplet’s hostname listed in the Name field like so:

Output
… Name: docker-host

One thing to keep in mind is that when you run a docker build command, the build context (all files and folders accessible from the Dockerfile) will be sent to the host and then the build process will run. Depending on the size of the build context and the amount of files, it may take a longer time compared to building the image on a local machine. One solution would be to create a new directory dedicated to the Docker image and copy or link only the files that will be used in the image so that no unneeded files will be uploaded inadvertently.

Once you’ve set the DOCKER_HOST variable using export, its value will persist for the duration of the shell session. Should you need to use your local Docker server again, you can clear the variable using the following command:

unset DOCKER_HOST

Conclusion

You’ve created a remote Docker host and connected to it locally. The next time your laptop’s battery is running low or you need to build a heavy Docker image, use your shiny remote Docker server instead of your local machine.

You might also be interested in learning how to optimize Docker images for production, or how to optimize them specifically for Kubernetes.

Thanks for learning with the DigitalOcean Community. Check out our offerings for compute, storage, networking, and managed databases.

Learn more about our products

About the authors


Still looking for an answer?

Ask a questionSearch for more help

Was this helpful?
 
5 Comments


This textbox defaults to using Markdown to format your answer.

You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!

This tutorial is extremely useful for us at Venezuela: our max Internet speed is 10 mbps ADSL…

So, I want give back with some little details, with all due respect:

  • I consider more practical use docker info | grep Name; of course we need modify the paragraph for explain what is grep command and pipe.
  • I think that inform, just in case, how use the local Docker host: export DOCKER_HOST=.

Thanks twice: for the tutorial first and for read my comment after. 8-)

Cool. This is very promising! I have it working for commands like docker compose build but I get errors when running docker compose run .... It’s as if it needs SSH again for the run command, but not finding instructions on how to make that happen. Ideas?

Error with redactions: error during connect: Get “http://docker/v1.24/containers/json…”: command [ssh -l {{user}} – {{host}} docker system dial-stdio] has exited with exit status 255, please make sure the URL is valid, and Docker 18.09 or later is installed on the remote host: stderr=ssh: connect to host {{host}} port 22: Connection refused

ssh root@your_server_ip docker info

ok export DOCKER_HOST=ssh://sammy@your_server_ip docker info Server: ERROR: stderr=root@46.101.135.158: Permission denied (publickey).

How can I connect to my remotely setup docker wordpress development environment use phpstorm ?

Hi, thanks for the write-up. It’s easy to follow but misses some steps:

“for security reasons” - first step after login - install updates!

Copy the authorized keys from root to the new “sammy” user and reconfigure sshd to deny any logins from root.

After you’re done with your work … delete the droplet to save costs with: doctl compute droplet delete docker-host

Cheers

Try DigitalOcean for free

Click below to sign up and get $200 of credit to try our products over 60 days!

Sign up

Join the Tech Talk
Success! Thank you! Please check your email for further details.

Please complete your information!

Featured on Community

Get our biweekly newsletter

Sign up for Infrastructure as a Newsletter.

Hollie's Hub for Good

Working on improving health and education, reducing inequality, and spurring economic growth? We'd like to help.

Become a contributor

Get paid to write technical tutorials and select a tech-focused charity to receive a matching donation.

Welcome to the developer cloud

DigitalOcean makes it simple to launch in the cloud and scale up as you grow — whether you're running one virtual machine or ten thousand.

Learn more
DigitalOcean Cloud Control Panel