Review: Docker Engine Version 1.13 New Features (Part.2 – docker secret)

As we have seen in previous post (Part.1 – docker system) there are new features on new docker engine version that will make our life easier. I think that one of the most requested features for some time had been use of secrets. When deploying containers, we need to use information to configure our applications, and we need to encrypt some of this values.

It is never recommended to use passwords or other kind of sensible information in images itself as anyone who uses that image will have access to this data. We can change information on runtime, while creating containers passing key/value pairs as environments variables but someone could access that info published on container (you can even read them using docker inspect).

There is another option, using config files between volumes or introduce these files on runtime, but we need to share these files across all cluster nodes because when we create a service swarm will start it in the best host (knowing its affinities, constraints or even hosts loads).

Docker secret come to simplify this and help us to manage this kind of information.

We will use experimental mode and test version (at the time of writing this post, it is 1.13-rc7). Please read the references to know how to enable experimental mode and you can use a quick vagrant environment for these examples.

Docker secret will let us to manage all secrets in the swarm cluster. We could create, delete, inspect and list all secrets.

ubuntu@swarmnode1:/run$ docker secret --help

Usage: docker secret COMMAND

Manage Docker secrets

Options:
 --help Print usage

Commands:
 create Create a secret from a file or STDIN as content
 inspect Display detailed information on one or more secrets
 ls List secrets
 rm Remove one or more secrets

Run 'docker secret COMMAND --help' for more information on a command.

 

Let’s start creating a new secret:

ubuntu@swarmnode1:~$ echo "mysecretpasswordformyapp" |docker secret create MYSECRETS -
2rhdvw2o2g53ferldl47544io

This command returns the id of the newly created secret object. There is an option when creating secrets that allow us to add a list of labels for better management.

We can list all secrets created in our cluster:

ubuntu@swarmnode1:~$ docker secret ls
ID NAME CREATED UPDATED
2rhdvw2o2g53ferldl47544io MYSECRETS 20 seconds ago 20 seconds ago

And as with other objects, secrets can be reviewed with inspect.

ubuntu@swarmnode1:~$ docker secret inspect MYSECRETS
[
 {
 "ID": "2rhdvw2o2g53ferldl47544io",
 "Version": {
 "Index": 66
 },
 "CreatedAt": "2017-01-15T15:22:19.164299433Z",
 "UpdatedAt": "2017-01-15T15:22:19.164299433Z",
 "Spec": {
 "Name": "MYSECRETS"
 }
 }
]

Now we are going to create a simple service that pings wwwo.google.com based on busybox that will use the newly created secret object.

ubuntu@swarmnode1:~$ docker service create --secret MYSECRETS busybox ping www.google.es
qtken9b03a2j76lq4nvaxlxdr

We need to know where are running the tasks created to maintain the desired state of this service.

ubuntu@swarmnode1:~$ docker service ls
ID NAME MODE REPLICAS IMAGE
qtken9b03a2j determined_knuth replicated 1/1 busybox:latest

ubuntu@swarmnode1:~$ docker service ps determined_knuth
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
m4kslfp5h2hb determined_knuth.1 busybox:latest swarmnode3 Running Running 28 seconds ago
We connect to swarmnode3 and show containers running on this host.

ubuntu@swarmnode3:~$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
2474ef1e7dfc busybox@sha256:817a12c32a39bbe394944ba49de563e085f1d3c5266eb8e9723256bc4448680e "ping www.google.es" 7 minutes ago Up 7 minutes determined_knuth.1.m4kslfp5h2hbxqxmm75tv90qq

We see that container 2474ef1e7dfc is associated with the task m4kslfp5h2hbxqxmm75tv90qq, as we can review in its name determined_knuth.1.m4kslfp5h2hbxqxmm75tv90qq.

When we use –secrets <SECRET_OBJECT_NAME> while creating a services, docker engine will create a file under /run/secrets with the name of that object on each task related to that service. In this example we will have a file named /run/secrets/MYSECRETS on all tasks related to this service.

Task 2474ef1e7dfc is running on swarmnode3 and we use a simple cat to see the content of /run/secrets/MYSECRETS.

ubuntu@swarmnode3:/run$ docker exec -ti 2474ef1e7dfc ls -l /run/secrets/MYSECRETS
-r--r--r-- 1 root root 25 Jan 15 15:26 /run/secrets/MYSECRETS
ubuntu@swarmnode3:/run$ docker exec -ti 2474ef1e7dfc cat /run/secrets/MYSECRETS
mysecretpasswordformyapp

The content of the secret created is available and we can use it to configure our application. It is read-only and accessible to any application that needs to use its information (remember to start container applications with its users, not always as root).

Information is secured when we review service or containers/tasks related.

Inspecting the service we can see now a new Secrets section:

ubuntu@swarmnode1:~/src$ docker service inspect determined_knuth
[
 {
 "ID": "qtken9b03a2j76lq4nvaxlxdr",
 "Version": {
 "Index": 67
 },
 "CreatedAt": "2017-01-15T15:26:14.53919447Z",
 "UpdatedAt": "2017-01-15T15:26:14.53919447Z",
 "Spec": {
 "Name": "determined_knuth",
 "TaskTemplate": {
 "ContainerSpec": {
 "Image": "busybox:latest@sha256:817a12c32a39bbe394944ba49de563e085f1d3c5266eb8e9723256bc4448680e",
 "Args": [
 "ping",
 "www.google.es"
 ],
 "DNSConfig": {},
 "Secrets": [
 {
 "File": {
 "Name": "MYSECRETS",
 "UID": "0",
 "GID": "0",
 "Mode": 292
 },
 "SecretID": "2rhdvw2o2g53ferldl47544io",
 "SecretName": "MYSECRETS"
 }
 ]
 },
 "Resources": {
 "Limits": {},
 "Reservations": {}
 },
 "RestartPolicy": {
 "Condition": "any",
 "MaxAttempts": 0
 },
 "Placement": {},
 "ForceUpdate": 0
 },
 "Mode": {
 "Replicated": {
 "Replicas": 1
 }
 },
 "UpdateConfig": {
 "Parallelism": 1,
 "FailureAction": "pause",
 "MaxFailureRatio": 0
 },
 "EndpointSpec": {
 "Mode": "vip"
 }
 },
 "Endpoint": {
 "Spec": {}
 },
 "UpdateStatus": {
 "StartedAt": "0001-01-01T00:00:00Z",
 "CompletedAt": "0001-01-01T00:00:00Z"
 }
 }
]

Inspecting the container we can’t see anything related to the secret object used.

ubuntu@swarmnode3:~$ docker inspect 2474ef1e7dfc
[
 {
 "Id": "2474ef1e7dfccd12fb4ac04cf8391b22edeea35c833d6a1800b8087502d34507",
 "Created": "2017-01-15T15:26:18.04161973Z",
 "Path": "ping",
 "Args": [
 "www.google.es"
 ],
 "State": {
 "Status": "running",
 "Running": true,
 "Paused": false,
 "Restarting": false,
 "OOMKilled": false,
 "Dead": false,
 "Pid": 22719,
 "ExitCode": 0,
 "Error": "",
 "StartedAt": "2017-01-15T15:26:18.13827557Z",
 "FinishedAt": "0001-01-01T00:00:00Z"
 },
 "Image": "sha256:7968321274dc6b6171697c33df7815310468e694ac5be0ec03ff053bb135e768",
 "ResolvConfPath": "/var/lib/docker/containers/2474ef1e7dfccd12fb4ac04cf8391b22edeea35c833d6a1800b8087502d34507/resolv.conf",
 "HostnamePath": "/var/lib/docker/containers/2474ef1e7dfccd12fb4ac04cf8391b22edeea35c833d6a1800b8087502d34507/hostname",
 "HostsPath": "/var/lib/docker/containers/2474ef1e7dfccd12fb4ac04cf8391b22edeea35c833d6a1800b8087502d34507/hosts",
 "LogPath": "/var/lib/docker/containers/2474ef1e7dfccd12fb4ac04cf8391b22edeea35c833d6a1800b8087502d34507/2474ef1e7dfccd12fb4ac04cf8391b22edeea35c833d6a1800b8087502d34507-json.log",
 "Name": "/determined_knuth.1.m4kslfp5h2hbxqxmm75tv90qq",
 "RestartCount": 0,
 "Driver": "aufs",
 "MountLabel": "",
 "ProcessLabel": "",
 "AppArmorProfile": "",
 "ExecIDs": null,
 "HostConfig": {
 "Binds": null,
 "ContainerIDFile": "",
 "LogConfig": {
 "Type": "json-file",
 "Config": {}
 },
 "NetworkMode": "default",
 "PortBindings": {},
 "RestartPolicy": {
 "Name": "",
 "MaximumRetryCount": 0
 },
 "AutoRemove": false,
 "VolumeDriver": "",
 "VolumesFrom": null,
 "CapAdd": null,
 "CapDrop": null,
 "Dns": null,
 "DnsOptions": null,
 "DnsSearch": null,
 "ExtraHosts": null,
 "GroupAdd": null,
 "IpcMode": "",
 "Cgroup": "",
 "Links": null,
 "OomScoreAdj": 0,
 "PidMode": "",
 "Privileged": false,
 "PublishAllPorts": false,
 "ReadonlyRootfs": false,
 "SecurityOpt": null,
 "UTSMode": "",
 "UsernsMode": "",
 "ShmSize": 67108864,
 "Runtime": "runc",
 "ConsoleSize": [
 0,
 0
 ],
 "Isolation": "",
 "CpuShares": 0,
 "Memory": 0,
 "NanoCpus": 0,
 "CgroupParent": "",
 "BlkioWeight": 0,
 "BlkioWeightDevice": null,
 "BlkioDeviceReadBps": null,
 "BlkioDeviceWriteBps": null,
 "BlkioDeviceReadIOps": null,
 "BlkioDeviceWriteIOps": null,
 "CpuPeriod": 0,
 "CpuQuota": 0,
 "CpuRealtimePeriod": 0,
 "CpuRealtimeRuntime": 0,
 "CpusetCpus": "",
 "CpusetMems": "",
 "Devices": null,
 "DiskQuota": 0,
 "KernelMemory": 0,
 "MemoryReservation": 0,
 "MemorySwap": 0,
 "MemorySwappiness": -1,
 "OomKillDisable": false,
 "PidsLimit": 0,
 "Ulimits": null,
 "CpuCount": 0,
 "CpuPercent": 0,
 "IOMaximumIOps": 0,
 "IOMaximumBandwidth": 0
 },
 "GraphDriver": {
 "Name": "aufs",
 "Data": null
 },
 "Mounts": [],
 "Config": {
 "Hostname": "2474ef1e7dfc",
 "Domainname": "",
 "User": "",
 "AttachStdin": false,
 "AttachStdout": false,
 "AttachStderr": false,
 "Tty": false,
 "OpenStdin": false,
 "StdinOnce": false,
 "Env": [
 "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
 ],
 "Cmd": [
 "ping",
 "www.google.es"
 ],
 "Image": "busybox@sha256:817a12c32a39bbe394944ba49de563e085f1d3c5266eb8e9723256bc4448680e",
 "Volumes": null,
 "WorkingDir": "",
 "Entrypoint": null,
 "OnBuild": null,
 "Labels": {
 "com.docker.swarm.node.id": "4w4k52i4x5wdgqc171f3v0w61",
 "com.docker.swarm.service.id": "qtken9b03a2j76lq4nvaxlxdr",
 "com.docker.swarm.service.name": "determined_knuth",
 "com.docker.swarm.task": "",
 "com.docker.swarm.task.id": "m4kslfp5h2hbxqxmm75tv90qq",
 "com.docker.swarm.task.name": "determined_knuth.1.m4kslfp5h2hbxqxmm75tv90qq"
 }
 },
 "NetworkSettings": {
 "Bridge": "",
 "SandboxID": "661c75e5f313b2d8eaf5c4622661ab055bba0e18108f186ddb6cfc9ee2d99f88",
 "HairpinMode": false,
 "LinkLocalIPv6Address": "",
 "LinkLocalIPv6PrefixLen": 0,
 "Ports": {},
 "SandboxKey": "/var/run/docker/netns/661c75e5f313",
 "SecondaryIPAddresses": null,
 "SecondaryIPv6Addresses": null,
 "EndpointID": "3f608078cda4de700bdffffdba06e19705d2123aba6d4d912e1ae3867e07ef5f",
 "Gateway": "172.17.0.1",
 "GlobalIPv6Address": "",
 "GlobalIPv6PrefixLen": 0,
 "IPAddress": "172.17.0.4",
 "IPPrefixLen": 16,
 "IPv6Gateway": "",
 "MacAddress": "02:42:ac:11:00:04",
 "Networks": {
 "bridge": {
 "IPAMConfig": null,
 "Links": null,
 "Aliases": null,
 "NetworkID": "ec8d0516e31ed9d14c21a2c6768407def6dcbcc1a1139a093f5cb9505924328f",
 "EndpointID": "3f608078cda4de700bdffffdba06e19705d2123aba6d4d912e1ae3867e07ef5f",
 "Gateway": "172.17.0.1",
 "IPAddress": "172.17.0.4",
 "IPPrefixLen": 16,
 "IPv6Gateway": "",
 "GlobalIPv6Address": "",
 "GlobalIPv6PrefixLen": 0,
 "MacAddress": "02:42:ac:11:00:04"
 }
 }
 }
 }
]

But, working as expected, there is a new secret object created on that node too.

ubuntu@swarmnode3:/run$ docker secret ls
ID NAME CREATED UPDATED
2rhdvw2o2g53ferldl47544io MYSECRETS About an hour ago About an hour ago

 

Next steps will lead to change all our entrypoint files to use this new secrets feature 🙂

 

REFERENCES:

Docker Releases
https://github.com/docker/docker/releases

Enabling Experimental Mode
https://github.com/docker/docker/tree/master/experimental#use-docker-experimental

Vagrant Docker Swarm Mode Testing Environment 
https://github.com/frjaraur/docker-swarmmode

 

Review: Docker Engine Version 1.13 New Features (Part.1 – docker system)

Docker Engine 1.13 new features Part.1 – docker system commands

I am quite interested in some of the new features announced in new Docker Engine 1.13. We will try to make a small review of the most important ones, or at least the ones that will make my life easier :).

At the time of writing this article, some of these features are only available in “experimental mode”. We will use 1.13.0-rc7 with daemon running with experimental support enabled (see references to learn how to enable it).

We start this series of articles with the new “docker system” command.

ubuntu@swarmnode1:~$ docker system --help

Usage: docker system COMMAND

Manage Docker

Options:
 --help Print usage

Commands:
 df Show docker disk usage
 events Get real time events from the server
 info Display system-wide information
 prune Remove unused data

As we can see in the help page, there are for commands available.

docker system df

Docker system df will show information about storage used in our host related to images, containers and volumes. This information is available for running and stopped containers.

Let’s see some examples. We will use an empty host for all examples for easy understand of data usage (you can use our vagrant environment docker-swarmmode, see references).

ubuntu@swarmnode1:~$ docker system df
TYPE TOTAL ACTIVE SIZE RECLAIMABLE
Images 0 0 0 B 0 B
Containers 0 0 0 B 0 B
Local Volumes 0 0 0 B 0 B

ubuntu@swarmnode1:~$ docker run -d busybox ping www.google.es
Unable to find image 'busybox:latest' locally
latest: Pulling from library/busybox
4b0bc1c4050b: Pull complete 
Digest: sha256:817a12c32a39bbe394944ba49de563e085f1d3c5266eb8e9723256bc4448680e
Status: Downloaded newer image for busybox:latest
aa0327042ba1065e0638f6d4d5ddfe241b029ba7d743d00a4a81bc7fec422006

ubuntu@swarmnode1:~$ docker system df
TYPE TOTAL ACTIVE SIZE RECLAIMABLE
Images 1 1 1.11 MB 0 B (0%)
Containers 1 1 0 B 0 B
Local Volumes 0 0 0 B 0 B

Latter we will see a more interesting example.

docker system df will show us the disk usage for downloaded and build images, stopped and running containers and all local volumes created.

docker system events

Docker system events show information about all docker engine events (it is the same as docker events command) .

As we already know, docker engine will generate events for almost all actions. Everytime a container is created, started/stopped or destroyed, events related to these actions are generated. Same happens when pulling or building images, volumes, networks, etc…

Here is a quick example

ubuntu@swarmnode1:~$ docker system events &
[1] 18176

ubuntu@swarmnode1:~$ docker run -d busybox ping www.google.es
856b75a829f5cc63c0a7aef919d1f91e60f054b5e0f9236ad98a9c40ae1f6f01

2017-01-15T12:16:31.354759539+01:00 container create 856b75a829f5cc63c0a7aef919d1f91e60f054b5e0f9236ad98a9c40ae1f6f01 (image=busybox, name=trusting_bell)
2017-01-15T12:16:31.363098524+01:00 network connect 13a0e22fb628078c032ab4732d5eed687fde7c62ee7720cdee9bd87fbb7dcbf9 (container=856b75a829f5cc63c0a7aef919d1f91e60f054b5e0f9236ad98a9c40ae1f6f01, name=bridge, type=bridge)
2017-01-15T12:16:31.447693344+01:00 container start 856b75a829f5cc63c0a7aef919d1f91e60f054b5e0f9236ad98a9c40ae1f6f01 (image=busybox, name=trusting_bell)

ubuntu@swarmnode1:~$ docker rm -fv 856b75a829f5cc63c0a7aef919d1f91e60f054b5e0f9236ad98a9c40ae1f6f01
2017-01-15T12:16:48.077769551+01:00 container kill 856b75a829f5cc63c0a7aef919d1f91e60f054b5e0f9236ad98a9c40ae1f6f01 (image=busybox, name=trusting_bell, signal=9)
2017-01-15T12:16:48.089993793+01:00 container die 856b75a829f5cc63c0a7aef919d1f91e60f054b5e0f9236ad98a9c40ae1f6f01 (exitCode=137, image=busybox, name=trusting_bell)
2017-01-15T12:16:48.119506809+01:00 network disconnect 13a0e22fb628078c032ab4732d5eed687fde7c62ee7720cdee9bd87fbb7dcbf9 (container=856b75a829f5cc63c0a7aef919d1f91e60f054b5e0f9236ad98a9c40ae1f6f01, name=bridge, type=bridge)
856b75a829f5cc63c0a7aef919d1f91e60f054b5e0f9236ad98a9c40ae1f6f01
2017-01-15T12:16:48.127383429+01:00 container destroy 856b75a829f5cc63c0a7aef919d1f91e60f054b5e0f9236ad98a9c40ae1f6f01 (image=busybox, name=trusting_bell)

ubuntu@swarmnode1:~$ kill %1

As a resume, we can say that docker system events works the same as docker events. I couldn’t find any difference between them 😐

docker system info

Docker system events show information about the docker engine and  (it is the same as docker info command) .

With docker system info we can review all configuration information related to docker engine. We will have different sections with information about different aspects of docker engine:

  • Number of containers created on this host engine.
  • Number of images available (remember tagging doesn’t count as they are the same).
  • Engine version.
  • Enabled drivers for graph-storage, logging and cgroups.
  • Enabled plugins (volume-storage, networking, etc…).
  • Swarm related information about its role in the cluster, the managers addresses, raft clustering consensus algorithm and internal certificates configuration.
  • Information about container runtime and versions.
  • Security options enabled for the engine.
  • Host OS information used for labeling this engine.
  • Docker Root location.
  • Debug (for client and engine), Experimental modes status.
  • Registries Associated.
  • Live Restore feature status.

This is an example ouput.

ubuntu@swarmnode1:~$ docker info 
Containers: 4
 Running: 3
 Paused: 0
 Stopped: 1
Images: 2
Server Version: 1.13.0-rc7
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 12
 Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins: 
 Volume: local
 Network: bridge host ipvlan macvlan null overlay
Swarm: active
 NodeID: ufhgrbz5br9lqtfjfukxi2kym
 Is Manager: true
 ClusterID: osklt6hzcrzp92ycg6hmcxaou
 Managers: 3
 Nodes: 4
 Orchestration:
 Task History Retention Limit: 5
 Raft:
 Snapshot Interval: 10000
 Number of Old Snapshots to Retain: 0
 Heartbeat Tick: 1
 Election Tick: 3
 Dispatcher:
 Heartbeat Period: 5 seconds
 CA Configuration:
 Expiry Duration: 3 months
 Node Address: 10.10.10.11
 Manager Addresses:
 10.10.10.11:2377
 10.10.10.12:2377
 10.10.10.13:2377
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 03e5862ec0d8d3b3f750e19fca3ee367e13c090e
runc version: 2f7393a47307a16f8cee44a37b262e8b81021e3e
init version: 949e6fa
Security Options:
 apparmor
 seccomp
 Profile: default
Kernel Version: 4.4.0-59-generic
Operating System: Ubuntu 16.04.1 LTS
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 1.45 GiB
Name: swarmnode1
ID: UGRA:FCQA:6RKE:L4PS:Z5BM:C3G4:6KHI:KECB:U6LY:NRF7:MV2B:TK5N
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
WARNING: No swap limit support
Experimental: true
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

You can notice that Experimental Mode is enabled and swarmnode1 is a manager on swarm cluster with ID osklt6hzcrzp92ycg6hmcxaou.

 

We found that docker system info and docker system events work the same as docker info and docker events, they could be aliases for same information. Sorry but I haven’t reviewed source code yet 😐

docker system prune

Finishing the review for docker system commands, we have docker system prune, which is quite interesting.

Docker system prune will clean all stopped containers, all volumes not used by at least one container, all networks not used by at least one container and all dangling images. We can even force to clean all unused images. This way, docker system prune will help docker engine maintenance. We have being doing this clean up works manually looping or filtering for docker rm, docker rmi or docker network|volume rm, but this command will do everything needed for us. Be careful because as uncle Ben said “With Great Power comes Great Responsibility”, and take a look at your docker images, containers, volumes and networks before executing docker prune because it will remove everything not needed …

A quick example.

ubuntu@swarmnode1:~$ docker run -d busybox ping www.google.es
ebe01b4a69e4e023892b500b9f2a7d2028c2674b0df9907537d8799aab0e7a1e

ubuntu@swarmnode1:~$ docker run -d busybox ping www.google.es
1048304de0e8be7279778a1509fdf31591e9f140e2355e6030c5720290417229

ubuntu@swarmnode1:~$ docker run -d busybox ping www.google.es
cb03d9bb00151029be99f7abff147969d773922114291d13521fd2c98324ae44

ubuntu@swarmnode1:~$ docker run -d busybox ping www.google.es
83c18ce521ba273f63f6532d1141ae6fb9d08f39b36ac6643bfa9e2c9662f632

ubuntu@swarmnode1:~$ docker stop ebe01b4a69e4e023892b500b9f2a7d2028c2674b0df9907537d8799aab0e7a1e 1048304de0e8be7279778a1509fdf31591e9f140e2355e6030c5720290417229 cb03d9bb00151029be99f7abff147969d773922114291d13521fd2c98324ae44 83c18ce521ba273f63f6532d1141ae6fb9d08f39b36ac6643bfa9e2c9662f632
ebe01b4a69e4e023892b500b9f2a7d2028c2674b0df9907537d8799aab0e7a1e
1048304de0e8be7279778a1509fdf31591e9f140e2355e6030c5720290417229
cb03d9bb00151029be99f7abff147969d773922114291d13521fd2c98324ae44
83c18ce521ba273f63f6532d1141ae6fb9d08f39b36ac6643bfa9e2c9662f632

ubuntu@swarmnode1:~$ docker system prune
WARNING! This will remove:
 - all stopped containers
 - all volumes not used by at least one container
 - all networks not used by at least one container
 - all dangling images
Are you sure you want to continue? [y/N] y
Deleted Containers:
83c18ce521ba273f63f6532d1141ae6fb9d08f39b36ac6643bfa9e2c9662f632
cb03d9bb00151029be99f7abff147969d773922114291d13521fd2c98324ae44
1048304de0e8be7279778a1509fdf31591e9f140e2355e6030c5720290417229
ebe01b4a69e4e023892b500b9f2a7d2028c2674b0df9907537d8799aab0e7a1e

Total reclaimed space: 0 B

Docker prune will report reclaimed space after executing for getting happy with the results of this maintenance task. You will get happier if your system is storing a large amount of big unused or dangling images and containers are writing a lot of data to its runtime or volume and you can remove those stopped containers.

Let’s see a better example using docker system df and docker system prune.

ubuntu@swarmnode1:~$ docker volume ls
DRIVER VOLUME NAME

ubuntu@swarmnode1:~$ docker run -d -ti -v TEST:/DATA busybox dd if=/dev/zero of=/DATA/100MB_file bs=104857600 count=1
9401cdc50ae56f67a78534f4bfdc593e37734c3ce9450930e19608ea99762244

ubuntu@swarmnode1:~$ docker system df 
TYPE TOTAL ACTIVE SIZE RECLAIMABLE
Images 1 1 1.11 MB 0 B (0%)
Containers 1 0 0 B 0 B
Local Volumes 1 1 104.9 MB 0 B (0%)

ubuntu@swarmnode1:~$ docker system df -v
Images space usage:

REPOSITORY TAG IMAGE ID CREATED SIZE SHARED SIZE UNIQUE SIZE CONTAINERS
busybox latest 7968321274dc 37 hours ago 1.11 MB 0 B 1.11 MB 1

Containers space usage:

CONTAINER ID IMAGE COMMAND LOCAL VOLUMES SIZE CREATED STATUS NAMES
9401cdc50ae5 busybox "dd if=/dev/zero o..." 1 0 B 7 seconds ago Exited (0) 6 seconds ago peaceful_dijkstra

Local Volumes space usage:

VOLUME NAME LINKS SIZE
TEST 1 104.9 MB

ubuntu@swarmnode1:~$ docker system prune -f
Deleted Containers:
9401cdc50ae56f67a78534f4bfdc593e37734c3ce9450930e19608ea99762244

Deleted Volumes:
TEST

Total reclaimed space: 104.9 MB

In this example we used a busybox image for creating a container that creates a 100MB file on /DATA, mounted on runtime as volume. Docker system df have shown us that our container haven’t stored anything big enough for showing statistics on ouput, but we have used a new volume where we created a 100MB file (/DATA/ 100MB_file). Executing docker system prune in this example reclaimed 104.9 MB of storage in /var/lib/docker (default mount point for docker root).

 

We have reviewed some basic new features in new docker version 1.13 and hope this will help anyone who had read this post 🙂 I will review other quite interesting new features in next posts.

 

REFERENCES:

Docker Releases
https://github.com/docker/docker/releases

Enabling Experimental Mode
https://github.com/docker/docker/tree/master/experimental#use-docker-experimental

Vagrant Docker Swarm Mode Testing Environment 
https://github.com/frjaraur/docker-swarmmode

 

 

Raspberry Alpine Linux 3.5 Docker Host

Docker on your raspberry is more than an experiment today, we can use docker for easily deployment of some home services such as owncloud, torrent downloader or VPN server. Docker allow us to reproduce environments for each application, avoiding library and binaries problems. All packages required for an application will be already configured on the image that will create the container.

raspberry-alpine-docker

Docker on your raspberry is more than an experiment today, we can use docker for easily deployment of some home services such as owncloud, torrent downloader or VPN server. Docker allow us to reproduce environments for each application, avoiding library and binaries problems. All packages required for an application will be already configured on the image that will create the container.

With Docker we can create applications that will have a high resilience behavior to errors using various hosts in a swarm cluster (with 3 raspberries you could have small services with a quick response to errors or even load balancing them in the cluster).

Installing Alpine 3.5 on Raspberry

1) Download (https://fr.alpinelinux.org/alpine/v3.5/releases/armhf/alpine-rpi-3.5.0-armhf.tar.gz) alpine version for armf type hosts. We will download a tarball named alpine-rpi-3.5-armhf.tar.gz.

2) Mount an empty SD card on your computer and create a small partition with at least 512MB (this will become your /boot later).

3) Create a FAT32 filesystem on the newly created partition and mark it as bootable.

4) Check that the filesystem is ok for extracting de alpine OS there:

# fdisk -l

Disk /dev/sdd: 32.0 GB, 32010928128 bytes
  102 heads, 36 sectors/track, 17026 cylinders
  Units = cylinders of 3672 * 512 = 1880064 bytes

Device Boot Start End Blocks Id System
  /dev/sdd1 * 1 287 524288 b Win95 FAT32
 5) Mount the created partition on your host and extract the tarball on it.

$ cd /_WHEREVER_YOU_HAVE_MOUNTED_THE_PARTITION_
  $ tar -zxvf /_WHEREVER_YOU_HAVE_YOUR_TAR_FILE/alpine-rpi-3.5.0-armhf.tar.gz

6) Now you are ready to boot from SD card on your raspberry.

7) Boot Alpine from SD will give us a On-Memory System so we will have to commit changes until we create a root partition and write down and change de root filesystem to boot.

8) Login as root without password and execute alpine-setup to configure your system.

# setup-alpine

9) Change your keyboard, timezone and network settings and commit the changes to the SD card. I always install openssh and chrony (raspberry doesn’t have hardware clock so I enable software clock on boot and disable hardware clock syncing).

# rc-update add swclock boot && rc-update del hwclock boot

10) Check if opensshd and crony are enabled on boot as well as swclock.

# rc-update

11) Update installed packages before commiting changes

# apk --update upgrade

12) Add support for creating ext4 filesystems for creating root filesystem in ext4 format.

# apk add e2fsprogs

13) Commit every change done to this point and reboot to check that everything is ok.

# lbu commit

# reboot

14) Login to the host and then we create a new partition for the root filesystem.

15) Then we will create a ext4 filesystem on the newly created partition

# fdisk -l /dev/mmcblk0

Disk /dev/mmcblk0: 32.0 GB, 32010928128 bytes
 102 heads, 36 sectors/track, 17026 cylinders
 Units = cylinders of 3672 * 512 = 1880064 bytes

Device Boot Start End Blocks Id System
 /dev/mmcblk0p1 * 1 287 524288 b Win95 FAT32
 /dev/mmcblk0p2 287 4542 7813800 83 Linux

# mkfs.ext4 /dev/mmcblk0p2

16) Now we will do a install to that partition using setup-disk and the data from the boot partition.

# mkdir /tmp_rootfs
 # mount /dev/mmcblk0p2 /tmp_rootfs
 # setup-disk -o /media/mmcblk0p1/$(hostname).apkovl.tar.gz /tmp_rootfs

NOTE: Execution should notice some errors about syslinux and extlinux but we can ignore them.

17) We will add the boot partition to the fstab file in /tmp_rootfs/etc/fstab

/dev/mmcblk0p1 /media/mmcblk0p1 vfat defaults 0 0

18) Them reboot (you can umount /tmp_rootfs, but you can not remount /media/mmcblk0p1 to make it writable).

19) Once logged again as root using On-Memory system, we will remount /media/mmcblk0p1 to make it writable for been able to change root fs.

# mount -o remount,rw /media/mmcblk0p1

20) Finally we will edit /media/mmcblk0p1/cmdline.txt to specify the root fs to use

We will add root=/dev/mmcblk0p2 at the end of the cmdline.

Reboot the system to check our locally installed Alpine Linux version.

At this point, we have alpine linux 3.5 installed on our raspberry (tested on raspberry 2, but changes may occur in cmdline.txt filename).

Installing Docker

Installing Docker using official Alpine repository is quite simple, but this probably will not install the latest official Docker version.

In this guide we will install Alpine’s repository version.

1) We will edit /etc/apk/repositories to uncomment  the apline 3.5 community repository.

http://mirrors.2f30.org/alpine/v3.5/community

2) We will install now docker packages

# apk add --update docker

3) Once installed, enable Docker on startup (on default stage).

# rc-update add docker

4) We will start docker engine from init.d script.

# /etc/init.d/docker start

5) Finally we can check docker started version (at time of writing this article, alpine version was 1.12.3)

# docker version

Client:
  Version: 1.12.3
  API version: 1.24
  Go version: go1.7.3
  Git commit: v1.12.3
  Built: 
  OS/Arch: linux/arm

Server:
  Version: 1.12.3
  API version: 1.24
  Go version: go1.7.3
  Git commit: v1.12.3
  Built: 
  OS/Arch: linux/arm

 

Hope this will help someone as it does to me write down my steps 🙂

 

REFERENCES:

Installing Alpine on Raspberry Pi
https://wiki.alpinelinux.org/wiki/Raspberry_Pi

Docker on Alpine Linux
http://wiki.alpinelinux.org/wiki/Docker