I’ve been using Docker a fair bit for work at ActiveState recently. It’s quite nice and makes creating and deploying services much simpler.
However, it can also be incredibly annoying when I’m using it locally on my desktop. By default, the Docker daemon (dockerd) messes with iptables in order to allow docker images to connect to the interwebs. But if you already have a firewall in place there’s a good chance that this won’t work. So every time I want to use Docker I disable my ferm-based firewall and restart the Docker daemon. Then when I’m done using Docker I bring the firewall back up. Tedious and unsafe!
Figuring Out What dockerd Does
Docker creates a new virtual network interface named docker0 and then sets up iptables to give this interface access to the internet. I could not find any documentation on what dockerd actually does with iptables. Fortunately, this is easy to figure out by dumping the iptables rules when dockerd is running:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
| $> sudo iptables -L -n -v
Chain INPUT (policy ACCEPT 30 packets, 5160 bytes)
pkts bytes target prot opt in out source destination
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
0 0 DOCKER-USER all -- * * 0.0.0.0/0 0.0.0.0/0
0 0 DOCKER-ISOLATION-STAGE-1 all -- * * 0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT all -- * docker0 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
0 0 DOCKER all -- * docker0 0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT all -- docker0 !docker0 0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT all -- docker0 docker0 0.0.0.0/0 0.0.0.0/0
Chain OUTPUT (policy ACCEPT 36 packets, 5305 bytes)
pkts bytes target prot opt in out source destination
Chain DOCKER (1 references)
pkts bytes target prot opt in out source destination
Chain DOCKER-ISOLATION-STAGE-1 (1 references)
pkts bytes target prot opt in out source destination
0 0 DOCKER-ISOLATION-STAGE-2 all -- docker0 !docker0 0.0.0.0/0 0.0.0.0/0
0 0 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0
Chain DOCKER-ISOLATION-STAGE-2 (1 references)
pkts bytes target prot opt in out source destination
0 0 DROP all -- * docker0 0.0.0.0/0 0.0.0.0/0
0 0 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0
Chain DOCKER-USER (1 references)
pkts bytes target prot opt in out source destination
0 0 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0
$> sudo iptables -L -n -v -t nat
Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
0 0 DOCKER all -- * * 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 8 packets, 560 bytes)
pkts bytes target prot opt in out source destination
0 0 DOCKER all -- * * 0.0.0.0/0 !127.0.0.0/8 ADDRTYPE match dst-type LOCAL
Chain POSTROUTING (policy ACCEPT 8 packets, 560 bytes)
pkts bytes target prot opt in out source destination
0 0 MASQUERADE all -- * !docker0 172.17.0.0/16 0.0.0.0/0
Chain DOCKER (2 references)
pkts bytes target prot opt in out source destination
0 0 RETURN all -- docker0 * 0.0.0.0/0 0.0.0.0/0
|
The next step is to translate this into ferm rules and integrate it into my existing ferm config.
Making It Work
Since I wanted to make ferm set up the Docker rules, I had to tell dockerd to stop doing it itself when the daemon was started.
Depending on what init system you’re using, there are two ways pass options to dockerd. If your system is using systemd, the daemon is configured via the /etc/docker/daemon.json. This disables the iptables setup:
1
2
3
| {
"iptables": false
}
|
For other init systems (sysv and upstart) you should edit /etc/default/docker and add --iptables=false to DOCKER_OPTS.
The Docker rules translate to the following ferm config (disclaimer: I am not an iptables or ferm expert so this may be a bit wrong):
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
| domain ip {
table nat {
chain DOCKER {
interface docker0 RETURN;
}
chain PREROUTING {
mod addrtype dst-type LOCAL jump DOCKER;
}
chain POSTROUTING {
saddr 172.17.0.0./16 outerface !docker0 MASQUERADE;
}
chain OUTPUT {
destination !127.0.0.0/8 {
mod addrtype dst-type LOCAL jump DOCKER;
}
}
}
table filter {
chain DOCKER {
}
chain DOCKER-USER {
}
chain DOCKER-ISOLATION-STAGE-1 {
interface docker0 outerface !docker0 jump DOCKER-ISOLATION-STAGE-2;
RETURN;
}
chain DOCKER-ISOLATION-STAGE-2 {
outerface docker0 DROP;
RETURN;
}
chain FORWARD {
jump DOCKER-USER;
jump DOCKER-ISOLATION-STAGE-1;
outerface docker0 ACCEPT;
outerface docker0 jump DOCKER;
interface docker0 outerface !docker0 ACCEPT;
interface docker0 outerface docker0 ACCEPT;
}
}
}
|
A Not So Great Solution
This works. I can enable my firewall with sudo service ferm restart and use Docker normally. Containers are able to access the internet. Yay!
One problem, however, is that the easiest way to make this work was to put the docker rules before all my other rules. This probably means my docker containers are a bit more exposed than is ideal. However, I only use Docker to build containers and test them locally, so that’s okay for now.
But the bigger problem is that this will almost certainly break. In the short time I’ve been using Docker (about 6 months) the way it does networking has changed at least once. A few months back dockerd would set up two virtual interfaces, docker0 and docker_gwbridge. The iptables rules it used were a bit different then as well.
So it seems likely that dockerd might change what it does again and my config will be broken. This is all quite annoying. I’m not sure what the best solution is, but at the very least it’d be good to see Docker document exactly what these rules need to be (and better yet, what they’re doing at a higher level).