Even if you used ebtables to filter out containers talking to each others' MAC addresses, wouldn't they be able to send broadcast or multicast packets to communicate with each other?
I guess it's not clear to me if the vulnerability/bug/whatever here is "two conspiring containers can establish a covert channel" or "a malicious container can send normal-looking traffic to a container, bypassing that container's firewall rules."
It does seem like the right answer is unique bridge networks per container. On physical networks, it's hard to prevent two untrusted devices on the same L2 domain from establishing a covert channel. (And it's hard to prevent two networked untrusted, conspiring devices anywhere on the internet from establishing a covert channel, if they're trying hard enough.)
> Even if you used ebtables to filter out containers talking to each others' MAC addresses, wouldn't they be able to send broadcast or multicast packets to communicate with each other?
I'm afraid my initiation to ebtables (and iptables) was yesterday, so feedback such as this is highly appreciated. I will explore this and add a test to the repo.
> I guess it's not clear to me if the vulnerability/bug/whatever here is "two conspiring containers can establish a covert channel" or "a malicious container can send normal-looking traffic to a container, bypassing that container's firewall rules."
In the context of a sandbox, I purport the vulnerability is that a malicious container can bypass the firewall. The covert channel is largely inevitable because ultimately the containers can modulate cpu usage to communicate via cpu timing (I read this in a security paper somewhere, not sure how actual implementation works).
The impact of bypassing the firewall can be serious if the non-malicious container exposes any service without proper authentication (as is often done behind VPCs in the cloud).
Given that a high number of official containers contain known vulnerabilities (according to dockerhub). This is a potential issue for a large number of docker users.
> It does seem like the right answer is unique bridge networks per container.
My thinking is along the same lines, although I'm wondering if a container can communicate with the host if L2 packets may be able to traverse the network bridges. If anyone has input here, it'd be greatly appreciated.
When the sysctl net.bridge.bridge-nf-call-iptables=1 is set, bridged packets will traverse iptables rules. I believe this is the default on modern Linux kernels, but may not be set by default on RHEL/CentOS systems.
The iptables filter table has three chains: INPUT, OUTPUT, FORWARD. In a traditional non-bridge context:
- INPUT means packets entering the host
- OUTPUT means packets leaving the host
- FORWARD means packets being forwarded through the host which are not destined for the host itself.
In the context of a bridge, FORWARD means forwarding packets from one bridge port to another. You can test the simplest case of this with:
iptables -I FORWARD -j DROP
by adding that rule, two containers on the same host cannot communicate with each other despite the fact that their communication did not require any traditional routing from the host. This is because the bridge traffic also traversed iptables rules.
> Even if you used ebtables to filter out containers talking to each others' MAC addresses, wouldn't they be able to send broadcast or multicast packets to communicate with each other?
ebtables supports «broadcast» as a destination to match ethernet frames on.
I do not know IPtables well (because the syntax is very dumb and vile IMNSHO - PF is so much better), but I digress. The rule as outlined would likely allow broadcast and multicast as the rule is specific to src/dst pairs so there would be no match for src/bcast.
Is the syntax as hard to parse as iptables, if so, still sucks :)
I have a friend that I worked with at now large company (now) with back in the startup days. Guy worked at ATT and has name on RFCs for things that make all this stuff work. We where doing the startup thing, ie the guys that understand it own it, for the company. No IT so you do it since you know it...firewall and vpn, etc (he wrote routing code, I was CSE), but startup so... It made me laugh so hard when he said “iptables makes no sense. Black magic. Let’s just use PFsense”. That had always been my view but to hear someone that wrote the stuff that makes this Internet magic work agree with me was joyful.
I guess it's not clear to me if the vulnerability/bug/whatever here is "two conspiring containers can establish a covert channel" or "a malicious container can send normal-looking traffic to a container, bypassing that container's firewall rules."
It does seem like the right answer is unique bridge networks per container. On physical networks, it's hard to prevent two untrusted devices on the same L2 domain from establishing a covert channel. (And it's hard to prevent two networked untrusted, conspiring devices anywhere on the internet from establishing a covert channel, if they're trying hard enough.)