LXC inside linodes with 'visible' (public and private) IPs
Is it possible to run LXC inside my linodes in a manner so that I can assign IP addresses that are 'visible' from outside the linode hosting the containers?
I have a linode, which has a public IP and private IPs.
I have installed lxc packages (I'm using Ubuntu Trusty) and fired up lxc container with default configuration. That works fine. I got lxcbr0 interface on my linode host, I got iptables rules that do NAT for the lxc network (where lxcbr0 is the gateway), and everything works ok. The lxc gets IP from 10.0.3.0/24 network, and can reach internet (I can ping google and so on). Of course, I can't reach lxc from the internet (without doing port forwarding using iptables or some other means).
Then Linode gave me another (private) IP for my linode, from 192.168.x.x range. I changed my linode host configuration - I added br0 interface which is bridge for eth0:
root@linodehost:~# cat /etc/network/interfaces
auto lo
iface lo inet loopback
auto br0
iface br0 inet dhcp
bridge_ports eth0
bridge_stp off
auto br0:1
iface br0:1 inet static
address 192.168.aa.bb/17
This is how my routes look:
root@linodehost:~# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 176.58.111.1 0.0.0.0 UG 0 0 0 br0
10.0.3.0 0.0.0.0 255.255.255.0 U 0 0 0 lxcbr0
176.58.111.0 0.0.0.0 255.255.255.0 U 0 0 0 br0
192.168.128.0 0.0.0.0 255.255.128.0 U 0 0 0 br0
root@linodehost:~#
This all works fine, I can access the public ip (176.58.1xx.yyy) of my linode from the internet, and private IP from other linodes.
However when I change my lxc confuration so that it uses br0:
root@linodehost:~# grep network /var/lib/lxc/ticketshop-dev/config
lxc.network.type = veth
#lxc.network.link = lxcbr0
lxc.network.link = br0
lxc.network.flags = up
lxc.network.hwaddr = 00:16:3e:XX:XX:XX
root@linodehost:~#
And then configure linode-assigned private IP to the container:
root@lxc01:~# cat /etc/network/interfaces
auto lo
iface lo inet loopback
auto eth0
#iface eth0 inet dhcp
iface eth0 inet static
address 192.168.203.66/17
# netmask 255.255.255.0
gateway 192.168.aa.bb # the private IP address on br0:1 on the host
dns-nameserver 8.8.8.8
root@ticketshop-dev:~# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.207.60 0.0.0.0 UG 0 0 0 eth0
192.168.128.0 0.0.0.0 255.255.128.0 U 0 0 0 eth0
I can ping the gateway:
root@ticketshop-dev:~# ping 192.168.207.60
PING 192.168.207.60 (192.168.207.60) 56(84) bytes of data.
64 bytes from 192.168.207.60: icmp_seq=1 ttl=64 time=0.057 ms
64 bytes from 192.168.207.60: icmp_seq=2 ttl=64 time=0.092 ms
^C
--- 192.168.207.60 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.057/0.074/0.092/0.019 ms
But I can't get out:
root@ticketshop-dev:~# ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
From 192.168.207.60: icmp_seq=2 Redirect Host(New nexthop: 192.168.207.60)
From 192.168.207.60: icmp_seq=3 Redirect Host(New nexthop: 192.168.207.60)
From 192.168.207.60: icmp_seq=4 Redirect Host(New nexthop: 192.168.207.60)
^C
--- 8.8.8.8 ping statistics ---
4 packets transmitted, 0 received, 100% packet loss, time 2999ms
What am I doing wrong? Is such configuration possible on Linode network? I'm assuming that I'm configuring wrong gateway inside my lxc, but I'm not sure what would the proper gateway be?
3 Replies
0.0.0.0 192.168.207.60 0.0.0.0 UG 0 0 0 eth0
All traffic not within 192.168.203.66/17 network needs to go to the publically addressable gateway, not the private one.
Also, I haven't done it yet myself, but if you have Linode Support assign you a /64 IPv6 pool, you can simply give each container its own IPv6 address that they can use.
But, what's confusing is that I can't even connect to my other hosts' private IPs.