Network help with unprivileged lxc containers on -current

Rinndalir · 09-07-2015, 03:19 PM

I have created an unprivileged lxc container but now I'm stuck getting networking setup on the host.

On my host I added this to rc.inet1.conf:

# for lxc networking
IFNAME[5]="br0"
BRNICS[5]="eth0"
USE_DHCP[5]="yes"

On the guest I have this:

lxc.network.type = veth
lxc.network.link = br0
lxc.network.flags = up

To complicate things I have two eth0 aliases:
eth0:1
eth0:2

When I reboot I get the br0 interface and I have a eth0 and eth0:1
(eth0:2 does not appear)

The host networking works but I never get my guest lxc container network working.
Don't know what I'm missing.

franzen · 09-08-2015, 03:53 AM

It depends how you want to setup your Networking in the client. I use static IPs for my containers. You may try this setup

Host
IFNAME[5]="br0"
BRNICS[5]="br0"
IPADDR[5]="10.0.1.1"
NETMASK[5]="255.255.255.0"

Guest
lxc.network.type = veth
lxc.network.flags = up
lxc.network.link = br0
lxc.network.name = eth0
lxc.network.ipv4.gateway = auto
lxc.network.ipv4 = 10.0.1.2/24
lxc.network.hwaddr = 00:aa:11:00:00:01

Rinndalir · 09-08-2015, 09:59 AM

Quote:

Originally Posted by franzen

It depends how you want to setup your Networking in the client. I use static IPs for my containers. You may try this setup

Host
IFNAME[5]="br0"
BRNICS[5]="br0"
IPADDR[5]="10.0.1.1"
NETMASK[5]="255.255.255.0"

Is this working for you now? According to rc.inet1.conf this is wrong.
BRNICS[5]="eth0" is the example given in rc.inet1.conf

From rc.inet1.conf:

# Note the added "BRNICS" variable which contains a space-separated list
# of the physical network interfaces you want to add to the bridge.

I never followed along on the progress of containers. And I am wondering if I should bother. Are they in wide use? Or should I look at VMs instead? How are you using lxc? I need a musl-libc environment and I thought lxc might be a suitable approach.

I am also considering using tuntap instead of a bridge.
Also, I don't like having interfaces in promiscuous mode with bridges.

franzen · 09-09-2015, 10:10 AM

This way it works me. BRNICS="eth0" didn't work, i didn't read the documentation.
Containers are lightwight, lxc needs no Patch for the Kernel. Don't know how widely used lxc is,
Other containervirtualisations like openvz used to be widely used.
I use lxc as web-server and mail-server, migrating to new Host-servers is easy.

Rinndalir · 09-09-2015, 01:09 PM

Quote:

Originally Posted by franzen

This way it works me. BRNICS="eth0" didn't work, i didn't read the documentation.
Containers are lightwight, lxc needs no Patch for the Kernel. Don't know how widely used lxc is,
Other containervirtualisations like openvz used to be widely used.
I use lxc as web-server and mail-server, migrating to new Host-servers is easy.

Curiouser and curiouser. Can you post the ifconfig -a from your host?

I tested the code from /etc/rc.d/rc.inet1 and created a tst case. First I'll post the output from my test case followed by the code. The code was cut-n-paste from /etc/rc.d/rc.inet1
I added the echos of course.

I am not clear on if the order of the brctl commands is significant. I would expect that it matters.

bash ./br-tst.sh
/sbin/brctl addbr br0
/sbin/ifconfig eth0 down
/sbin/ifconfig eth0 0.0.0.0 promisc up
/sbin/brctl addif br0 eth0

Code I used to show how the bridge gets setup on the host.

Code:

cat ./br-tst.sh 
MAXNICS=6
IFNAME[1]="eth0"
IFNAME[2]="eth0:1"
IFNAME[3]="eth0:2"
IFNAME[5]="br0"
BRNICS[5]="eth0"


# Function to assemble a bridge interface.
br_open() {
    echo "setting up interface $1"
    # argument is 'i' - the position of this interface in the IFNAME array.
    echo "/sbin/brctl addbr ${IFNAME[$1]}"
    for BRIF in $(echo ${BRNICS[$1]}); do
        echo "/sbin/ifconfig $BRIF down"
	echo "/sbin/ifconfig $BRIF 0.0.0.0 promisc up"
	echo "/sbin/brctl addif ${IFNAME[$1]} $BRIF"
    done
}
		    
if_up() {
    # Determine position 'i' of this interface in the IFNAME array:
    i=0
    while [ $i -lt $MAXNICS ]; do
        [ "${IFNAME[$i]}" = "${1}" ] && break
	i=$(($i+1))
    done
    # If the interface is a bridge, then create it first:
    [ -n "${BRNICS[$i]}" ] && br_open $i
}

# Function to start the network:
start() {

    for i in ${IFNAME[@]} ; do
        if_up $i
    done
}
	    
start

franzen · 09-10-2015, 02:10 AM

Quote:

Originally Posted by Rinndalir

Curiouser and curiouser. Can you post the ifconfig -a from your host?

ifconfig -a
br0: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST> mtu 1500
inet 10.0.1.1 netmask 255.255.255.0 broadcast 10.0.1.255
ether fe:b0:9b:b1:50:af txqueuelen 0 (Ethernet)
RX packets 219627 bytes 79846700 (76.1 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 234060 bytes 425040160 (405.3 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet EXTERNALIP netmask 255.255.255.0 broadcast EXTERNALBROADCAST
ether MACADRESS txqueuelen 1000 (Ethernet)
RX packets 426124 bytes 439413724 (419.0 MiB)
RX errors 0 dropped 1446 overruns 0 frame 0
TX packets 309592 bytes 90462476 (86.2 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 16 memory 0x80400000-80420000

lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
loop txqueuelen 0 (Local Loopback)
RX packets 22 bytes 2000 (1.9 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 22 bytes 2000 (1.9 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

mail2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:c5:81:1f:55:c0 txqueuelen 1000 (Ethernet)
RX packets 162054 bytes 47024816 (44.8 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 181250 bytes 377281109 (359.8 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

web2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:b0:9b:b1:50:af txqueuelen 1000 (Ethernet)
RX packets 15677 bytes 1000603 (977.1 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 6177 bytes 4745979 (4.5 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

Rinndalir · 09-10-2015, 11:59 AM

Quote:

Originally Posted by franzen

ifconfig -a

Thanks, I think I will try to set things up using static ips as you have done.