Iocage Jails

This post contains the information I have missed when I have started to use iocage for jails management on FreeBSD.

Disclaimer: I am relatively new to FreeBSD and while what is written below works for me it is most probably not the only (or even a right) way to do things.

Jails

Jails is a killer feature of FreeBSD. One can think about jail as about chroot on steroids, or as about a very light-weight virtual machine. A jail provides an environment isolated from the rest of the system where FreeBSD is running (usually; one can run, for example, Linux in a jail).

I run a web server in a jail, which does SSL and for some URLs reverse proxy job, passing requests to other applications in other jails.

Iocage

While it is absolutely possible to deal with jails using only tools built in the FreeBSD, I am using iocage as my jail manager (pkg install sysutils/iocage). As far as I understand, iocage needs ZFS which is not a problem for me since all my disks are ZFS. I use OpenZFS (and it is built into FreeBSD 13.1).

Network interfaces for jails

I use cloned network interfaces for jails. Basically, the following lines in /etc/rc.conf

cloned_interfaces="lo1 lo2"
ifconfig_lo1="192.168.a.b/24"
ifconfig_lo1_ipv6="inet6 <ip6 address 1> prefixlen 64"
ifconfig_lo2="192.168.a.c/24"
ifconfig_lo2_ipv6="inet6 <ip6 address 2> prefixlen 64"

will create the interfaces lo1 and lo2 with IPv4 addresses 192.168.a.b, 192.168.a.c and network mask 255.255.255.0 (24 bits). If you have some internal network on your FreeBSD machine (as I do), make sure it does not clash with the network 192.168.a.0 used by jails. My system also runs IPv6, so I also assign to the interfaces IPv6 addresses from the block, allocated to me by the provider.

I think that the cloned interfaces (i.e. interfaces not backed up by any hardware) can be brought up without reboot by service netif cloneup, but, frankly speaking, I do not remember.

After the interfaces are created, the ifconfig command for them should show the assigned addresses.

Creation of jail

A jail can be created like this: iocage create -r 13.1-RELEASE -n <jail name> resolver="nameserver 9.9.9.9;options edns0" ip4_addr=192.168.a.b ip6_addr=<ip6 address 1> host_hostname=<host name> . This command will create a new jail, running FreeBSD 13.1, with the given name, with the hostname set to the given value, with the given addresses. The value of resolver goes to /etc/resolv.conf in jail, with semicolons (;) changed to the new lines (\n). Please notice that if you run local_unbound on your main system, it will not be by default accessible from jail. I do not like when my jails talk with services on my main system so I usually specify Quad 9 as a DNS server.

I do not remember if the jails are started immediately after creation; if they are not, the jail can be started with iocage start <jail name>.

When the jail is started, one might connect to it (as a root) using iocage console <jail name>, and use it as a normal FreeBSD system.

Networking in jail with pf

Warning: to have things below working, you will need configured and enabled pf on your system.

After the jail is created, there will be no (IPv4) network in it, as it has an IPv4 address in a private network. It is possible to setup NAT for jails with pf. To do that put the following in /etc/pf.conf:

jail = 192.168.a.b
...
nat on $ext_if inet from $jail to any -> ($ext_if)

This line supposes that you have an external network interface defined in /etc/pf.conf, for example, ext_if = "re0". After that, pf rules should be reloaded with pfctl -f /etc/pf.conf, and then you should have (IPv4) networking in a jail.

Port forwarding to jail with pf

Let us say that you have jailed network service, listening on (IPv4) TCP port 1234. However, the jail has an (IPv4) address in a private network, and NAT works only for outbound connections. How the connections to (IPv4) TCP port 1234 of the main system can be redirected into the jail? Again, this can be achieved with pf rule:

rdr on $ext_if inet proto tcp from any to ($ext_if) port 1234 -> $jail

When pf rules are reloaded with pfctl, the (IPv4) connections to the TCP port 1234 of the main system will be redirected into the jail.

Updating the jail

  • If you want to update FreeBSD system, running in a jail, to the latest patchlevel, use iocage update <jail name>.
  • If you want to upgrade the packages, installed in a jail, use iocage pkg <jail name> upgrade (make sure that DNS in the jail works).
  • If you want to upgrade FreeBSD release in a jail, you are on your own. Go RTFM.

Mounting the host directories into a jail

Sometimes you need to mount host directory into a jail (for example, I have a directory which the main system serves to internal network using Samba, and the same directory served over WebDav to the Internet by one of the jails – explain to me that I am wrong, please!). The functionality, somewhat similar to Linux bind mounts, can be achieved with nullfs.

The iocage program has fstab command which can help: iocage fstab -a <jail name> <host directory> <jail directory> nullfs rw 0 0. Make sure that the mount point (“jail directory”) does exist in the jail. After the jail is restarted (iocage restart <jail name>) the host directory should be visible in the jail under the specified mount point. Replace rw with ro for read-only access (and, in general, you can provide any line in fstab format).

The problem with changing IPv6 block

Unfortunately, my provider changes every now and then (even when the system is running) the IPv6 block, allocated to me. This leads to the situation, where jails are losing IPv6 connectivity, because the IPv6 addresses, assigned to them, are no longer routed to me. I solve this problem by asking cron to run the script, which does the following:

  1. Check which IPv6 block is allocated to me.
  2. Check one by one IPv6 addresses of lo* interfaces.
  3. If the address is in the allocated block, do nothing.
  4. Otherwise, calculate the proper (IPv6) address, assign it (with ifconfig) to the interface.
  5. Delete the old (IPv6) address from the interface (with ifconfig ... -alias).
  6. Assign the new (IPv6) address to the corresponding jail (with iocage set ip6_addr=...).
  7. Restart the jail (with iocage restart).

After this script is run, IPv6 works in jails again. And if allocated IPv6 block did not change, nothing happens.

comments powered by Disqus