Sometime back I came across VirtualBox, an excellent replacement for commercial Vmware. I installed few OSes in VirtualBox and found the performance pretty good. In my opinion the performance of Windows virtual machines is much better in VirtualBox, perhaps because of the installed guest additions.
Immediately a thought came into my mind. What if I want to create an internetwork of VirtualBox virtual machines and QEMU virtual machines.
The default NAT networking in VirtualBox does not solve any purpose. The only useful networking option I found was the “Host Interface”. I went through several documents on VirtualBox website and realise that I needed to use bridging in order to do this. All the documents explained beautifully how to accompalish bridging with the ethernet interface on the host machine, but this is not what I wanted. I wanted the virtual machines to be in an isolated network connected together using VDE Switch as I was doing with QEMU. Searching on google for quite sometime did not help either, so I decided to give it a try.
Following is what I did to accompalish this. After executing the following –
- QEMU virtual machines can talk to VirtualBox virtual machines and vice-versa.
- Host can talk to all virtual machines and vice-versa.
- All virtual machines can connect to internet via the hostmachine.
As suggested on the VirtualBox website, in order to use bridging I need the uml-utilities and bridge-utilities packages:
$ sudo apt-get install uml-utilities bridge-utils
I need to load the tun/tap module and the kqemu module for QEMU
$ sudo modprobe kqemu $ sudo modprobe tun
Next I wanted to make the “tun” device writeable by the users, so I out myself into the “vboxusers” group and gave the group write permission on the device.
$ sudo chgrp vboxusers /dev/net/tun $ sudo chmod g+rw /dev/net/tun
Then I created couple of persistent tap interfaces owned by an ordinary user:
$ sudo tunctl -t tap0 -u ajitabhp $ sudo tunctl -t tap1 -u ajitabhp
Next I started the VDE Switch connected to the tap0 interface and make the control file world writeable:
$ vde_switch -tap tap0 -daemon $ chmod 666 /tmp/vde.ctl
A bridge then needs to be created:
$ sudo brctl addbr br0
All interfaces which are connected to network segments to be bridged are to be put into promiscuous mode.
$ sudo ifconfig tap0 0.0.0.0 promisc
Add the tap0 interface to the bridge and start the DNSMASQ server to listen on br0 interface.
$ sudo brctl addif br0 tap0 $ sudo /usr/sbin/dnsmasq --log-queries --user=named --dhcp-leasefile=/var/lib/misc/vbox-dhcpd.leases --dhcp-range=10.111.111.100,10.111.111.199,255.255.255.0,10.255.255.255,8h --interface=br0 --domain=virtual.lan --addn-hosts=/etc/my-host
Next configure the br0 interface with an ip address. This br0 interface will work as the gateway for the virtual machines.
$ sudo ifconfig br0 10.111.111.254 netmask 255.255.255.0 up
Now add the tap1 interface to the bridge and bring it up.
$ sudo brctl addif br0 tap1 $ sudo ifconfig tap1 up
Enable IP forwarding on the host machine and setup MASQUERADING:
$ sudo su - # echo "1" > /proc/sys/net/ipv4/ip_forward $ sudo iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
Following is how various interfaces look like after finishing this setup.
$ brctl show br0
bridge name bridge id STP enabled interfaces
br0 8000.ae7729b64a80 no tap0
tap1
$ ifconfig
br0 Link encap:Ethernet HWaddr 6E:EA:39:83:8C:D4
inet addr:10.111.111.254 Bcast:10.111.111.255 Mask:255.255.255.0
inet6 addr: fe80::6cea:39ff:fe83:8cd4/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:106 errors:0 dropped:0 overruns:0 frame:0
TX packets:82 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:10696 (10.4 KiB) TX bytes:10165 (9.9 KiB)
eth0 Link encap:Ethernet HWaddr 00:12:F0:28:6E:C3
inet addr:192.168.0.176 Bcast:192.168.0.255 Mask:255.255.255.0
inet6 addr: fe80::212:f0ff:fe28:6ec3/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:426 errors:32 dropped:32 overruns:0 frame:0
TX packets:327 errors:0 dropped:0 overruns:0 carrier:1
collisions:0 txqueuelen:1000
RX bytes:356289115 (339.7 MiB) TX bytes:18085137 (17.2 MiB)
Interrupt:11 Base address:0xa000 Memory:d0200000-d0200fff
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:7425 errors:0 dropped:0 overruns:0 frame:0
TX packets:7425 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:610482 (596.1 KiB) TX bytes:610482 (596.1 KiB)
tap0 Link encap:Ethernet HWaddr 6E:EA:39:83:8C:D4
inet6 addr: fe80::6cea:39ff:fe83:8cd4/64 Scope:Link
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:81 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:0 (0.0 b) TX bytes:10808 (10.5 KiB)
tap1 Link encap:Ethernet HWaddr B2:03:C1:BE:1E:4E
inet6 addr: fe80::b003:c1ff:febe:1e4e/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:106 errors:0 dropped:0 overruns:0 frame:0
TX packets:46 errors:0 dropped:6 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:12180 (11.8 KiB) TX bytes:4978 (4.8 KiB)
$ sudo iptables -t nat -L
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
MASQUERADE 0 -- anywhere anywhere
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Now start the QEMU virtual machines:
$ vdeqemu -net vde,vlan=0 -net nic,vlan=0,macaddr=52:54:00:00:EE:01 -m 256 -hda ~/qemu/netbsd.img 1>~/logs/qemu-logs/netbsd.log 2>~/logs/qemu-logs/netbsd.error &
In VirtualBox attach the network interface of all virtual machines to “Host Interface” and then select tap1 as the interface name.