Setting up Ubuntu 7.04 (Feisty) as vmware host with a wireless card

Phew, after fighting with this setup for 3 days, I have managed to bring up Bridged networking to a wireless interface in a Ubuntu Feisty server.

I am listing a short summary of my wanderings here so other users who are stuck at various places in the process can pick it up from there. There are a whole bunch of solutions out there which unfortunately trample over each other.

I also want to thank all the users who contributed on this topic all over the internet. Awesome!!

For the final solution mix, scroll down to the end of the post.

-- My setup

I have an old HP DL-140 Proliant server sitting in my garage. I wanted to use it as a vmware host to run some development oses. Since the server is situated in the garage with no ethernet wiring, I decided to use a cheap D-Link DWLG510 802.11G card.

I have a Hardware firewall/router that DHCP's for my entire Home LAN and my wireless access point is set up in the bridge mode to connect my wireless network to the ethernet network. This allows me to get a single network topology, allowing easy access to machines and their data.

I wanted to set up the Ubuntu server so that the Guest VM's could bridge to the host's wireless interface and use DHCP to acquire dynamic IP address from the hardware router.

[VMware Guest] <--> (vmnet 0 bridge) <--> [VMware host/Ubuntu] <--> (Wireless NIC ath0) <--> [Wireless AP] <--> (ethernet) <--> [HW Router/DHCP] <--> WAN

I wanted to be able to run multiple web servers on these Guest VMs and reach any server using the IP address (or a dns name) directly instead of using natted ports.

So over the weekend I installed Ubuntu 6.10 server which I later updated to 7.04 (linux kernel 2.6.20) via apt-get.

%> lspci
00:00.0 Host bridge: Broadcom GCNB-LE Host Bridge (rev 32)
00:00.1 Host bridge: Broadcom GCNB-LE Host Bridge
00:03.0 VGA compatible controller: ATI Technologies Inc Rage XL (rev 27)
00:0f.0 ISA bridge: Broadcom CSB6 South Bridge (rev a0)
00:0f.1 IDE interface: Broadcom CSB6 RAID/IDE Controller (rev a0)
00:0f.2 USB Controller: Broadcom CSB6 OHCI USB Controller (rev 05)
00:0f.3 Host bridge: Broadcom GCLE-2 Host Bridge
00:10.0 Host bridge: Broadcom CIOB-E I/O Bridge with Gigabit Ethernet (rev 12)
00:10.2 Host bridge: Broadcom CIOB-E I/O Bridge with Gigabit Ethernet (rev 12)
01:06.0 Ethernet controller: Atheros Communications, Inc. AR5005G 802.11abg NIC (rev 01)
02:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5704 Gigabit Ethernet (rev 02)
02:00.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5704 Gigabit Ethernet (rev 02)

Setting this up with Ubuntu is tricky. I tried a few of the solutions and ended up updating the 6.10 to 7.04 in the process. I finally found this script which worked great and I was able to use my wireless interface as my primary network interface. If you need further instructions with setting this up, drop me a comment and I will try to help.
Note that the madwifi code needs to be patched eventually for vmware bridging to wireless NICs to work. See below.

Next up was the VMware server. I installed a couple of Ubuntu packages listed here as dependencies for the server.

I then tried to run the vmware-install.pl included in VMware-server-1.0.3-44356.tar.gz to install the VMware host. The script failed while trying to build vmmon.

%> cd vmware-server-distrib 
%> sudo vmware-install.pl
...
Using 2.6.x kernel build system.
make: Entering directory `/tmp/vmware-config0/vmmon-only'
make -C /lib/modules/2.6.20-16-386/build/include/.. SUBDIRS=$PWD SRCROOT=$PWD/. modules
make[1]: Entering directory `/usr/src/linux-headers-2.6.20-16-386'
  CC [M]  /tmp/vmware-config0/vmmon-only/linux/driver.o
In file included from /tmp/vmware-config0/vmmon-only/linux/driver.c:80:
/tmp/vmware-config0/vmmon-only/./include/compat_kernel.h:21: error: expected declaration specifiers or â...â before âcompat_exitâ
/tmp/vmware-config0/vmmon-only/./include/compat_kernel.h:21: error: expected declaration specifiers or â...â before âexit_codeâ
/tmp/vmware-config0/vmmon-only/./include/compat_kernel.h:21: warning: type defaults to âintâ in declaration of â_syscall1â
make[2]: *** [/tmp/vmware-config0/vmmon-only/linux/driver.o] Error 1
make[1]: *** [_module_/tmp/vmware-config0/vmmon-only] Error 2
make[1]: Leaving directory `/usr/src/linux-headers-2.6.20-16-386'
make: *** [vmmon.ko] Error 2
make: Leaving directory `/tmp/vmware-config0/vmmon-only'
Unable to build the vmmon module.

I used vmware-any-any-update fix to resolve this issue.
Later however I found that vmware-any-any update causes problems with madwifi patch to allow vmware bridging to wireless interfaces on the host. See below how to fix the vmmon compile problem.

Note: if you get vmware server is not configured message when you run vmware, simply delete the /etc/vmware/not_configured file and rerun vmware.

I set up the networking to bridge vmnet0 to ath0 (my Ubuntu server's wireless NIC). I skipped Host only and NAT portions.

I happily bought up my CentOS development VM. But unfortunately the network didn't work. So it was back to googling.

I saw various posts on how it worked for some people by changing /etc/vmware/locations file (make sure VNET_0_INTERFACE is set to ath0), but my version of the file had it already set to that.

Then I came across posts where folks had patched there madwifi drivers. Though my setup wasn't reporting the problem, I tried it anyways, but no dice.
Eventually I found that this patch works, but the vmware-any-any update seems to break it in some way.

Some posts claimed that since wireless frames are different than ethernet frames, the bridging of Guest-eth0 to Host-ath0 is not possible. Soon after I read that VMware server supports bridging to wireless NICs, but it just didn't work for me.

Explored other approaches such as Host only networking, NAT etc to resolve the issue. Spent a day in reading up on iptables. But couldn't figure out how to get a single network topology using my setup. If any networking experts could comment, I would really appreciate it.

I jumped back to vmware bridge solutions and re-read the madwifi ticket. One user warned against installing vmware-any-any patches. So I completely cleaned up my vmware server install and performed the following steps:

Solution Mix:
Note: do this before installing vmware.

1) Apply the madwifi patch described by xgerligand@free.fr. make clean, make and make install and then reboot.

2) Now untar vmware-server-distrib and apply the patch described here by turn1200 to fix the vmmon problem that occurs when you run vmware-install.pl without using the vmware-any-any patch. Recording the commands here...

cd vmware-server-distrib/lib/modules/source/
tar -xvf vmmon.tar
vi vmmon-only/include/compat_kernel.h

change the line that says:
static inline _syscall1(int, compat_exit, int, exit_code);
to the following by removing two commas:
static inline _syscall1(int compat_exit, int exit_code);

sudo tar -cvf vmmon.tar vmmon-only/

3) Now run sudo vmware-install.pl and vmware installation should complete successfully.

Hopefully this will be of help to others.