As part of my Master’s program, I’m attending a class called Operating Systems for Parallel and Distributed Architectures. As a requirement of the class, we have been asked to set up a virtual Rocks Cluster on our laptop. The class is using Virtual Box as the virtualization platform of choice, but since I’ve spent so much time working with VMware in my day job, and because I already had a VMware Fusion license, I chose to use this instead. The process was fairly straightforward, but did require a bit of work configuring a virtual network adapter for the cluster to use as its private network.
For those of you not aware, a Rocks cluster relies on a private network for its inter-machine communications. The head node also provides the DHCP and gateway services for the individual compute nodes. This means the typical VMware “Private to My Mac” (or “Host Only” for VMware Workstation users) network isn’t the right answer. By default that adapter has a DHCP server that provides addresses to all VMs on the network.
Instead, we need to create a new network that can be used to share connections. The screenshot below shows the configuration that I ended up using. (Note: I ended up repurposing a separate NAT network I had previously configured, hence the 192.168.116.x subnet IP listed). The key items are to disable the DHCP server and uncheck the “Allow virtual machines on this network to connect to external networks (using NAT)”. Now, to be fair, I did leave the NAT setting enabled and it worked as well, but isolating the network is still likely a good idea.
Head Node
Once you have that configured, you can build the VMs. For the head node, you’ll see two virtual network adapters: “Network Adapter” and “Network Adapter 2”. These correspond to eth0 and eth1 and for a Rocks cluster, eth0 needs to be the private network. The other network should be connected to the Internet in some form or fashion. In the rest of this HOWTO, I will refer to the networks by their Linux name, either eth0 or eth1. Just know that eth0 refers to “Network Adapter”.
First, let’s review the configuration for eth0. For it’s connection, I used the custom vmnet2 adapter/network. (Again, note the subnet in the pictures is set, but since DHCP has been disabled, it’s irrelevant).
Next, let’s look at the configuration of eth1. For it’s connection, I used the “Share with my Mac” network. This is a habit I got into when I worked at IBM and used a VM for development. Initially I used Bridged Networking, but over time found that if I need to work in a disconnected state (say at a customer site or somewhere I don’t have an active network connection), I would lose the ability to communicate from the host PC/Mac to the VM as the state of connection. That caused me a bit of frustration over the years, so I ended up relying on the NAT network. It allowed me to continue to develop and communicate with the VM itself from my host machine and the VM Internet and local network access as needed. The only real issue with this came when I wanted a separate system to communicate with the VM (remote debugger, mobile device, etc). In Workstation, I would typically deal with that via port forwarding as long as the protocol supported it, otherwise I’d open up an additional interface set to bridged. Which you use is up to you and how often you will be without network access.
One other thing I did was provide eth1 with a hard coded IP address that matched the vmnet8 (NAT) network. This way the head node will always be at the same address (I’ve noted that VMware is aggressive about changing your IP address).
Compute Node
With the head node installed and configured, let’s look at the compute node(s). They are configured similarly with the exception that there is only one network adapter. This one is connected to your newly created vmnet (vmnet2 in my case). That adapter’s configuration should look the same as eth0 on the head node. (See the picture above).
Any other requirements should match the prerequisites for Rocks itself, including memory and disk space.
Other Notes
One other thing to note is that your compute nodes need to be able to boot using PXE. By default, VMware provides a boot off removable media, the internal HDD and then if nothing else, fall back to PXE. For Rocks, this needs to be reversed. VMware offers a neat option in newer versions that allows you to boot straight to the firmware as opposed to hitting F8 or whatever the key is. Under the Virtual Machine menu, you can select the item shown below:
Once the VM boots, it’ll take you to the firmware screen where you can change the boot order to match below:
One other thing I saw was that the initial boot for a new node seemed to take two tries. First I’d boot the VM and let it time out and then just watch. Eventually the insert-ethers command would show the new machine as available. I’d then reboot the new compute node and it would finish bootstrapping and be fine. It could have been a WAN issue (at the time my Internet connectivity was hosed), but I can’t say for certain.
