Archive for July, 2011
Disclaimer: These thoughts are my personal opinions. Since I work on vFabric these days and not vSphere I’m not very familiar with the specific reasons behind any choices made with respect to the new licensing scheme.
I missed the big launch yesterday because I’m hanging out in India with the GemFire and SQLFire teams, but I got up this morning and saw the discussion is completely dominated by talk of the licensing changes. That’s too bad because vSphere 5 has some pretty cool new features that aren’t getting the attention they deserve, for instance Storage DRS and SRM failback.
The new scheme is quite different from anything we’ve seen and any change is naturally met with skepticism. I think people who are worried about the new scheme should consider what would have happened if VMware maintained the status quo. vSphere 4 was not licensed per host, rather it’s “per CPU socket up to X cores” where X depends on the license level, 6 for Enterprise and 12 for Enterprise Plus if I recall correctly.
The big push in hardware is to increase core counts as fast as possible. Predictions are core counts will follow a Moore’s law-like trajectory of doubling about every 18 months. I know there are 10 core CPUs on the market today and much more dense CPUs are in the pipeline. So consider that in a few years you’re going to have computers with 32 or 48 or maybe even 64 cores per CPU. If you’ve got a computer with 4 CPU sockets each with 64 cores you’d need 44 licenses of Enterprise or 24 licenses of Enterprise Plus to have the host in compliance. (The formula is ceiling(# cores / max cores at license level) * number of sockets. Get that??)
Now consider that each host in your inventory is going to have different hardware profiles and require the same kinds of calculations. What a mess!
The fundamental thing is that to deal with this mess VMware had to change to some sort of pooling mechanism. Other vendors are going to have to do similar things. These sorts of host-by-host calculations on discrete boundaries are just unsustainable as IT environments get more and more complex. Pooling is the only answer. The only other option that is tenable is per-host licensing, but you won’t see that from anyone because it can’t monetize big hosts and small hosts differently.
VMware could have chosen to do core-based pooling or memory-based pooling. As it happens they chose memory-based, I don’t know the specifics of why, these hardware assets seem to be on the same growth trajectory. But my understanding is the amounts of memory they chose (24/32/48 depending on license level) were chosen so that most common current hardware configurations would not be affected.
So if you’re worried about vSphere’s new licensing scheme, consider the mess you would be in without this change, and also bear in mind that you were going to need to buy a lot more vSphere 4 licenses to enable the next wave of hardware anyway, because of exploding core densities. With pooling the operational aspects get a whole lot easier.
Update: In case you missed it, the vRAM licensing scheme was changed with higher ratios and some other tweaks. Read about it on the official blog.
A couple of products I work on, SQLFire is one of them, have WAN features for linking different datacenters together with asynchronous replication. Whenever I tell people about it the first thing they ask is what happens if the same data is updated in both datacenters at the same time. I don’t really know all that well, so I wanted to tool that would help me learn more about it, I needed a WAN emulator.
I looked at a couple of WAN simulators, specifically WANem and Dummy Cloud. WANem gave me a pretty bad impression, it’s text UI is extremely buggy and I didn’t even notice that it is supposed to be driven by its Web UI. Maybe the rest of the product is better but I’m looking for something as simple as possible. Dummy Cloud looks pretty good, and is loaded with features. Too many features actually, for what I want, and the free version doesn’t let you do really high latencies. So I decided to roll my own. I packaged the results as the WANatronic 10001 virtual appliance you can run on VMware Workstation, and probably ESX and Fusion as well.
- Can work with nothing but a laptop (no actual network required).
- As small and lightweight as possible.
- Simple, as close to zero configuration as possible.
- Really cool name.
I think things turned out fairly well. Namely:
- Traffic sent directly to WANatronic 10001 is sent right back to whatever host it originated from. If you ping WANatronic 10001 you’re actually pinging yourself. Same with SSH, etc.
- Packaged as a VM needing 32MB and a download size of about 170MB, which unfortunately is small as far as virtual appliances go.
- There are 4 things you can configure. If you’re lazy and don’t configure anything, WANatronic 10001 will still work.
- That’s pronounced WAN-a-tron-ic ten-thousand-one in case you were wondering. Just try to forget that name. See? You can’t do it.
Using WANatronic 10001
Possibly the most interesting thing, and the thing I think I’ll use it most for, is simulating slow links on a single host. In this case there is nothing you need to do on your computer. In the screenshot above my WANatronic is at 10.24.0.13. Any traffic I send to that IP is sent right back to me. So if I have two processes on my host that I want talking slowly, let’s say one is on port 8000 and one is on port 9000, all I do is tell the first application that its peer is on 10.24.0.13:9000 and tell the second one its peer is on 10.24.0.13:8000. Done.
WANatronic can also function as a router-on-a-stick, all you need to do is route traffic through WANatronic. For example Google’s free DNS is located at 220.127.116.11. If you want to simulate being on a slow network between here and there run:
ip route add 18.104.22.168 via 10.24.0.13
Your IP address will be different from mine, check the console to see what it is. You may need to adjust that command based on your OS. With that set up, ping 22.214.171.124 to see the effect of latency and packet loss.
Last but not least, configuration is via the console since there is no way to remotely connect to WANatronic. Changes are automatically saved and applied.
Technical Mumbo Jumbo
WANatronic uses Ubuntu 8.04 JEOS as a base. I used this older distro to cut down on the size of the virtual appliance, which unfortunately is still huge. For the most part WANatronic is a very basic use of the iptables and tc utilities. The WANatronic source code is hosted on GitHub.
There is one exception that required a little bit of science. I really wanted any traffic sent to WANatronic to be NATted directly back to me. Without this I would need to use multiple network interfaces which I wanted to avoid. As far as I can tell there’s no way to do that with iptables, so I made a custom version of the iptables NETMAP target that would replace the packet’s destination address with its source address in PREROUTING. After that the packet is NATted so it appears to originate from WANatronic. This is the magic that lets you talk to yourself on what appears to be a very crappy link. Perfect for a demo that needs to reside completely on a single laptop.
Stuff It Doesn’t Do
Plenty of things really but things that come to my mind include:
- Static IPs (seems useful)
- Routing stuff like OSPF (not going to happen)
- Remote management (not sure it’s worth it)
- IPv6 (does anybody actually use that)
- Really fancy stuff like packet reordering (not likely)
If you’ve got comments or would like to see something let me know.
In case you scrolled to the bottom for a download link here it is again: WANatronic 10001.