This is a bit of a different post based on some configuration I did just recently to enable the bridging of a Virtual Switch between 2 ESX hosts.

There is an VMWare option for this called a “VMware vSphere Distributed Switch” however this requires one of the higher end licencing packages so isn’t available on the free or basic packages, but there are many different uses you might have for this,  from simply creating a temporary bridge while you migrate servers to a remote host, or in my case, creating a bridge network across 2 hosts that use a RouterOS vm as the gateway/firewall for the servers.

ESX Setup

We start off by creating an internal switch group in vSphere (or vCenter depending on what you have access to).

1. Head to the host you want to add the switch to
2. Head to the “Configuration” tab
3. Click “Add Networking…”
4.  Select “Virtual Machine” as the type
5. Select the topmost option “Create a virtual switch” as you don’t want any physical interfaces connected*
6.  Choose a network label and leave VLAN ID as is, “None (0)”
7.  You should now have an interface that looks like this.

Redo this config on the second ESX host machine (so both have a Virtual Switch with the same label)

You now need a RouterOS VM installed on each host, with 2 ethernet interfaces:
ether1 = WAN connection, or private connection routable to the other ESX host
ether2 = Virtual Switch connection, this connects to the switch you created just above.

For our example, lets say that the RouterOS vm on ESX host1 has the wan ip 198.51.100.1/24 and the RouterOS vm on ESX host2 has the wan IP 198.51.100.2/24 while obviously these are on the same range for the simulation, this would work across the internet if both routers had public addresses and internet access.

The internal network range is largely irrelevant to the RouterOS vm’s themselves because although they are connected to it they will not participate in it as clients (unless you also want to use one of them as a gateway).  TL;DR: You can use whatever LAN range you want on the VM’s that are to be bridged.

RouterOS config

Onto the juicy stuff.

Here’s config for VM1 (RouterOS device) and VM2 (RouterOS device)

VM1:

/interface bridge
add admin-mac=00:00:00:00:00:00 ageing-time=5m arp=enabled auto-mac=yes comment="Distributed Switch - RSTP recommended" disabled=no forward-delay=15s l2mtu=65535 max-message-age=20s mtu=1500 name=esx-bridge priority=0x8000 protocol-mode=rstp transmit-hold-count=6
/interface ethernet
set 0 arp=enabled auto-negotiation=yes cable-settings=default comment="WAN" disable-running-check=yes disabled=no full-duplex=yes mac-address=00:50:56:B5:00:B7 mtu=1500 name=ether1 speed=100Mbps
set 1 arp=enabled auto-negotiation=yes cable-settings=default comment="ESX LAN" disable-running-check=yes disabled=no full-duplex=yes mac-address=00:50:56:B5:00:B8 mtu=1500 name=ether2 speed=100Mbps
/interface eoip
add arp=enabled comment="From VM2" disabled=no l2mtu=65535 local-address=0.0.0.0 mac-address=FE:A8:3F:E2:C4:62 mtu=1500 name="EOIP-TO-VM2" remote-address=198.51.100.2 tunnel-id=1

/interface bridge port
add bridge=esx-bridge disabled=no edge=auto external-fdb=auto horizon=none interface=ether2 path-cost=10 point-to-point=auto priority=0x80
add bridge=esx-bridge disabled=no edge=auto external-fdb=auto horizon=none interface="EOIP-TO-VM2" path-cost=10 point-to-point=auto priority=0x80

/ip address
add address=198.51.100.1/24 disabled=no interface=ether1 network=192.51.100.0

#Note this is not required for the example but is required if you're bridging services over multiple hops
#In the event you were using VPLS or Vlan bridging you would rely on interim devices to pass your taggged traffic between the 2 routers

/ip route
add disabled=no distance=1 dst-address=0.0.0.0/0 gateway=192.51.100.254 scope=30 target-scope=10

/system identity
set name="VM1 Mikrotik Bridge"

VM2:

/interface bridge
add admin-mac=00:00:00:00:00:00 ageing-time=5m arp=enabled auto-mac=yes comment="Distributed Switch - RSTP recommended" disabled=no forward-delay=15s l2mtu=65535 max-message-age=20s mtu=1500 name=esx-bridge priority=0x8000 protocol-mode=rstp transmit-hold-count=6
/interface ethernet
set 0 arp=enabled auto-negotiation=yes cable-settings=default comment="WAN" disable-running-check=yes disabled=no full-duplex=yes mac-address=00:50:56:B5:00:B7 mtu=1500 name=ether1 speed=100Mbps
set 1 arp=enabled auto-negotiation=yes cable-settings=default comment="ESX LAN" disable-running-check=yes disabled=no full-duplex=yes mac-address=00:50:56:B5:00:B8 mtu=1500 name=ether2 speed=100Mbps
/interface eoip
add arp=enabled comment="From VM2" disabled=no l2mtu=65535 local-address=0.0.0.0 mac-address=FE:A8:3F:E2:C4:62 mtu=1500 name="EOIP-TO-VM1" remote-address=198.51.100.1 tunnel-id=1

/interface bridge port
add bridge=esx-bridge disabled=no edge=auto external-fdb=auto horizon=none interface=ether2 path-cost=10 point-to-point=auto priority=0x80
add bridge=esx-bridge disabled=no edge=auto external-fdb=auto horizon=none interface="EOIP-TO-VM1" path-cost=10 point-to-point=auto priority=0x80

/ip address
add address=198.51.100.2/24 disabled=no interface=ether1 network=192.51.100.0

#Note this is not required for the example but is required if you're bridging services over multiple hops
#In the event you were using VPLS or Vlan bridging you would rely on interim devices to pass your taggged traffic between the 2 routers

/ip route
add disabled=no distance=1 dst-address=0.0.0.0/0 gateway=192.51.100.254 scope=30 target-scope=10

/system identity
set name="VM2 Mikrotik Bridge"

Alternative builds

This gives the most complex solution (bridging over EoIP) however replacing the EoIP interfaces with a VPLS tunnel or even just a simple VLAN would do the same thing.
Why would you want to run the vlan via RouterOS rather than just attach directly to the ESX interface you ask? Firewall restrictions, netflow tracking of traffic passing through the router, the ability to apply transparent queuing, the list goes on.

Enabling vMotion

Something you’ll notice if you’re attempting to use vMotion to shift live vm’s using this new link, is that vCenter will stop you; complaining that the current interface is connected to a virtual LAN and must be disconnected before the move can be completed.

A simple kludge around this (because we already know the segments are connected) is to either:

a) find an unused physical interface you can make part of the switch on each host
b) create a new vlan on a used  physical interface (if none others are available) but on an unused vlan ID

This is just to be used as a loopback port to trick vCenter into believe these 2 are switch groups are directly connected and allowing live vMotion to take place.

Words of warning

I would recommend attempting any of this configuration on a test bench first so you can confirm you understand the process involved and the associated performance hit you will see on networking performance due to the interfaces being in promiscuous mode.  This tutorial and code are provided as-is and I take no responsibility for any disasters you cause.

Anyway, I hope this is useful for someone. Feel free to leave suggestions or comments below.

*There are a few cases in where you would want a physical interface connected, purely so vMotion believes it can migrate hosts (because otherwise it doesn’t think the 2 virtual switches are connected).

Random fact of the day: Did you know IANA allocated specific ranges for use in documentation? Remember this next time you go to design an example network using 1.1.1.1/24 🙂