openvswitch and libvirt: vnet port "russian roulette" on restart (solution)

Update: This issue has been resolved in libvirt 1.2.7 release, or commit. The below instructions are no longer required if your distribution has updated the package.

libvirt has openvswitch integration. When a virtual machine is started that is using openvswitch for the network port, a vnetX interface is created (where X is an incremental number, from 0) on start and destroyed on shutdown by libvirt. openvswitch's configuration is persistent, being that the vnetX interface created by libvirt is saved to a database and will be available on the following reboot.

As outlined in my bug report submitted in September 2013, this quickly breaks down if libvirtd is shutdown after openvswitch because libvirt can't delete the port it's created or the machine is restarted/shutdown incorrectly. If you have virtual machines that are on different VLANs, or interfaces you can quickly have them being assigned to the wrong virtual machine as libvirt doesn't error out if the interface already exists when it tries to create it (imagine swapping around LAN and WAN ports on a firewall.)

I solved this by adding creating an upstart job override on the Ubuntu LTS releases in /etc/init/openvswitch-switch.override:

post-start script
    ovs-vsctl show | grep 'Port \"vnet[0-9]*\"' | awk -F\" {'print $2'} | xargs -I {} ovs-vsctl del-port {} || :
end script

I've tested this issue and proven it's existence in OpenSuSE 12.3 (Dartmouth), Debian (stable) and Ubuntu 12.04/14.04 (LTS) distributions.