That is certainly a legitimate concern. Perhaps disable the internet connection to the client that is operating the admin panel. Pretty impractical and would not work well for malicious JavaScript in memory.

And REALLY screw up my virtual network switch for VMs... man, recently changed routers, and getting it to see the new router, so I could change the network to match my old router was painful enough... Wouldn't want to have to do that every time I make a configuration change.