It will depend on the capabilities of the switch you buy - managed switches offer a much greater diversity of "features" than unmanaged ones (which are generally all much of a muchness feature wise) so you have to choose your managed switch according to what you want it to do and thereby what features you need it to have.
However, I don't think what you want will be easy or foolproof, others may have better ideas. This is just a flavour of some of the complexities I can envisage:
One idea might be to use a switch that implements VLAN's which often avail to ability to assign which VLAN a particular (physical) port is bound to. So one would set up two VLAN's (let's say) VLAN 100 & VLAN 200. Bind a port from network "A" into VLAN 100 and a port for network "B" into VLAN 200, then similarly bind a port connected to router 1 into VLAN 100 and router 2 into VLAN 200. Then to "swap" you reassign which VLAN the switch ports connected to the routers belong too. The hassle is, you can easily "cut off" one network or another if you are not careful about the order in which you make the changes and (if you are using SOHO type routers with DHCP servers in them) IP addressing for clients will get screwed when you swap. And this is assuming you can reliably remotely access the management interface of your managed switch. It's really "messy" and not something you would want to be doing regularly (I wouldn't want to do it remotely at all - I'd prefer to be on site, just in case and then I'd have to go reboot all the client to refresh the DHCP allocated IP addresses.)
If I were doing this I'd look to do something at layer 3 (IP level) with a "proper" load balancer or (proper not SOHO) router/firewall that avails such facilities, put each client network and router on separate subnets (each client net with their own DHCP Server/Relay agent) and implement traffic management at the aggregation point using using routes, or some load balancing agrorithm, etc.
Even just physically repatching on a PP as you suggest would screw up any DHCP allocated IP addressing every time you swapped. It could get ugly really fast.