After players choose a new empty server, redirect people to that server for some time even if the player count drops below
Sovietshark mentioned a situation where it could happen that players choose a server, but not all of them stay playing on it or connect to it, which might leave very few players playing on that server stranded, while the lobby isnt redirecting to it, because the player amount isnt enough.