Vanished

In the episode Many Happy Returns in the show The Prisoner our hero wakens to a Village without people. Where have they vanished to?