I think there's no reason there should be a single person on both sides of the equation, never mind arbitrary choices of the harm scenario. In a contrived utilitarian scenario, you could save 1,000 people and send one to prison, or vice versa.