Don't think you're irreplaceable. You can be replaced, and if he needs help you can do some consulting. Everyone justifies staying in a job by claiming that they are being a martyr - just watch Clerks.

I can't count the number of times I've heard an employee (sometimes myself) looking for a change of scene say "I'm doing this place a real favour, staying on".