I've been trying to generate my own maps using Voronoi diagrams as well. I was using Lloyd's algorithm [0] to make strangely shaped regions "fit" better, but I like the insight of generating larger regions to define islands, and then smaller regions on top to define terrain.

One of the things I like about algorithms like this is the peculiarities created by the algorithm, and trying to remove that seems to take some of the interesting novelty away.

- [0] https://en.m.wikipedia.org/wiki/Lloyd%27s_algorithm