Middle click paste should "just work" and be transparent to terminal apps. It's not encouraging that vim is willing and able to break this in its default settings.
I could be misremembering but I believe that middle-click paste just makes the terminal feed the contents of the X selection to stdin of the running process. I could be wrong (because there's so much arcane knowledge in terminal emulators that I won't even pretend to have scratched the surface) but I'm not sure if Vi(m) can even tell where the input comes from. Well, maybe it could by using the speed at which the characters come in I suppose, although even that might end up being unreliable over remote connections and the like.
In my humble opinion expecting good X integration for a terminal application is bound to result in frustration. I use vim in the terminal but I also never use the mouse and I configure Vim not to interact with the X selection unless I specifically instruct it to using "*. If you want better integration with the graphical environment why not just use gvim? That's what it's for.
> middle-click paste just makes the terminal feed the contents of the X selection to stdin of the running process.
Recent versions of vim hijack this by default (`set mouse=a`). This makes the middle click paste the default vim register instead of the X selection. To paste from X, you must shift+click.
Personally, I can't understand why anyone would want this, but anyway. The best part is that you can't easily disable this globally, i.e. by putting `set mouse=` in /etc/vim/vimrc. Because if a given user does not have a ~/.vimrc of their own, then defaults.vim gets loaded after /etc/vim, overriding anything set there.