Even a custom OL3 build which only has the canvas renderer, a WMS layer for background and a Vector layer for POIs does not get smaller than 200k. I think leaflet is half the size.

And I find this to be even more astonishing as OL3 uses Googles Closure compiler who's very advantage should have been better JS minification (not only minimizing/uglifying but actually removing JS parts that are not needed).