I really thought this was what the plan would be when Alphabet was formed.

Google X/X Development LLC is the Google incubator for possible spin-offs as far as I can tell.

What would be the benefit of doing it spun-off vs. in-house when it's still owned by the same company and making the same amount of money?

Less red tape, more freedom to operate.

If done inside a conglomerate, it can be the best of both worlds. Access to Google tech and funding, but free from rigid corporate hierarchies, at least at the beginning.

Spinning it off limits the downside/risk to the parent company. Basket of options worth more than an option on a basket and all that.