OP here - I agree with this. If you're at an Ivy the banks and consulting firms come to you, then it's just a matter of 1) studying the basic interview questions and being able to speak semi-intelligently about financial topics and 2) interviewing a ton until you find a firm with people similar to you and like your personality.
I left because I was depressed during the job and realized I wanted work with fewer hours that was more intellectually challenging. I saw people 2-3 years ahead of me in their careers leaving for startups and thought I'd skip the 2 years of misery and do that straight away.
I've worked in both. I'm of the opinion that early on in your career it really doesn't matter, you could do either. There are far more important things to think about (company & personal growth trajectory, coworkers & culture, learning opportunities, etc.) when choosing a company.