I'd definitely second the ray tracer recommendation, starting from a 2D canvas and building a ray tracer to render a shaded sphere was really an "ah-ha" moment, as the math is quite approachable and you can get a cool result with not much more than some trigonometry.