Maybe a stupid question: Is this the standard measurement of effectiveness? I.e. you give the control group a dummy vaccine, the test group the real vaccine, and you just wait for them to be infected? Why not test for antibody development?
Exactly. It's a good sign if people develop antibodies, but that does not ensure protection. As I understand it, it is difficult to predict, in advance of actual testing, what immune response will actually be protective.
The immune system is complex and has multiple mechanisms to recognize and attack pathogens; besides antibodies I've seen mention of multiple types of T-cells and also there's apparently a dimension of mucosal immunity vs. systemic immunity.
Antibodies are easy to test for (the chemistry required can be put into mass-produced home test kits), while it's more difficult to test for other responses (a recent long-term study of covid-19 T-cell response was with only ~100 subjects).
A hypothetical vaccine that produced a meh antibody response but mobilized the T-cell or mucosal systems could be effective at preventing the disease but not look like much on an antibody screen.