The same reason we try to answer the original trolley problem in earnest: It forces us to confront tough moral trade-offs and clarify our ethical beliefs. Answering a trolley problem in earnest helps us learn about ourselves and our world on a philosophical level.
The reason the AI should answer the question in earnest is similar, it will help us learn about the AI, and will help the AI clarify its own "thoughts" (which only last as long as the context).
The reason the AI should answer the question in earnest is similar, it will help us learn about the AI, and will help the AI clarify its own "thoughts" (which only last as long as the context).