Not to pick on your quick example, but actually testing it, these two terms [1] [2] have nearly identical results. Top result in both cases is history.com, followed by wikipedia, and the next 4-6 results are the same but in slightly different order.
I think this is actually an example of the benefit of their AI. Despite the big difference in the "style" of phrasing (simple english vs more formally naming the subject noun), both seem to map to a very similar representation in their embedding space. I've run into frustrations with this myself, but for basic questions like this it seems like the search works Pretty Good.
Consider that "Why did the Roman Empire fall?" consists of 1 word that describes the type of question being asked ("why"), 2 useless junk words ("did the"), and 3 "key words" ("Roman Empire Fall"), of which 2 should really be treated as a single word referring to a single concept/entity ("Roman Empire", for which "Rome" is a synonym in some cases).
Humans instinctively know this, so we are able to construct queries like "why rome fall".
But that "why rome fall" query, which we think of as a purely mechanical keyword search, already requires quite a bit of sophisticated processing in the search engine. The system has to recognize that "fall" is synonymous for "collapse" or "wane in power" and not synonymous for "autumn". It also has to recognize that "rome" means "the (Western) Roman Empire" and not the modern city of Rome in Italy or "the Holy Roman Empire" or the city of Rome, NY, USA. It furthermore needs to interpret "why" in such a way that it emphasizes results with "reasons" or "explanations", rather than something like a "timeline" or "summary".
Personally I find it really weird that Google is interested in pushing users more to interact with its digital librarian / AI assistant, instead of continuing to improve keyword search.
I have a few guesses as to why they are going this way:
1. It makes the user interface simpler from an engineering perspective (fewer user-facing buttons and options to implement and test).
2. There is strategic benefit to making search more of a black box. Maybe they are specifically trying to "educate" users to expect and be comfortable with such black boxes. Maybe the plan is to get people so accustomed to "AI assistant" search that they see keyword search as outdated, and thereby secure a competitive advantage for the next several years over other search engines, by having the biggest and best AI models.
3. They are trying to increase the amount of rich "natural language" user search inputs in their data. Making keyword search worse will encourage people to use queries that more closely resemble natural language. I assume that this has strategic benefit related to Guess 2 above.
I think this is actually an example of the benefit of their AI. Despite the big difference in the "style" of phrasing (simple english vs more formally naming the subject noun), both seem to map to a very similar representation in their embedding space. I've run into frustrations with this myself, but for basic questions like this it seems like the search works Pretty Good.
[1] https://www.google.com/search?q=why+rome+fall&oq=why+rome+fa... [2] https://www.google.com/search?q=why+did+the+roman+empire+fal...