If you want random access to a map you should write your own implementation. This is a bit like complaining that random access to a linked list is too slow.

Choose appropriate data structures for the task at hand, no one data structure will fit every use.