I doubt it's just this, since on my laptop just brew list from warm cache with 102 packages takes about a second, and twice as long if files must be read from SSD. That's a lot of reading and computation for list of 102 packages.