Emacs and Vi are not IDEs. They operate on strings, not ASTs. Intellij Idea is an IDE for Java- it allows you to eg change the name of a method and have all call sites be updated. A text editor forces you to do this yourself with a string based find/replace. Having said that, text editors are easier to extend.
Sounds like you haven't used emacs in a few years (or longer). There are tools like CEDET that allow building language modes with AST-level understanding of the source code. I don't know about the vim world in general, but I use omnisharp in emacs to provide C# refactoring tools like you describe, and that was originally built for vim.
vi: runs everywhere, installed by default on most linux distributions.
emacs: incredibly customizable, you can re-write the way your editor works. Some features that I find important: editing files remotely over SSH, running a shell inside the editor (so the same editing commands work in the shell), easy to write custom file finding functions for searching different parts of a codebase, git integration (a semi-GUI interface to git), and if another editor has a feature, someone has probably created a package that adds that functionality to emacs. Also has tetris and pong.
I used Visual Studio, Zend Studio and Komodo IDE in the past. Some (it's not exhaustive list) of the things Emacs can do and mentioned IDEs can't:
Receiving and sending emails. Being an IRC/Jabber/others client. Displaying images and pdfs directly. Bulk editing of many lines, even from different files, at once. Drawing ASCII-art diagrams. Playing tetris. Being an amazingly powerful "calculator" (http://nullprogram.com/blog/2009/06/23/). Transparently editing file on remote machines, also executing commands on remote machines transparently. Formatting ASCII-art tables. Editing outlines/todos with support for semantic movement inside it (Org-Mode). Working in a terminal. Working in a background (--daemon mode) so that new instances just connect to the backend, reducing startup time to almost zero. Dumping a whole Emacs into single binary - all the memory Emacs uses is written to disk and next time you run it it starts very quickly, because you don't need to compile,, load and run any libraries; it's like suspend/hibernate for OSes. Having a decent mode for almost every file format out there. Displaying info about currently running OS processes, like top (proced). Multiple cursors. Rectangular selection and manipulation of rects. Registers for storing pieces of text, locations in files and others. Easy running of external commands, with or without capturing their output. Find&replace with an Elisp function as replacement (quick and effective way to transforms more complex files). Being a terminal emulator (you can run mc inside Emacs, although it's not the best possible experience). Being a shell (Eshell). Easy integration with external REPLs (like IPython, with colors and all). Displaying and working with IPython Notebooks. Splitting windows horizontally and vertically as many times as you want. If you made a bit too many windows (these are Emacs windows, they live inside one OS level window) you can enable tiling window manager for them. Easy integration with external linters for many languages. Displaying man and info pages. Having easily reachable docstrings for all the functions implemented in the editor. Hierarchical keymaps. Semantic editing of sexp (any lisp) code.
And probably much more - I still learn about some new, crazy useful feature at least once a week.
IDEs win in the autocompletion department and project navigation, but that depends on a language being used. For example editing Python with Jedi and Jedi.el (and Rope) in Emacs feels like working with full-featured IDE, with very accurate, context sensitive autocompletion, refactoring support, visual debugger and others.
What can emacs and vi do that a regular IDE can't ?