I find it depends heavily on both the nature of the problem I'm working on and my mental state, and also the nature of what I switch to.
If I haven't slept all night, context switches are total killers.
If the problem requires me to keep in mind a ton of factors and dependencies while I develop (design or code), context switches can be quite bad.
If the distraction require me to drag up a ton of context from a couple of weeks ago, then it can be quite bad.
If I'm well rested and someone comes with a technical question that's hard but limited in scope, it's mostly fine and I can get back to it within a minute if not less.
If I haven't slept all night, context switches are total killers.
If the problem requires me to keep in mind a ton of factors and dependencies while I develop (design or code), context switches can be quite bad.
If the distraction require me to drag up a ton of context from a couple of weeks ago, then it can be quite bad.
If I'm well rested and someone comes with a technical question that's hard but limited in scope, it's mostly fine and I can get back to it within a minute if not less.