It depends on your goals and where your interests align. The domain of philosophy is fundamental and all encompassing, and you have philosophical opinions that you can choose to examine or not. From a career perspective you are almost certainly right that time is better spent extending your skills in the empirical domain but others have a different temperament and have use for philosophy. And for some of those people, their contributions are long lasting and impactful.
See this link titled “The Purpose of Philosophy”: http://assets.press.princeton.edu/chapters/s10113.pdf