I remember reading an article a while back about how professional photography has become a commodity. You can find and license any imaginable photograph, for pennies, in minutes. From the best photographers in the world. So from a business perspective, photography as a pure art has been devalued to almost nothing. Nobody pays you to take the very best possible photo.
So why do people still hire photographers? To put themselves or their product into a photo. Most photographers are no longer paid to take the very best photos, but to make their photos specific to what the particular client cares about. That's something that can't be commoditized.
I think that's where software is now as an industry. Very few of us these days get to innovate at a "pure art" level. Instead, we get paid to pull together a bespoke solution that's sensitive to a given business and its needs. It isn't novel at the macro-level, only at the micro-level.
This can be kind of depressing, but it's also a little comforting that those micro-level solutions are so hard to fully commoditize. As long as the real world is messy and varied, the leaves in the software tree will have to be too, and there will be work that needs doing.
I don't see how that's depressing at all. Another way to phrase what you're saying is "Developers get paid to write software that solves problems people actually need solved.
For many of us, writing something pure and generic and beautiful is much more gratifying. And it's depressing to think that we'll probably never write something like that that's good enough to be picked over the version that was developed at Google and then open-sourced.
That's not unique to software, or the modern world at all, though.
Most people have never created the best-in-the-world implementation of anything. The math just doesn't work that way. Only one can be the best. We need to stop turning such unobtainable things into goals.
That's not what I'm saying at all. Of course very few people make "the best" of anything. What I'm saying is that the internet, and in our case open-source software, have caused "the best" to be all that matters, because it's accessible to everyone. There's no longer a practical reason for most people to engineer most (generic) things themselves because the better solution isn't hidden behind a license, or proprietary to an organization, or inaccessible due to physical media. People used to buy things like linkers. Today, the state-of-the-art is nearly always at your fingertips, for free, in seconds.
I'm not saying this is a bad thing. Progress always makes certain endeavors obsolete over time. You don't see job postings for telegraph operators any more because it's no longer useful. But if a person enjoys one of those endeavors for its own sake, it's bittersweet to see it relegated to a hobby.
I think software is unique in that there isn't a neat "research" vs. "engineering" side, at least in so far as a lot of boundary pushing in the field has happened by engineers actually building software.
I think there will always be a frontier where that is the case in software—there will always be hackers building open source projects that push some boundary, fostering their own community, and potentially changing broader paradigms—but that more and more, we're seeing a neater boundary between research (Universities, labs, projects that are essentially R&D teams at big companies) and engineering.
I mean, nobody's stopping you from writing your perfect, beautiful Haskell implementation of Dijkstra's Algorithm.
I guess it's simply a different approach, but I'd rather not waste time re-implementing the wheel when I could be making software that will help somebody fulfill a concrete need.
My point is that the ideal scenario, where you write something generic that also meets people's real needs (better than what's already out there), has become a very rare thing.
So why do people still hire photographers? To put themselves or their product into a photo. Most photographers are no longer paid to take the very best photos, but to make their photos specific to what the particular client cares about. That's something that can't be commoditized.
I think that's where software is now as an industry. Very few of us these days get to innovate at a "pure art" level. Instead, we get paid to pull together a bespoke solution that's sensitive to a given business and its needs. It isn't novel at the macro-level, only at the micro-level.
This can be kind of depressing, but it's also a little comforting that those micro-level solutions are so hard to fully commoditize. As long as the real world is messy and varied, the leaves in the software tree will have to be too, and there will be work that needs doing.