Yes, I'm not arguing that he's (edit: you're) wrong. Just that the decision seems to be causing people to choose other solutions, I suspect because managing one thing at least seems easier than two.
I think the threshold question will always be "Does software X make my life better?" and if it does not, you should ditch it if you can.
There is always a huge bias in reporting: People are eager to tell you why they started using your software, but they always forget the "exit-interview" when they drop it again.
The reason I hear most often for people dropping Varnish is that they have cleaned up the mess of legacy web-services, or at least transitioned it all to the New Fantastic Platform.
Other people drop Varnish for other versions of "this is now surplus to requirements" and I am totally fine with that: I dont want people to run Varnish if it doesn't make their life better.
Mostly anecdotal, since there's no real definitive way to tell.
There's two spaces I see.
- Caching within the webserver, as the caching there has improved over time. The caches in Nginx, Apache, LiteSpeed, etc, perform much better than they did in the past.
- Caching at a load balancer tier. Nginx again (though in a LB context), Haproxy, etc.
Both spaces seem to have less talk about Varnish than they used to, and more about other platforms.
You can see a plateau, then decline (though slight) that starts for Varnish around mid-2018, here:
(Though these kinds of surveys are tricky, as they depend on outward facing headers that don't always exist, or don't always tell you enough...like Nginx doesn't always imply the cache is in use)
I also wonder what percentage hit Varnish might take if Fastly moves away. I'm sure they regret not varnish specifically, but exposing the varnish vcl directly to end users.
Edit: Yes, I agree that sites like "builtwith" are flawed, and mentioned some of the reasons above. And, I didn't mean for this comment to sound like a criticism...just an observation. I noticed builtwith's chart has a similar plateau + slight decline for Apache, starting also near mid-2018.
Varnish has a pretty big market-share as "the intelligent HTTP-router" which can be used to sort traffic to piles of legacy webservers etc, and also be the central clearing-house for detecting and fixing trouble.
Surveys such as builtwith, despite their hard work, can often not "see through the sandwich" and spot if there is a Varnish in it.
Also, I dont know about you, but from a "I want the world to keep working" reliability point of view, I do not like it when any single piece of software, FOSS or non-FOSS, becomes too dominant.
See for instance how dysfunctional the GCC-monopoly was until LLVM gave them competition.
So taking builtwith at their numbers, I'm actually fine with Varnish having "stagnated" at a market share of one fifth of the worlds top 10K websites: That keeps me humble about my code quality, but does not keep me awake at night.
> Though these kinds of surveys are tricky, as they depend on outward facing headers that don't always exist
It's not just “don't always exist”: those headers are actively recommended against by various security guidelines so many large sites heavily use things you can only infer from other characteristics.
This is also the kind of environment where I see some movement against Varnish: internal TLS requirements increase the cost of managing two services instead of one, and if you're increasingly using something like an external CDN the level of benefit from Varnish's cache declines somewhat even though the powerful request routing and manipulation features are still appealing.
I've been generally wondering what it would take to be able to flip the model to something like Cloudflare's Argo Tunnel feature where you could secure internal communications by having your various web services make an _outbound_ connection to the Varnish box which all of the requests will be tunneled over so you only need to manage one certificate there rather than one for every service/container in a complex application.
https://varnish-cache.org/docs/4.0/phk/ssl.html
Which is somewhat interesting now, as Varnish has lost a lot ground to other caching proxies that did choose to implement SSL/TLS.