A new bug appears, it’s in an encryption layer. You solve this by deciding to disable the encryption layer because user experience is better without the errors. You write it up as a recruitment piece for your engineering team.
There may be some good answers and lessons, but they didn’t make it into the article. Saying it’s on a cloud provider’s private network so encryption between your nodes isn’t necessary is a bold choice. Also, what happened to the root cause? Why did it start failing a week ago? Was a downgrade of the offending code not possible?
Not all bug investigations are worth really digging into. Sometimes the right call is to find any fix and move on. But all the nuance, judgement, implications, and lessons learned failed to make it into this post. And they are what make reading incident reports interesting for most engineers.
As pointed out in the article, naproxen is an NSAID like Ibuprofen, though slightly more COX1 selective. It likely has a somewhat lower risk of serious renal and cardiovascular events, but higher risk of GI bleeds. There are some studies that show little to no increase cardiovascular risk, but most do show some or even comparable to ibuprofen.
Convenience vs ibuprofen is a thing given the longer half life, but it still generally comes with similar risks. If you are taking anything for more than just an occasional headache, definitely discuss with a doctor, COX2 selectives like celecoxib may be a better risk profile and even more convenient.
(COX1 and COX2 selectivity loosely separate which systems get the brunt of the side effects)
I weirdly always found Naproxen much more effective than ibuprofen but also find Celebrex great which seems to further confuse the whole COX 1 vs 2 situation
Recently, I've had to take Celebrex (celecoxib) for a back injury and I've been taking it longer than is normally recommended—and that truly worries me (I've determined I have to come off it ASAP).
Where I am (Australia), most doctor's prescriptions that have to be taken long-term are issued as the first script plus five repeats. Not so with Celebrex, a script can only be dispensed three times (3 x pk of 30 200mg capsules — one per day, for 90 days max) and scripts can only be dispensed every 21 days. Reason: Celebrex is only recommended for short-term use because it's considered a dangerous drug with possible irreversible side effects if taken for too long.
This was not news to me even before taking Celebrex, way back in the 1990s I was prescribed its sister drug called Vioxx (rofecoxib) for back pain and it was much more effective than Celebrex (at least it was for me).
Anyway, sometime around 2000 I read an article in the journal Science about a significant statistical increase in deaths by heart attack, stroke etc. by those talking rofecoxib. At the time I said to myself it won't be long before Vioxx is banned. It took another three to four years for that to happen as Merck Pharmaceuticals fought the decision every inch of the way. It's worth reading the Wiki about this (when it's between a drug company and millions of dollars profit patients come off second best):
What's relevant here is that the related drug Celebrex survived because its side effects—whilst manifestly similar—aren't quite as bad as Vioxx. In short, Celebrex's COX-2 selectivity versus other less selective NSAIDs like aspirin (which target both COX-1 and COX-2) was deemed sufficiently beneficial despite its potential serious side effects.
Note: I'm not offering medical advice here and you should always take that from your medical practitioner. I mention this because only several days ago I had a discussion with two younger doctors who'd never heard of Vioxx let alone the Vioxx/Celebrex controversy.
You may be interested in this YouTube video on Vioxx. Unfortunately it's over hyped and designed to alarm but it's essentially factually correct: https://m.youtube.com/watch?v=K0GrFnOpJoU
The higher risk of GI bleeds is could be somewhat balanced by not having to take as many.
There are also slow release forms of naproxen. (Which make sense given its long action: lets people fade in the next one while the previous dose slowly fades out). That could also help make it easier on the GI tract.
> The higher risk of GI bleeds is somewhat balanced by not having to take as many!
Unless I am missing something, the data really doesn't back that up. naproxen is much more longer lasting and has a higher chance of causing ulcers. Hence why its not over the counter in the UK and is prescribed with omeprazole to reduce the risk of issues.
I'm reading about this in more detail. Indeed, it's not the contact between the medication and the digestive tract that is the problem, but simply its presence in the blood stream. By inhibiting those certain enzymes, it reduces the production of prostaglandings, causes problems for the lining.
Naproxen will be around longer due to its long half-life, so it creates more opportunity for this problem.
Neither WebRTC or WebGL are remotely ‘useless’. Very fair though to say that you would prefer to have them disabled and/or whitelisted for certain sites.
Calling the industry largely a scam is pretty strong. Of course for the small minority of technically competent people with interest and time it will always be cheaper to do something yourself rather than pay a business to accomplish the same thing. But most people cannot/do not want to do it themselves, and regulations are there at least partially to help protect them against the house-fire-waiting-to-happen untrained handyman.
Sure a bunch of businesses opportunistically up-charge, some I'm sure are predatory, and there are obviously efficiencies to improve, but overall scam it is not.
Not really. The majority of data center water withdrawal (total water input) is consumed ("lost" to evaporation etc...) with a minority of it discharged (returned in liquid form). I believe it's on the order of 3/4ths consumed, but that varies a lot by local climate and cooling technology.
There's lots of promising lower-consumption cooling options, but seems like we are not yet seeing that in a large fraction of data centers globally.
Best comment by far. The post suffers from making good points about the lack of rigor and narrative nature of the book, but then does exactly the same thing to claim the opposite conclusion the book makes.
Primary energy sources are what they are, both your comment and the linked article seem to imply discussing them should lead to a deserved punch in the face. Can you help me understand why?
As far as I can tell, your link argues that if we overcome all the practical challenges (politics, resources, financing, technical innovation) and go all-electric for global energy, we only need ~1/3 as much input energy potential as we use today for the same useful work. That’s useful, but the hard part lies in those practical challenges. And the primary sources of global human energy use are a long way away from that goal.
So should we strive to get there? Sure. Should we be tactical about how? Yes. And the link seems to argue that as well. But is it reasonable to hit our 2050 goals based on the current global fossil fuel usage? Not really. So I’m really missing how this refutes Smil’s article, and why “primary energy” is such a stupid thing.
The problem with it is that it makes it easy to make bad faith arguments for why we can’t or shouldn’t transition (kinda like is being done here).
Take for example paragraphs like:
> Primary electricity (hydro, nuclear, wind, solar, and a small contribution by geothermal plants) accounted for no more than about 18% of the world’s primary energy consumption, which means that fossil fuels still provided about 82% of the world’s primary energy supply in 2022.
Are used as justification for why green green energy is a scam, it can’t be done, or it’s too expensive, etc., etc. after all 82% of primary energy is still from fossil fuels.
Except we don’t have to replace 82%, since 2/3rds of that is wasted. Of 100 kWh we’re already done 12 kWh and only need to add 27 (NOT 82) more kWh of electricity to replace all the fossil fuel usage. And that’s before talking about any efficiency gains (e.g heat pumps with COP >4).
Ah, seems like I was missing some context where the fossil fuel and anti-renewable folks have been using the term in arguments against trying to change.
I’m not sure of Smil’s politics but to be fair, there’s nothing in that quote that is inherently misleading. I can see through how others could spin it, and I’ll be more careful knowing the term has some politics behind it now. To me his argument in the article is that it’s not practical to expect a transition in a 25-year timescale, not that it’s impossible or not worth working on.
Heat pumps are a good example where the practice has been a lot harder than we might hope. Sure COP > 4 for heating is great, but the units are very expensive today, and in most of the US and Europe with sub-zero winter temps operate with much worse efficiencies, making them significantly more expensive to operate. I’m sure with effort those issues will improve, and major policy shifts can help mitigate some of the costs. But especially without a strong will today those changes are practically too far off for the 2050 target.
> there’s nothing in that quote that is inherently misleading
The discussion of the issue in terms of primary energy is the very thing that's inherently misleading. To move away from fossil fuels we do not have to replace the primary energy, we have to replace the useful energy that comes out the other side. From the Sankey diagram in the article I linked [0], 67.5 units of energy are not useful energy.
To put it to an extreme, instead of 67.5 units beings wastes, it could be 100 billion units for 32.5 units of useful energy produces. Focusing on the 100 billion is inherently misleading since they are irrelevant when the replacement technology basically creates the useful energy with over 100% efficiency at times.
Heat pumps. Yes their COP is lower during cold winters, but that brings in 2 discussions.
1) any COP value above 1 means that we'll need less primary energy than when buying something, and even in cold weather they manage a COP above that [1].
2) Lower COPs will cost you more, depending on what your natural gas prices are like due to any crazed lunatics invading their neighbours. Which conincidentally is only what pushes electricity prices up in many places that use natural gas for electricity (even just peak demand).
The capital cost difference also depends drastically on situation. Many climates need both heating and cooling, so the price of heat pump versus furnace + AC unit is much smaller than heat pump versus furnace.
> But especially without a strong will today those changes are practically too far off for the 2050 target.
I agree, and even replacing the 1/3rd of the primary energy will be a tough challenge. But Vaclav continual framing in terms of primary energy is actively used to push inaction. His critics have been vocal about this point (and others) for a while, he should know better by now.
Thanks, you have helped at least me think a bit differently about this. I still believe primary energy is a valid way to look at the problem, but see more clearly how easily it can lead an uninformed audience to a bad conclusion.
And on heat pumps - it’s sad to reflect that even if we replaced all heating, it’s still only a couple % of the total rejected heat. There are few easy wins in this game, just many different ways we need to chip away at it.
> it’s sad to reflect that even if we replaced all heating, it’s still only a couple % of the total rejected heat.
It's actually not as bad as it looks.
Even if the home heating is not the biggest contributor from that chart, it is still a worthwhile target. Though EVs are likely a more impactful choice.
One thing not captured by that chart are the 2nd order effects of either heat pump or EV switching. Part of what makes switching economically unattractive (aside from allowing the fossil fuel options to pollute for free) are the economies of scale present for fossil fuels. However, those same economies of scale can easily flip to diseconmics of scale as customers switch away. Every ICE car replaced by an EV makes gasoline and diesel more expensive, the same thing for heat pumps and natural gas andheating oil.
So for natural gas, removing the stream going to residential, significantly impacts the economic calculation for commercial and industrial uses.
For gasoline and diesel, the impact as even more serious. Out of every barrel of crude oil 70% gets turned into gasoline/diesel [0]. The unit economics there are going to be even worse as gasoline demand continues to drop.
A gas furnace converts 1 J of chemical potential energy (higher heating value for natural gas) into essentially 1 J of internal energy in the air (raising it’s temperature).
A heat pump can take 1 J of electricity and move (realistically) up to 5 J of internal energy from A to B.
A layman description (if not actually accurate) is that while a gas furnace (or electric resistive heating) can be 100% efficient, a heat pump can be 500% efficient.
Can you point me to the actual literature on the mechanics of heat pumps b/c I don't think you're explaining properly what's going on. If you can get 5J out of 1J then you have a perpetual motion machine & those are physically & logically impossible (assuming physically relevant axioms).
Links below, tldrs here: A heat pump does what the name suggests: it pumps heat. Resistive heating and burning gas is converting energy from one form into another. A heat pumps moves energy from A to B (making A colder and B hotter in the process) in literally the same way AC units
You get 5 J of heat out for every 1 J of electricity in because we're being funny with the units. You put in 1 J of electricity and the rest is put in as heat from your source (A) and then moved into B.
That obviously doesn't make sense if you know basic physics. Converting gas into heat by combusting it heats up the air in the room directly. Burning that gas in a turbine & then transferring that energy through a bunch of transformers to get to the heat pump can't give you more heat than what went into combusting the gas in the first place. This obviously doesn't work in reverse to cool a room but the direction I laid out is obviously correct.
You're not using funny units, you're confused about the thermodynamics of the situation. Heat pumps are more efficient air coolers than standard air conditioners but they're not giving you more energy than what you put into it.
> you're confused about the thermodynamics of the situation.
I assure you I am not, it is actually you who still seem confused about where the energy is coming from.
> they're not giving you more energy than what you put into it.
When you burn gas in a furnace, all of the energy that raises the temperature of the house comes from the gas.
When you run a heat pump you have two sources of energy:
1. Electricity running the thing.
2. Heat from the outside that you're moving inside.
#2 is where most of the energy that actually heats up the house comes from. The electricity is used to move it from outside the house to inside.
> Burning that gas in a turbine & then transferring that energy through a bunch of transformers to get to the heat pump can't give you more heat than what went into combusting the gas in the first place.
It actually can. A combined cycle plant can be ~60% efficient (chemical energy -> electricity). Say another 70% for getting it from the plant to your heat pump, then a COP 3 (or "efficiency" of 300%) gives you 0.6x0.7x3 or 1.26. So for every J of natural gas you burn in that plant, you'll heat your house with 1.26 J (compared to at best 1 J, realistically 0.9 J, for a gas furnace).
If you instead look at a ground source heat pump, you can get a COP of ~7 [0]. You're now putting ~3 J of heat into your house of each J of natural gas.
That's clever accounting but your overall system (house + outside + plant + fuel) is still less than 100% efficient. That's basic physics/thermodynamics.
Heat pumps are basically taking advantage of solar + geothermal radiation that ends up in the ground & air but once you account for the solar + geo radiation then it becomes obvious that all a heat pump is doing is accelerating the production of entropy. You're either normalizing the delta between inside & outside or increasing it but in both cases the overall entropy of the system goes up. Whereas your accounting seems to suggest you somehow get more energy than what was available which is obviously unphysical.
But my accounting was always about the energy that we as humans need to supply. This discussion was originally about how talking about the energy transition in terms of primary energy is inherently misleading.
A heat pump does not magically create the difference between work input and heat output, it pulls it from second source. But that source is free. All we have to provide is the work.
Replacing a gas burner with a heat pump does not require us to replace 1 J of chemical potential energy with 1 J of electricity, instead replace it with 0.33 J of electricity (or even less).
So there’s another force at work here that to me answers the question in a different way. Agents also massively decrease the difficulty of coming into someone else’s messy code base and being productive.
Want to make a quick change or fix? The agent will likely figure out a way to do it in minutes rather the than hours it would take me to do so.
Want to get a good understanding of the architecture and code layout? Working with an agent for search and summary cuts my time down by an order of magnitude.
So while agree there’s a lot more “what the heck is this ugly pile of if else statements doing?” And “why are there three modules handling transforms?”, there is a corresponding drop in cost to adding features and paying down tech debt. Finding the right balance is a bit different in the agentic coding world, but it’s a different mindset and set of practices to develop.
In my experience this approach is kicking the can down the road. Tech debt isn't paid down, it's being added to, and at some point in the future it will need to be collected.
When the agent can't kick the can any more who is going to be held responsible? If it is going to be me then I'd prefer to have spent the hours understanding the code.
This is actually a pretty huge question about AI in general
When AI is running autonomously, where is the accountability when it goes off the rails?
I'm against AI for a number of reasons, but this is one of the biggest. A computer cannot be held accountable therefore a computer must never make executive decisions
The accountability would be in whoever promoted it. This isn't so much about accountability, as it is who is going to be responsible for doing the actual work when AI is just making a bigger mess.
The accountability will be with the engineer that owns that code. The senior or manger that was responsible for allowing it to be created by AI will have made sure they are well removed.
While an engineer is "it" they just have to cross their fingers and hope no job ending skeletons are resurrected until they can tag some other poor sod.
What does Germany use to manage microorganism growth in it's water distribution system? As I understand cloramine/chlorine is used to keep the small amounts of microorganisms that will always be present in water and pipes from growing into a problem while it travels/sits in the distribution system.
There may be some good answers and lessons, but they didn’t make it into the article. Saying it’s on a cloud provider’s private network so encryption between your nodes isn’t necessary is a bold choice. Also, what happened to the root cause? Why did it start failing a week ago? Was a downgrade of the offending code not possible?
Not all bug investigations are worth really digging into. Sometimes the right call is to find any fix and move on. But all the nuance, judgement, implications, and lessons learned failed to make it into this post. And they are what make reading incident reports interesting for most engineers.
reply