Hacker News new | past | comments | ask | show | jobs | submit | underrun's comments login

What if computing machines were the distributed system?


but i really want 3d vim.

...no seriously. i may have a problem...


How does one exit out of 3d vim?


Its author did recently, but it might be hard to reach him.


Carefully, so as to not accidentally exit the simulation


You can accomplish this with most VR hardware now but it's not very good. The issue is that text is hard to work with in VR (for the complex reasons the article describes). The eyestrain is also incredible.


I've tried working in VR on side projects. It sucks after an hour. Granted, I have a Samsung Odyssey+ from a few years ago.

The PPD talking point is absolutely a problem. If my particular headset had the ability for "many floating windows" arranged in arbitrary places, it might be better (but it doesn't - I think there's a desktop app for Meta Quests which supports this though).

So, aside from the eyestrain due to low PPD, it is legit amazing, and I could potentially have productive workdays in VR with the right system. I also purchased the counterweights which help quite a bit.


from the article: "Being able to debug any past production code is a huge step up..."

this is pretty epic. but it also means you need to keep track of what version of your code ran every past workflow because you need to run it the exact same way when you replay it, right? Is there an easy way to track in workflow metadata or something which version of a worker (commit sha or something) ran a workflow?

also i love the beginning section about history. It would be awesome if every article I read about some new technology started with a reference to how whatever-it-is really grew out of PARC or bell labs or some research paper written pre-1980.


Thanks!

You do need to run it on the same code version. There are different ways deploy code changes. If you use one of our in-built versioning systems, the version is recorded in the workflow history (and you can either keep track of version-sha mapping or use a sha as the version). Otherwise, you can add code that adds the current code version as workflow metadata.


this is a bit dense but i hope this meta-study results in some focused research on foot ware like vibram five fingers. there are some inferences one could make from this review, and despite the closer-to-barefoot aspect of some shoes, any constraints and added protection from the ground are going to have an impact on how we walk and it would cool explore and optimize the balance between protection for the foot and taking full advantage of its anatomical function.


much more in depth approach to designing for simplicity with examples of why alternatives aren't as good:

https://zguide.zeromq.org/docs/chapter6/#Designing-for-Innov...


Adrian Colyer dug into this a little further on the morning paper:

https://blog.acolyer.org/2019/04/18/keeping-master-green-at-...

His analysis indicates that what uber does as part of its build pipeline is to break up the monorepo into "targets" and for each target create something like a merkle tree (which is basically what git uses to represent commits) and use that information to detect potential conflicts (for multiple commits that would change the same target).

what it sounds like to me is that they end up simulating multirepo to enable tests to run on a batch of most likely independent commits in their build system. For multirepo users this is explicit in that this comes for free :-)

which is super interesting to me as it seems to indicate that an optimizing CI/CD systems requires dealing with all the same issues whether it's mono- or multi- repo, and problems solved by your layout result in a different set of problems that need to be resolved in your build system.


> For multirepo users this is explicit in that this comes for free :-)

Only if you spend the time to build tools to detect commits in your dependencies, as well as your dependent repositories, and figure out how to update and check them out on the appropriate builds.

So, no, it doesn't come for free.


sorry, "this" is rather ambiguous.

You are totally correct that to achieve the same performance, correctness, and overall master level "green"ness in a multirepo system you would have to either define or detect dependencies and dependent repos, build the entire affected chain, and test the result. That part is much easier in monorepo.

What I was referring to with "this" is that Uber's method of detecting potential conflicts. In multirepo land it would be a "conflict" if two people commit to the same repo. In multirepo, therefore, detecting potential conflict is trivial.

If Bob commits to repo A and Sally commits to repo B, their commits can't result in a merge conflict. Well, unless the repos are circularly dependent - which would be bad :-) don't do that. Of course, monorepo makes that situation impossible so there's an advantage for monorepo.

It seems like whether you have mono- or multi- the problems solved by one choice will leave other problems the build system has to solve that it wouldn't have to solve if the other option were chosen.

Different work would be required in multirepo but it would be work to solve the problems that monorepo solves just by virtue of it being a monorepo.


> You are totally correct that to achieve the same performance, correctness, and overall master level "green"ness in a multirepo system you would have to either define or detect dependencies and dependent repos, build the entire affected chain, and test the result. That part is much easier in monorepo.

You also would need to do that as an atomic operation (in the mono-repo + especially with a commit queue you're building on the atomicity of git).

Having to unwinding that transaction if you aren't atomic can get you into a big mess at larger scales.

Here's a good related talk: C++ as a "Live at Head" language: https://www.youtube.com/watch?v=tISy7EJQPzI by Titus Winters (from Google).


thanks for the link - looking forward to watching.

Currently, I'm not convinced that you need to track and apply commits across dependencies and dependent systems atomically/transactionally to have a sane build environment even at scale, but you definitely get that part free with monorepo.

Any links to docs or presentations that address that specific issue would be very welcome :-)


> If Bob commits to repo A and Sally commits to repo B, their commits can't result in a merge conflict.

This holds true when A and B are leaf repos, but gets tricky with repos inside a dependency graph. More concretely, if C depends on both A and B, and it turns out that C depends on A and B in such a way that A_bob and B_sally are mutually incompatible, you need some kind of mechanism for reconciling that.

Of course, exactly as you point out, mono and multi are two tradeoffs for the problems that large codebases intrinsically are.


Package managers solve it quite well. Just depend on the latest version of your dependencies and tag a new version whenever they change.


This doesn’t work when an underlying system changes, and upgrading is mandatory for all clients or package dependants (happens often at scale for a multitude of reasons).


That's not good stewardship. You have a better API? Great, convince us it's worth investing in soon, you can even deprecate the known-good version.

There's always a window where both will be in use, because we can't synchronously replace every running process everywhere (not that it's even a good idea without a canary). The shorter you try to make that window, the more needless pain is created and plans disrupted. While we could use prod to beta test every single version of everything, that shouldn't be our priority.


That's not reality for most large companies, even though it's the right mindset for most software libraries. Ex: A new legal requirement comes in, resulting in a mandate that fields X and Y for certain queries, that are being done all over the codebase, now have to be tokenized. This is a breaking and mandatory change, with no room to allow systems to stay behind, and expensive consequences.

In this case you'll have a very short transition, between all consumers updating their client code (possibly in a backwards compatible way) and the change in the implicated system being deployed, not the other way around.


If adopting the new API is mandatory, every team should be told why it's mandatory, and we'll reprioritize and get it done. Doing it to our code behind our back is passive-aggressive and likely to break stuff, because who is monitoring and reporting on a change in our system's behavior that we didn't even see?


This is essentially the problem go has had, precisely because you could only ever pull the latest from master.

Introduce a breaking change into a common library and now you have to update every other dependency to support it.

Not so bad in a monorepo. But when your codebases are distributed?


It's almost like semantic versioning exists for some reason


Or realize that it is an advantage to control your dependencies.


Now you have to desperately try to get people to upgrade every time an important change goes through. And you quickly live in a world where you need to maintain tons of versions of all your services.


Not exactly for free, but there are free tools that handle this job for you very nicely:

https://zuul-ci.org/


You are right to say that conflict analyzer tries to treat commits independently based on the service or app (which are usually in separate repositories in a multi-repo world). However, note that the problem of conflicting changes (or a red master) exists even when you are in a multi-repo world as you could have one repository getting a large number of commits.

In fact, at Uber we have seen that behaviour with one of our popular apps when we did not have a monorepo. The construct of probabilistic speculation explained in the paper applies even in this scenario to guarantee a green master.


Do you mean the construct of probabilistic speculation applies in multirepo because you may end up with a hot spot repo that receives a high volume of commits at once?

Or do you mean that multirepo could also benefit from the construct of probabilistic speculation by ordering commits across multiple repos such that you are maximizing the number of repos that have changed before you build and minimising the number of commits applied to single repos?

Or both :-)


the former actually.

If you have 1 app that's the bread and butter of your company and, 60% of your 2000+ engineers working on various features of that one app, then even in a multi-repo world, you are going to have that 1 repo receiving ton of commits and the problem of keeping it green remains. Prob. speculation helps there.


Funny that you would draw a comparison to a Merkle tree. At one client they had such coupling between systems CI/CD was nearly impossible without either an explosion of test environments or grinding everything to a near halt.

We began working with the idea of consensus based CI/CD. If you pushed a change, you published that to the network. It gave other systems the opportunity to run their full suite of tests against the deployment of your code. Some number of confirmations from dependent systems was required to consider your code "stable". This progressed nearly sequentially assembling something like a block chain.

Ultimately the client was unable to pull this off for the same reason they were unable to decouple the systems: lack of software engineering capability.


The advantages of advancing technology and building smaller safer reactors would go a long way to avoiding such disasters. Because things can go wrong is not a reason to stop research into making them better and safer. The real way to keep ahead of this is not to extend the life span of existing reactors but to build aggressively in order to take older ones offline as newer smaller safer reactors are ready.

And keep doing that.


Iterating aggressively is great when the consequences aren't as dire as nuclear mistakes.

The last thing I'd want is more plants built aggressively when stupid mistakes happen at the non-aggressive rate, such as installing the reactor backwards.

https://en.wikipedia.org/wiki/San_Onofre_Nuclear_Generating_...


+1

Design is not actualization.

So the salient question becomes:

How robust are modern reactor designs to manufacturing/construction/installation/etc. errors?


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: