A surprising number of solutions can be realized in ways that don't actually need much of a control plane if you introduce a few design constraints.
But if you do need one, I guess Kubernetes is perhaps the safe bet. Not so much because I think it is better/worse than anything else, but because you can easily find people who know it and it has a big ecosystem around it. I'd probably recommend Kubernetes if I were forced to make a general recommendation.
That being said, this has been something that I've been playing with a bit over the years. I've been exploring both ends of the spectrum. What I realized is that we tend to waste a lot of time on this with very little to show for it in terms of improved service reliability.
On one extreme we built a system that has most of the control plane as a layer in the server application. Then external to that we monitored performance and essentially had one lever: add or remove capacity. The coordination layer in the service figured out what to do with additional resources. Or how to deal with resources disappearing. There was only one binary and the service would configure itself to take on one of several roles as needed. All the way down to all of the roles if you are the last process running. (Almost nobody cares about the ability to scale all the way down, but it is nice when you can demo your entire system on a portable rack of RPis - and then just turn them off one by one without the service going down)
On the other extreme is having a critical look at what you really need and realize that if the worst case means a couple of hours of downtime a couple of times per year, you can make do with very little. Just systemd deb packages and SSH access is sufficient for an awful lot of more forgiving cases.
I also dabbled a bit in running systems by having a smallish piece of Go code remote-manage a bunch of servers running Docker. People tend to laugh about this, but it was easy to set up, it is easy to understand and it took care of everything that the service needed. The kubernetes setup that replaced it has had 4-5 times the amount of downtime. But to be fair, the person who took over the project went a bit overboard and probably wasn't the best qualified to manage kubernetes to begin with.
It seems silly to not take advantage of Docker having an API that works perfectly well. (I'd research Podman if I were to do this again).
I don't understand why more people don't try the simple stuff first when the demands they have to meet easily allow for it.
I did something similar once for a mining technique called “core logging”. It’s a single photo about 1000 pixels wide and several million “deep”: what the earth looks like for a few km down.
Existing solutions are all complicated and clunky, I put something together with S3 and bastardised CoGeoTIFF, instant view of any part of the image.
I'm curious about the "core logging" photo. Where can I find one? Do you have an implementation of your solution? I would be curious to have a look at it.
I don't know enough about the science side to take it any further on my own.
The "tech" part of what I started building is really quite simple: convert the images to Cloud-optimised GeoTIFF, then do range requests to S3 from the browser.
Might not be possible to find any, they’re expensive and niche. If you reach out (email in profile) I can show/share how it works (nothing currently public).
I would have given the OOP the effort and due respect is formulating my response if it was phrased in the way you're describing. It's only fair that comments that strongly violate the norms of substantive discourse don't get a well-crafted response back.
I’m just a casual observer of this thread, but I think you’d find it worthwhile to read up a bit on zero-copy stuff.
It’s ~impossible in Python (because you don’t control memory) and hard in C/similar (because of use-after-free).
Rust’s borrow checker makes it easier, but it’s still tricky (for non-trivial applications). You have to do all your transformations and data movements while only referencing the original data.
If you do both (use flipped parentheses around the operators), it makes even more sense, and makes the parsing trivial to boot: just surround the entire expression with parentheses and parse normally. For instance:
1 + 2 )( 3
Becomes
(1 + 2 )( 3)
Which is actually just what the author wants. You might even want multiple, or an arbitrary numbers of external parentheses. Say we want to give the divide the least precedence, the multiply the middle, and the add the most. We could do that like:
1 + 2 )/( 3 ))(( 4
Surround it with two sets of parens and you have:
((1 + 2 )/( 3 ))(( 4))
I haven't just proved to myself this always does what you expect, though...
Wait, no. It makes no sense to use the same characters! An "inverted" opening parens is visually identical to a "normal" closing parens. IMHO the entire proposition is inane.
Users of a library will generally instruct their type-checker not to check the library.
So only the outer API surface of the library matters. That’s mostly explicitly typed functions and classes so the room for different interpretations is lower (but not gone).
This is obviously out the window for libraries like Pydantic, Django etc with type semantics that aren’t really covered by the spec.
A Python ORM, inspired by Drizzle and the like. Whenever I come to Python I'm frustrated by the ORM options. They generally lack type-safety on inputs and outputs, or useful type hints.
SQLAlchemy is an institution but I think it's hard to use if it's not your full-time job. I check the docs for every query. I want something simple for the 80-99% of cases, that lets you drop easily into raw SQL for the remaining %.
I'm going to keep hacking at it, would love to from anyone who thinks this is worthwhile (or not). Also:
- The interface for update queries is clunky. Should I add codegen?
- Should I try to implement a SQL diffing engine (for migrations). Or just vendor sqldef/similar...?
You could get an IKEA Dirigera, but one of the upsides of Matter is that you do not need the manufacturers hub anymore. So an Apple Homepod or a Home Assistant instance with a Thread stick will do as well! (Or any other Matter hub for that matter of course)
You're looking for a Thread border gateway. Lots of stuff already has it. Someone mentioned AppleTV and HomePod mini. Newer Google Nest speakers/displays have it as well.
Personally, I pair my wifi and Thread matter devices to my Apple Home, as each Apple TV behaves as a redundant, ethernet connected gateway. I then do a secondary pairing to Home Assistant and Google Home. Local control and it works very well.
Is it absolutely necessary to have a base/gateway? This Verge article[0] seems to imply not, but it's not at all clear to me what I lose.
If I just want a smart switch that controls a smart light, can I do that without a hub? Can I use my phone to control that light/switch in a pinch? I'm not averse to spending $100 or whatever, but it's just more _stuff_ that I'd rather not think about.
It's not strictly necessary to have separate a hub/controller, no: After all, a Matter controller is just a small computer-widget that sits on a network and talks with IP packets.
A pocket supercomputer, such as an iPhone, theoretically works just as well: It's a small computer-widget that sits on a network, right? It just happens to run on batteries and be carried around in your pocket.
It's just software at that point.
At the end of the day: The Matter devices are paired with the controller, similar to how Bluetooth devices are also typically paired with a main brain-box. (Except: A Matter device can be paired with many controllers concurrently, whereas a lot of Bluetooth devices can only be paired with one at a time.)
The network connection doesn't have to be permanent: It can work when controller is present on the network, and it will [perhaps obviously] cease to work when the controller is absent.
So if Apple has software that runs within an iPhone and acts as a Matter controller, then: Sure, no additional hardware is needed to wiggle the state of a Matter light bulb using your iPhone.
(And if that kind of local control is all you ever care about for controlling stuff then... that's good enough.)
Can't Matter devices connect directly to Apple HomeKit? So if you have a HomePod (or even a MiniPod though I'm not sure) it should connect all these Ikea devices too.
Bit of a plug but I just started working on a drizzle-esque ORM[1] for Python a few days ago and it seems somewhat appropriate for this thread. Curious whether anyone thinks this is a worthwhile effort and/or a good starting point syntax-wise.
I use Coolify for side projects, haven’t investigated whether I’d want to use it for bigger/importanter stuff.
reply