> we brought down image creation time to mere seconds, even for images that were multiple GiB in size
this sounds interesting; for e.g., was wondering the other day if we could build images without actually pulling base images.. everytime we compile, we copy artifact(s) onto a multi-hundred MB base image which definitely doesn't need to be pulled everytime.
You absolutely can just build things and put them on base images without building or pulling the base image at any point. This is a central feature of ko, a simple container build tool for most Go applications: https://ko.build
There are many ways to achieve this. If you are just copying artifacts, and not using RUN, then there is in principle no need to download the base image at all. Feel free to message me if you'd like to chat (email is in my profile).
I can't comment on BuildKit, unfortunately, since I haven't used it. My experience comes from building bespoke systems (an image builder and a custom registry) fully from scratch, because we needed to have full control in order to achieve the performance we were aiming at.
Controlling both the builder and the registry is super nice btw, because they can work together. If the builder knows some of the layers already exist in the registry, it merely has to create and push the remaining ones (without downloading any of the other layers, not even those from the base image). That gives you near-instant builds once the biggest layers are cached in the registry!
Since builds usually happen in CI, and pulls happen elsewhere (e.g., a kubernetes node), in the end layers are only downloaded when the resulting container image is actually used.
Base image independent layers is something I’ve pondered about, but doesn’t feel compatible with things like apt. This conversation is giving me more reason to go implement lazy/fuse base layers though. My exports are already pretty fast due not not using tar+deduplication with similar layers, but pulling the base can still take several minutes.
> None of these are the most efficient way to do this (you can use git show and git log -S or maybe git grep to accomplish something similar), but personally I always forget the syntax and navigating a filesystem feels easier to me.
i feel like some of the old-school commands will benefit from long args, e.g., '--search'. at the time of writing, the current `git log` documentation[1]'s `-S' has _one_ instance of the word 'search'.
(un)related to the article, author went on to contribute documentation updates to git, which were much needed [2]
Funny, I use -S routinely and -G rarely: I’m almost never interested in cases where a symbol was moved around, it’s just noise, so always reach for -S first.
-S does not mean “search” tho, it means specifically searching the change history for the symbol being added or removed (but not moved, the number of occurrences has to vary). -G includes the symbol being moved (it will flag commits which both remove and add an occurrence).
“git grep” is the correct tool for Julia’s grep, however as usual it has the worst defaults: the work tree (so it’s just a slightly better “grep”), you need to give it a list of revisions to search to get a history search (so usually you pipe-xargs a rev-list into it).
thankfully more and more projects are supporting the "standard" SSL_CERT_DIR/SSL_CERT_FILE environment variables [1]
i think requests is a tricky one, as it _should_ be supporting it already based on the PR [2], but looks like it was merged in the 3.x branch and idk where that is, release-wise.
there is also native TLS on linux (idk what exactly you call it); but
slightly related, one of the more interesting issues i've faced due to mitm tls by the $job mandated CASB (cloud-access security broker)
is when python 3.13 [1] introduced some stricter validations and the CASB issued certs were not compliant (missing AKI); which broke REQUESTS_CA_BUNDLE/SSL_CERT_FILE for us
well, there was this, a while back: "experiencing a critical and recurring email delivery issue affecting recipients at outlook.com, live.com, hotmail.com, and msn.com"
"Emails to Outlook.com rejected due to a fault or overzealous blocking rules"
one of the non-intrusive approaches i have for this [1] is kubenetmon[2] which uses a kernel feature called nf_conntrack_acct to have counters for (src, dst).
it's not perfect [3] but gets the job done for me
[1] not as much "control" as it is "logging", of sorts; "especially when you just need to answer “what is my cluster talking to?”"
this sounds interesting; for e.g., was wondering the other day if we could build images without actually pulling base images.. everytime we compile, we copy artifact(s) onto a multi-hundred MB base image which definitely doesn't need to be pulled everytime.
reply