Does anyone have a good explanation for what Docker+WASM actually is? This announcement explains what WASM is and what Docker is, but I already knew both of those and am still confused about what Docker+WASM is. Is it just wrapping a WASM binary in a Docker container?
"Docker+Wasm" is just a shorthand for the Technical Preview build, which allows you to build both traditional container apps, as well as Wasm apps. Behind the scenes, we try to let Wasm apps be developed largely without interference from any container technology — just giving you a good local environment you can use to code against. That said, if you want, we do offer the ability to run Wasm apps within a Docker Compose application. We do also offer the possibility to package Wasm apps within an OCI image, with an embedded Wasm runtime (WasmEdge) so you can a) easily share these via an image registry like Docker Hub, AWS ECR, etc. and b) easily run this anywhere you’d run a container. That said it’s not mandatory, and if you want the benefits of (a) without the benefits of (b) you can easily unpack the image to just get the Wasm payload and run that however you want. We dove into the details of the approach at Kubecon today, and the video should be coming out shortly.
I'm still confused. This is the big thing I'm not really getting:
> which allows you to build both traditional container apps, as well as Wasm apps
I can already do that. Using Rust for the sake of example: `cargo build` can give me a WASM binary, and `docker build` can give me a container. Is Docker+WASM going to replace `cargo build`? Or is it going to wrap the WASM binary produced by cargo in another layer of abstraction? If the latter, how is this new layer of abstraction different from just using one of the WasmEdge docker containers [0]?
I'm not trying to be combative, I'm just sincerely confused at what problem this technical preview is intended to solve.
> I'm just sincerely confused at what problem this technical preview is intended to solve.
This is all good feedback, and we’ll definitely try to explain the added value better in the future. The main advantages we see in this technical preview are:
1. Easy, reproducible dev environment to quickly & reliably develop cloud/edge apps that target Wasm, or code frontend apps that target a Wasm backend (for example, as part of a microservice architecture). This is particularly helpful if you build apps that have a mix of Wasm & container components[0]
2. Easy way to share & deploy Wasm artifacts, using trusted infra like Docker Hub, but also Dockerfiles and Docker Compose
3. Transparent, reliable way to deploy Wasm applications to existing container-based infrastructure such as k8s (via OCI images) — but these apps can also be “unpacked” to run natively on edge infrastructure
> is it going to wrap the WASM binary produced by cargo in another layer of abstraction? If the latter, how is this new layer of abstraction different from just using one of the WasmEdge docker containers [0]?
Our approach is close to this. First, it was built with the WasmEdge folks, so you’re correct to detect the similarity. Second, it does wrap resulting artifacts into a OCI image, because we believe that can generate a lot of advantages (points #2 and #3 above) BUT you can also easily unpack the Wasm payload from the binary image at deploytime/runtime if you’d rather deploy your app on Wasm-native infrastructure (as opposed to container-native infra)
I think you are falling into a “deeply-technical explainer” trap. You’ve been so deeply engrossed in the use-cases of this novel technical combo that they are all assumed understood from your POV.
And, this tech is so general that there’s a second trap of “What can it do? It can do anything! Sure, but what can it do??” Again, because I’m your head “anything” is pre-supposed to a narrow set of goals that this tech fulfills very nicely. It cannot, for example, take my dog for a walk. So, it can’t do “anything” ;p It’s hard to get out of that head-space because you’ve been so deep in it for so so long.
But, I and 85% of people here have no idea what narrow set of goals Docker+Wasm fulfills. Something about apps and security something something. Mostly for servers probably.
Some awesome-fit exemplar use cases would help a lot.
What's confusing to me is the "main advantages" Tim describes are the same as Docker, so I'm left wondering what's different about it from Docker? The only thing I can parse out of it that's different is 'apps can also be “unpacked” to run natively on edge infrastructure', but I'm not entirely sure what that means.
My best guess is that Tim is trying to say, "now can run your Docker apps on AWS Lambda"?
AWS announced support for containers on Lambda last year[0]
> the "main advantages" [redacted] describes are the same as Docker
Yup, that’s it! If you value Docker to build container apps, we think this will help you build Wasm apps in the same way, and the only container-centric abstraction this Technical Preview uses (packaging artifacts as OCI images) can be bypassed, if you prefer to deploy your artifacts as native Wasm binaries. The latter can be helpful if you are trying to get the full speed & efficiency benefits of a Wasm-native deployment, as the shim in our OCI package introduces a small performance penalty.
"Docker images can now be deployed directly to the WASM runtime! This means your AWS Lambdas, Cloudflare Workers, etc. will boot faster and cost less..."
When you rehash what Docker already does, it's watering down the messaging. Even adding "AWS announced support for containers on Lambda last year" in the last reply made the voice in my head ask again, "What's different about it? How is it better?"
That’s not quite right. You can’t take an existing container app and just "export" it as Wasm. (Technically you might, but it would require a pretty big re-architecture and re-write, as Wasm doesn’t support garbage collection or multithreading at the moment. It also requires you use a language that can be compiled to Wasm, which can be limiting. Due to this, Wasm — at this stage — is probably best fitted to functions rather than full apps, although that is changing quickly.)
What you can however, is build apps for Wasm (or apps that combine Wasm and containers) with the same ease you currently enjoy when building pure container apps, i.e. see my comment above [0]
I think what's missing is why you'd want to do that.
If I have a rust app on a scratch image, why would I want to turn it into a wasm container?
My assumption is because wasm can run on multiple platforms (x86 and arm) so one image supports both, is that correct? Are there other reasons not as obvious?
Assuming you had code that somehow could either be packaged as a linux container, or as a Wasm binary, then the advantage of the latter would be that yes Wasm supports multiple CPU architectures out of the box, it also consumes less resources (memory, etc.), will usually have faster start times, and the Wasm security sandboxing is stronger.
Really? I would not have expected that. Is that just under the assumption that most apps have an underlying OS (like alpine) and aren't on a scratch container?
what are the limitations of this? ok, for example can you deploy postgres on the browser with this? can you have a full OS container running on the browser with this, say ubuntu shell frontend with some javascript and an ubuntu container running on the browser?
The first impression from 'Docker + WASM' surely sounds like that is what it is, but after about 30 mins, I am not so sure that is actually the case.
I have some feedback, though more general: can you please stop pushing your flavor of Docker Dev Environments that only works with your proprietary application and get behind the https://containers.dev standard AKA Dev Containers?
You are going to lose this battle because your Dev Environments don't have a lot of buy-in and you're not standardized and open specced. I'd love to see Docker take a more OSS-friendly approach.
So if I understand you correctly, you want Docker to be a trusted host for wasm binaries as well as container images? Where the tooling you've made is intended to aid users along this path.
It sounds like you're anticipating a new market segment for artifact deployment and want to be its primary service provider.
This seems like a good way to muddle the remaining value prop that docker has. I have zero idea why I'd want wasm via docker tooling vs what exists, especially as people more more and more to not-docker for building and running their containers. I think I see what someone is trying to do, but I don't know any dev looking for this or having a problem solved by it.
> I don't know any dev looking for this or having a problem solved by it.
Judging by the reception at KubeCon & elsewhere today, we think at least some folks are excited by it. But it’s still early, and who knows, you may be right in the end. We launched this as a technical preview to test a hypothesis and learn from it, and so far the interactions from this HN thread alone have been greatly helpful.
I tried to answer this above[0]. Instead of trying to explain it again, I’d encourage you to give it a try[1], and if after going through the 5-minute tutorial you still don’t get the point then a) maybe we messed up (and I’ll be sorry for having wasted your time!) or b) maybe it’s not for you (and I’ll also be sorry I wasted your time). It took me a while to wrap my head around this Docker+Wasm thing too when I first heard about it internally — then again it took me months to wrap my head around my first demo of Docker, so maybe I’m just dense!
It might be easier if you simplify it to, "We're trying not to be just the app you choose to run containers with. You can now use non-container runtimes like WASM."
So docker+WASM does not build a Linux container image with the WASM application inside but it builds a WASM application packaged as a docker image and started with docker run instead of (say) an ELF or Windows or Mac binary started with the OS exec, whatever it is?
If this is the case I'd leave docker outside the name of the technology. I imagine the confusion. At least one customer of mine doesn't fully get the difference between a docker image and a VM yet, after years they are using docker in production.
> So docker+WASM does not build a Linux container image with the WASM application inside but it builds a WASM application packaged as a docker image
With this preview, we are leveraging the OCI specification that defines how to build an image. Linux/Windows containers are the most common use of this specification, but many other types of artifacts exist (OPA policies, Helm charts, etc.).
When using any of these other artifact types, tooling has to know how to use that specific artifact type and run it (eg, extract the image and run it as a container or extract the Helm chart and deploy it). In our Wasm use case, we are doing the same thing... package and ship a Wasm module and we'll extract it and run it on the new Wasm runtime. That runtime then "converts" the Wasm module into native machine code for the OS you're running on.
> If this is the case I'd leave docker outside the name of the technology. I imagine the confusion.
That's great feedback! While most know us as "the container company", our mission doesn't even talk about containers. We want to help all developers succeed by reducing app complexity. We can certainly do more to help educate folks between the different types of workloads you might be running. We're still very early on in this process, so stay tuned (and keep the feedback coming)!
'Docker + WASM' definitely gave me the impression that docker is now leveraging wasm to be able to run your normal container workloads directly on the browser.
Like if you have any docker container, you can now take that, with some modification and run that directly on the browser. Reading further, I think this is totally not what it actually is.
You could say 'Docker for WASM applications' or 'Docker to deploy WASM apps' that would make the relationship more clear I think.
That’s our experience as well… This Technical Preview is an early downpayment, and we’re definitely looking for feedback on how one may could make the Wasm development experience better!
Great question! In the demo app we showed yesterday (source here - https://github.com/second-state/microservice-rust-mysql), the Dockerfile is leveraging a multi-stage build where the first stage builds the Wasm module and the second stage extracts the Wasm module into a "FROM scratch" image.
In Compose, the "server" service is running using the image that is produced by that build. It's then handed off to the new Wasm runtime, where the module is extracted and executed. Hope that helps! Feel free to follow up with more questions!
I don't really get it either. It's kind of missing which interfaces are actually provided to the WASM script, and how the magic webserver that is described in the docs is working.
Most WASI implementation so far seem to just make a stdin and stdout available for a WASM program - which wouldn't really be enough to run a webserver in it.
Is this example running a non-wasm webserver that then proxies to WASM modules? Is that wasmedge? Or does wasmedge really make syscalls like accept() available to the module?
If it's the latter, how could it actually do something meaningful without threading - given any accept() would block the whole module? Run in non-blocking mode and expect the whole WASM module to be written in async style?
I know these are lots of questions - but they seem rather fundamental to what you could actually use the system for.
> Is this example running a non-wasm webserver that then proxies to WASM modules? Is that wasmedge? Or does wasmedge really make syscalls like accept() available to the module?
> If it's the latter, how could it actually do something meaningful without threading - given any accept() would block the whole module? Run in non-blocking mode and expect the whole WASM module to be written in async style?
Yeah, I'm with you on this. I'm hoping that there is more to come soon that might flesh out this story. (I mean, surely just having an entrypoint into wasmtime + args or runtime of choice) gets these things inline with existing docker goals.
I think this is mostly a land grab to have docker image repositories be the distribution method of choice for wasi.
Except that would've been too easy to understand, and instead seems like they're using wasmedge anyway. Needlessly confusing, why do companies do this?
The missing piece. WASM, specificially WASM run time environments, have the potential to replace containers in the future. If you can target WASM with Go, Rust, Python, C, C++ ... then you don't need containers at all.
BTW, the two WASM runtimes jockeying for position are wasmtime and wasmer. It looks like Docker is getting behind wasmtime.
I'd love to see this future, but I wonder if a WASM VM can match the performance of something like V8 that is fine-tuned to run a single language well.
That’s pretty much the point of WASM, isn’t it? V8 is a billion dollar investment in running as best as possible a language that is very, very difficult to run performantly. WASM instead opens up options to run well languages that are much easier to run performantly.
Definitely it is, and it does open doors for this, I'm just curious if running say, a JavaScript or Python interpreter on top of WASM would be more performant than V8, given that there could be two layers of VMs, ie the WASM VM and the JS/Python VM running on it.
I'd hope that they'd be comparable in performance, especially when it comes to doing client-side web app things in languages other than JavaScript.
Currently JIT in wasm is incomplete. That is a wasm program cannot modify itself (it would need to create new modules and ask the host to load them with a shared memory)
So compiling JS to wasm would probably be limited to the performance of interpreting JS on native
This is great. We’ve been chatting with Docker for a little bit (I’m Syrus, from Wasmer) and it’s great to see that they are making progress on improving the developer experience around WebAssembly.
However, I see Docker (and thus, Kubernetes) is as technologies of the past (I don’t intend to start any flamewar, I really believe this by heart).
Metal compute (Web 1.0) lead to VMs which then lead to Containers (Web 2.0 / Cloud) and now we are seeing what the next wave will look like thanks to WebAssembly.
Of course, there’s ton of profit that has been generated thanks to the added complexity of current technologies and abstractions.
We have now the opportunity to rethink what the future will look like. Long live Wasm
Long live Wasm indeed! Obviously we feel a bit differently about Docker and k8s being in the past — Docker is used by 68% of professional developers according to the latest SO survey[0], and k8s is still growing in popularity at 28%. But obviously the technology landscape changes rapidly, and maybe one day (we hope) Wasm will be at 28%, 68% or higher. We’re frankly just excited about the possibilities, and wanted to help along the way :)
Hi Tim. The stats logic will probably apply as well for gas car makers vs electric ones, or any field to that matter that might be at risk of disruption in a perceivable timeframe (but are not disrupted just yet).
Don’t get me wrong though, I have tons of admiration for Docker (in fact, Solomon Hykes is an investor in Wasmer) and the great ergonomics you introduced along the way to help developers and reach the current status quo. Without you guys probably we would have reached the cloud advancements much later in time. However, we paid those advances with an order of magnitude greater complexity in other layers (with the likes of cloud providers profiting from it).
But now I sincerely believe we need more powerful abstractions for the edge, serverless and Web 3.
In any case, I’m incredibly excited that you are researching more into WebAssembly. That’s great for the ecosystem and also will help to bring more devs onboard. Thanks for all the work!
Don’t forget this tweet from a day earlier, which he quotes:
> “So will wasm replace Docker?” No, but imagine a future where Docker runs linux containers, windows containers and wasm containers side by side. Over time wasm might become the most popular container type. Docker will love them all equally, and run it all :)
Even if you change the unit of computation in k8s from a container, that's only one problem that it solves. What about deployments, services, ingress, configmaps/secrets, jobs, volumes, etc. If you're going to create your own distributed system, all these concepts are going to exist in some form.
Going from coarse-grained compute to fine-grained compute. The finer grain you have, the more "composability" and "flexibility" you get.
At some point you become so fine-grain that all you have are just lambdas floating in the cloud. And universal pointers to data floating in the cloud. Wire them up and everything scales automatically.
Back when I was working on this sort of stuff, there was an intermediate development between containers and wasm, that being of library OS like mirage OS, unikernels... Etc. I think wasm has probably better positioning compared to those unikernels.
Could you explain how WASM replaces containers? Let's say I have a backend API written in Java (spring boot). Containers basically solve two things, building and running the application. What does WASM solve here?
Wasm "solves" running your application in a shared environment. i.e not your machine but say a cloudflare machine, that has to run dozens of applications like yours.
Without wasm, the unit of containerization for them is operating system process. Each process spins up a jvm that runs your app/backend API.
With wasm, the unit of containerization is the exact functions of your backend API. So they can fit more "containers" on a single machine, compared to docker. And that brings faster startups etc..
Nice. I'm excited because there are lots of different ways to run WASM on a server, different ABIs, different runtimes, etc etc. A big move like this might make it easier to standardize around some kind of server-side WASM happy path and also provide pressure to move some of the Wasi feature work along faster.
For serverless usecases, cloudflare etc.. found that it would be faster to just call a function from a wasm binary than spin up a whole docker container [1]. Which basically translated to cost savings.
In the last company I worked for [2], we saw wasm as a way to easily ship user code in a cross platform way. We mostly targeted edge-ml use cases. Wasm allowed us to package all the "libraries"/functions needed to run the user code securely. So users created and tested out their ML apps in browser and deployed them to all the platforms we supported. (Mobile, Browser, Embedded). As an added bonus they could write each of their functions in any language that compiled to wasm, and just ship it as one "app".
This seems like the best reason to use WebAssembly. The benefit is for the system integrator / platform and the value prop is clear vs. any other technology out there
Great question! The promise (and excitement) of Wasm is to have portable, secure, quick-to-start, and low resource usage apps. So, write code in Go, Rust, C++, or any other language that can output to Wasm (up to over 40 now!) and you're good to go! The binaries are super small too. Happy to dive in more too!
Are there realistic benchmarks that quantify these gains?
A first view of the promise reads reminiscent to the JVM days, which promised to solve portability. The main difference is JIT vs. AOT, but hidden behind that remains also the complex management of FS access, threading, spawning, SIMD, GPU, and other unportable differences. While I imagine that the shim would avoid having a whole Linux, what do we lose in the change?
Why not run that go/rust/etc code natively on the machine though? Is there extra sandboxing, network/filesystem virtualization or anything gained by compiling to and running in a WASM environment?
One benefit of using wasm is architecture-agnostic binary. Right now you can't run x86 binary on ARM or vice-versa. So basically you need to build your containers twice if you have macbook people and x86 servers. And technically those are different images, so there's chance that you'll hit some non-trivial difference. With wasm everything could be simpler.
This does not work for me. Java image that takes 30 seconds to build on x86 machine took 40 minutes to build on M1 after which I killed it. So this feature essentially does not exist as it's not usable. I don't think that's how most people use it. I, personally, rent x86 VPS just for docker.
Most images nowadays have ARM version, so probably that's how most people use it.
Yes, we need a compiler for Java. Java sources are compiled to Java bytecode along with many other actions for complex projects like downloading dependencies, generating sources, running tests which might run platform-dependent binaries and so on.
> That was the advantage given for the JVM. The .jar ran on x86 and ARM.
Yes, wasm and Java bytecode are the same in that regard. But Java bytecode failed to get adoption outside of Java world. Wasm might not, we will see. One significant improvement of Wasm over Java is good security story. Java sandbox is well known for its CVEs. Browser wasm implementations are battle-tested on billions of devices in the wild Internet. So you basically can't run untrusted Java bytecode without further boundaries like KVM or at least containers. But you definitely can run untrusted Wasm bytecode because that's what your browser does all the time.
And it is starting an additional emulated x86/amd64 vm that is emulated. The battery takes a big hit. Now imagine you can run wasm that doesn‘t need an extra vm and doesn‘t drain your battery much faster.
I think the # of people losing a lot of of battery to this is miniscule, especially since the OSX x86 emulation is pretty fast.
A bigger peeve is that there is no general reasonably fast cross build ability, to eg build, test and debug ARM images (for runnign on ARM servers) on x86 Linux or Windows etc.
Gained by you running your app on your own server? not much really.
Gained by some "serverless" provider who tries to run multiple apps like yours, in parallel on the same machine? Yes. Less process overhead. Probably, less memory footprint too.
A majority of software already does this. WASM seems more like a direct attempt to run binary blobs and to simply "trust" the intrusive binary sandbox while leaking all kinds of information when trustworthy sandbox solutions and multi architecture compilation are nothing new and not a new problem that needs solving. It seems if anything the problem people are trying to solve with WASM is how to get people enthusiastic about handing over more privacy.
Depends on which "this" you're referring to. We (Docker) are trying to make it easier for developers to use the tools and capabilities they know and love to build, share, and run Wasm applications.
As far as Wasm itself, it's designed to provide a fast, lightweight, secure, and portable binary format. While it was originally designed to help bring native code to the browser, it's quickly spreading to the server side. Many folks are using it for edge/IoT, but it's growing into other areas (saw demos today of even using it in databases as psuedo stored procedures). Happy to dive in more if you have more questions!
I'm honestly really confused about the specific role docker is playing here.
What, exactly, is Docker doing? Is it compiling the application? Is it making a runtime for the wasm binary? Is it being the runtime for the wasm binary, so the end user builds through whatever usual build processes and gets a binary they can then easily run?
If the wasm binary is lightweight and portable, why is docker useful?
edit: Given the other comments about "what problem does this solve" I think maybe the blog post has missed its mark slightly
Yeah. Not trying to be cynical, but this feels like someone at Docker said "we need to have a WASM story" and this is what they came up with. I really don't see the point.
Sorry for the delayed response, but the problem we're trying to help solve is how to help lower the barrier of entry to using Wasm apps leveraging the tool many developers are already using. As an example example, with Docker+Wasm you can use a Dockerfile to build the app in a container image, distribute the Wasm bundle as an OCI artifact, and test it locally using the Wasm runtime. While the ability to build using containers has existed all along, the ability to actually run the Wasm app locally side-by-side your other containers (and leverage the same container network) is new. Hope that helps!
This should be the copy of the announcement. Finally it's clear what it does. The announcement doesn't explain at all what docker + WASM is.
Summary of summary: Dockerfile -> docker build -> docker container with the WASM app and runtime inside -> docker push/pull -> docker run container.
However, even if I never compiled anything to WASM I think that it was already possible to build an image with a WASM container and runtime inside (as for any other language/runtime.) So what's the friction this is removing from the process?
That does help thanks. It's still all a bit abstract to me because I (like many others) are still not familiar with WASM itself so this next step is probably going to go unappreciated for a while.
I'm a bit surprised by several comments in this post talking about how WASM could replace containers but I don't have the context around it
Easy orchestration and deployment is the only thing I can think of. Because of docker's infrastructure.
But tbh if orchestration is really the concern, Docker + wasm seems less efficient than having a dedicated app that can orchestrate multiple wasm modules within the same process. But maybe that's something docker can solve later as the actual requirements emerge.
the only thing I see is that after wrapping wasm inside a docker you can leverage all those docker tools(k8n,etc)?
the docker solves portability and scalability issues with a reasonable overhead, so wasm hides inside docker can benefit from some?
I wonder if it could make testing frontend code that uses WASM (but not DOM) fast and easy since you wouldn’t need to fire up a complete browser environment.
I’m not sure that it’s possible at the moment. In the past when I needed to test a WASM integration, I ended up using that approach and it was kind of a pain not to get immediate feedback on the WASM code’s API tests since it was essentially only testable through the complete integration environment.
I like integration tests, but I like smaller and faster test suites for easing development in some conditions as well.
You can currently test a WASM-targeted API if you use unit tests or other language-level testing approaches, but you won’t get the constraints of the WASM runtime as far as I know. Maybe the lack of garbage collection could be a critical constraint to test against.
I suppose you could even test dynamically linked binaries without a browser as well.
I’m sure there’s far more to it that I’m not aware of, and maybe testing really isn’t that useful of a feature here — I’m just guessing based on my own experience.
Great question! There isn't a way to run Docker directly in the browser. But, there are tools (like Play with Docker at play-with-docker.com) that lets you interact with a CLI in the browser to run commands against a remote cloud instance. I personally use this a lot for demos and workshops!
But... certainly a neat idea to think about what Wasm-based applications could possibly look like/run in the browser!
Hey! Peter from Snaplet here. This is really exciting stuff. We created the OSS postgres-wasm (https://github.com/snaplet/postgres-wasm) example a few weeks ago. An idea I'm playing around with is something like:
Edit.com opens a text-editor and terminal where I have access to the NodeJS binary and a connection string to PostgresQL. Want Redis? Open a new tab at https://redis.com/try, where the connection string will appear in the edit.com tab.
I used https://wasm.supabase.com/ to make sure some SQL commands I was writing for a blog were correct. It was super useful and faster than starting docker desktop, looking for the postgres image name, starting it etc..
I miss a feature where I can share a link with some data/schema pre-seeded (maybe from a gist?)
All three links that you posted appears to be either broken or malicious. Are you just trying to explain a concept using example domain names? Consider ".example" or ".example.com" (see RFC 2606) instead of potentially malicious domains.
Is it possible to sandbox the host system from the guests in WASM?
Are there namespaces and cgroups and SECCOMP and blocking for concurrent hardware access in WASM, or would those kernel protections be effective within a WASM runtime? Do WASM runtimes have subprocess isolation?
- TIL about teh Endokernel: "The Endokernel: Fast, Secure, and Programmable Subprocess Virtualization" (2021)
https://arxiv.org/abs/2108.03705#
> The Endokernel introduces a new virtual machine abstraction for representing subprocess authority, which is enforced by an efficient self-isolating monitor that maps the abstraction to system level objects (processes, threads, files, and signals). We show how the Endokernel can be used to develop specialized separation abstractions using an exokernel-like organization to provide virtual privilege rings, which we use to reorganize and secure NGINX. Our prototype, includes a new syscall monitor, the nexpoline, and explores the tradeoffs of implementing it with diverse mechanisms, including Intel Control Enhancement Technology. Overall, we believe sub-process isolation is a must and that the Endokernel exposes an essential set of abstractions for realizing this in a simple and feasible way.
> gVisor is an application kernel, written in Go, that implements a substantial portion of the Linux system surface. It includes an Open Container Initiative (OCI) runtime called runsc that provides an isolation boundary between the application and the host kernel.
Funnily enough, I believe that's completely achievable with enough time spent on it (probably a few weeks of an engineer working full time).
We have technology like WebVM [1] (from leaningtech / CheerP) or Copy86 [2] that already allows x86 machine code execution/emulation on the web. If you add an OCI client layer on top that is executable in the browser, it should be possible to run Docker containers in the browser.
Sorry I can't reset my docker password to double check, but I think something like this exists [1]. I'm not sure what its limits are, but its a great tool for learning.
I just played with it and it doesn't execute the container in the browser. It executes it remotely and proxies the stdin/stdout and such from/to the browser (the browser is just a client instead of a runner)
I think the biggest challenge here is getting the resources through the browser - e.g. enough memory to run a decent container, or the ability to open a listening port.
My first thought was “what would docker have to do with the client side”… I didn’t realize wasm was running server side these days and apparently with some success?
I wouldn’t have considered running my Java or python on wasm because it’s all server side… that’s just what it does… but the single execution environment with sandboxing is very interesting.
There has been a large amount of progress on wasm GC actually! If you haven't been attending the group meetings though then you might have missed it. In particular:
* The GC MVP spec is mostly done at this point.
* There are working toolchains for multiple languages: Java, Dart, Kotlin, and others have begun work as well.
There are a few things we're excited about, many of which we fully recognize are used in marketing/buzzword bingo. But, they really are around portability (compile to a Wasm module and run on any architecture), isolated sandboxing (leveraging much of the research and investment used to protect web browsers), and super fast startups (obviously depends on the app too).
There are a few limitations right now, some of which lack of built-in garbage collection and multi-threading. You also have to use a language that can completely compile to Wasm, making it hard (sometimes impossible) to leverage already built libraries/binaries.
But... the tech is growing rapidly and we're excited for it! Happy to dive in or answer more questions if you'd like!
One thing I've seen come up a lot is that people got used to treating containers as a portable unit of code, and then Apple Silicon threw a wrench in that assumption. A WASM module is the same regardless of whether you're running it on arm, amd, or anything. (Alas, the cost of this portability is that it's 32-bit)
Wasm = just a bunch of bytecode executing in a VM / a sandbox. Even with JIT, it will still be slower than the native code.
And since it is a sandbox.. either you, or your friendly wasm interpreter (via. wasi etc...) needs to provide the system access APIs for various things like storage/filesystem access, hardware acceleration via. GPU, TPU etc.. And some of the standard APIs we take for granted.
Is WASM typically interpreted? I would have assumed one of the advantages of an assembly language is it would be easy to JIT compile down to machine code.
That depends on the runtime. Some runtimes like wasmtime support both JIT and AOT compilation.
And for places where you can't use JIT compilation (like iOS... - which is where most of the users/developers of our app were) there are interpreters like wasm3.
How does this work with things like WASI--back when I did WASM things I recalled that WASM relies on imported functions for things like Io, etc. so what functions is the Docker runtime providing here?
I am very confused about how wasi calls work in something like wasmer or wasmtime vs a browser. My guess is the former expose a runtime directly to the wasm executable, and in the browser this has to be done via autogenerated js bindings?
I for one love this now this reduces attack area by making my app as run time plus i get to integrate it with existing tooling like kubernetes and my app can scale. It is a win win
I read that press release or whatever a couple of times
and I still can't figure out what the point is.
When I first saw it I was hoping that it would free Docker
from the chains of VMs. (On Windows and Mac)
If images target WASM then you should be able to run on them on any platform
supporting WASM right?
As a rule of thumb, when a company is writing blog posts about how "they see" X as a "complementary technology" to their technology... they are in trouble.
Seriously, if you're in a position to be running native code in your environment why would you chose WASM? If you're building WASM for the web, is it really more expensive to build a pure native target out of the same source? Surely, performance wise you'll always be ahead going native.
Put simply, more powerful isolation than Docker (with explicit, granular managed effects), and a simpler operating model, i.e. no container orchestration. You can actually compile modules from disparate stacks that call into one another in a single runtime.
And for just that give up all the freedom docker runtimes give us? I don’t want to be forced to use go or rust - maybe I’d like to use bash, node or even the jvm depending on the problem I need to solve.
And I don’t understand why docker is supporting this as docker + wasm makes no sense at all as wasm wants to replace docker as the runtime provider …
> And for just that give up all the freedom docker runtimes give us?
Constraints give you freedom because they allow you to make guarantees. Checksums, reliable test suites, and good software abstractions are all examples.
> I don’t want to be forced to use go or rust - maybe I’d like to use bash, node or even the jvm depending on the problem I need to solve.
Apologies in advance for a bit of a rant, but I sincerely hope the tools growing up around WASM significantly diminish the need for Docker. As an industry, we're so bad at isolation that we have to put a computer (container) inside a computer (VM) inside a computer (hardware) just to achieve any sort of sane guarantees.
Then we immediately trade any semblance of sanity for container orchestration.
(And please don't try to convince me that Kubernetes is sane. I operated a production SaaS on it for 4 years, and spent another year at a DevSecOps consultancy with 40+ CKAs—sanity and familiarity are not the same.)
Broadly, we agree. The goal of this Technical Preview is not to encourage Wasm to be mediated by containers in production, but rather to enable people to locally build & package Wasm apps easily. In production, that could look like a number of different scenarios, from "bare metal" edge (just running in a Wasm VM), to running your Wasm workloads in a nomad/k8s cluster if that’s what you need (e.g. if you want hybrid container/wasm orchestration).
Hey Tim, I appreciate your willingness to engage. What I'm getting at is a bit more pointed. In terms of composability, Docker and WASM are opposites: Docker requires a stand-alone runtime, and an orchestration layer, etc., whereas a WASM module & runtime can be embedded directly into my application, into a browser, or into (comparatively) simple, time-tested tools like Apache[0].
So, to your point, the opportunity here is to provide WASM-native alternatives to existing container-specific technologies, that are more composable by design, because they don't have to deal with the complexity of trying to ape an extra operating system just to run some software.
For example, I'd love to see container orchestration be supplanted by something like a la carte Erlang-style service discovery—that's a primitive that could easily be composed with other primitives, and wouldn't result in the combinatorial explosion of nouns we see in systems like k8s / Swarm / others.
No disagreement here! Just to contextualize: Docker — the company as it exists today — is singularly focused on the development experience, i.e. the inner loop of code/test/build, not the outer loop of deploying to production (which is largely controlled by cloud platforms and k8s at this point). I know that separation can seem arbitrary, considering containers are largely successful because they help bridge the two, but that’s just the reality of what we’re focused on in our daily jobs @ Docker.
Within that framework, we see Wasm as extremely compatible with our goals of improving the local development experience, and yes, giving people alternatives to container-centric approaches. I’m personally inclined to agree with the points you are making about opportunities in orchestration, but we’re starting today by just trying to give people to a solid toolset that lets you iterate on your Wasm apps locally, and easily export the resulting artifacts, so you can deploy them as you see fit. In the process we try to be careful about shedding any container-centric assumptions, while porting over some of the wins of the docker tooling that we think can translate well to Wasm (easy local dev environment, standard artifacts, broad platform compatibility across Windows/Linux/M1, etc.) We will happily work with anyone interested in working with us to improve the production/deployment landscape for Wasm, and in fact I would say the main reason that drove us to launch this technical preview today, was to attract feedback on how the Wasm community (ourselves included) could best deliver an alternative path to production for applications going forward.
Hope this makes sense and that I understood your point accurately!
> Hope this makes sense and that I understood your point accurately!
I'm happy to hear all of that and, yes, I believe you did.
> Docker [...] is singularly focused on the development experience, i.e. the inner loop of code/test/build, not the outer loop of deploying to production (which is largely controlled by cloud platforms and k8s at this point).
I would push back on this a bit: although it may not be a focus, and that's totally fair, deployment is undeniably part of the developer experience, particularly within the context that containers were born into (i.e. DevOps values of continuous deployment & no silos). I certainly wouldn't presume to tell you what to do, but I would suggest that Docker is uniquely situated to address developer experience from end to end.
> We will happily work with anyone interested in working with us to improve the production/deployment landscape for Wasm
If you gave us a path out of containers for the end-to-end developer experience, I believe many of us would be eternally grateful. Let me know how I can help.
> As an industry, we're so bad at isolation that we have to put a computer (container) inside a computer (VM) inside a computer (hardware) just to achieve any sort of sane guarantees.
We don't. But shortsighted design, rushing, and herd mentality around "hyped" tech make it a virtual certainty more like this will exist, just in different shapes and sizes.
You can run most stuff on a baremetal box or VM w/ direct install of dependencies. All of this container stuff, sorry to say, is a grift save for some extreme multi-tenant use cases.
