This is what GraphQL was designed for. Only select fields you really need. We've built an OSS Gateway that turns a collection of GraphQL queries into an MCP server to make this simple: https://wundergraph.com/mcp-gateway
The way we've solved this in our MCP gateway (OSS) is that the user first needs to authenticate against our gateway, e.g. by creating a valid JWT with their identity provider, which will be validated using JWKS. Now when they use a tool, they must send their JWT, so the LLM always acts in their behalf. This supports multiple tenants out of the box. (https://wundergraph.com/mcp-gateway)
You can often fool yourself by using sync.Pool. pprof looks great because no allocs in benchmarks but memory usage goes through the roof. It's important to measure real world benefits, if any, and not just synthetic benchmarks.
Let's say you have constantly 1k requests per second and for each request, you need one buffer, each 1 MiB. That means you have 1 GiB in the pool. Without a pool, there's a high likelihood that you're using less. Why? Because in reality, most requests need a 1 MiB buffer but SOME require a 5 MiB buffer. As such, your pool grows over time as you don't have control over the distribution of the size of the pool items.
So, if you have predictable object sizes, the pool will stay flat. If the workloads are random, you have a new problem because, like in this scenario, your pool grows 5x more.
You can solve this problem. E.g. you can only give back items into the pool that are small enough. Alternatively, you could have a small pool and a big pool, but now you're playing cat and mouse.
In such a scenario, it could also work to simply allocate and use GC to clean up. Then you don't have to worry about memory and the lifetime of objects, which makes your code much simpler to read and reason about.
Long before sync.Pool was a thing, I wrote a pool for []bytes: https://github.com/thejerf/gomempool I haven't taken it down because it isn't obsoleted by sync.Pool because the pool is aware of the size of the []bytes. Though it may be somewhat obsoleted by the fact the GC has gotten a lot better since I wrote it, somewhere in the 1.3 time frame. But it solve exactly that problem I had; relatively infrequent messages from the computer's point of view (e.g., a system that is probably getting messages every 50ms or so), but that had to be pulled into buffers completely to process, and had highly irregular sizes. The GC was doing a ton of work when I was allocating them all the time but it was easy to reuse buffers in my situation.
I don't get it. The pool uses weak pointers under the hood right? If you allocate too much up front, the stuff you don't need will get garbage collected. It's no worse than doing the same without a pool, right?
What the top commenter probably failed to mention, and jensneuse tried to explain is that sync.Pool makes an assumption that the size cost of pooled items are similar. If you are pooling buffers (eg: []byte) or any other type with backing memory which during use can/will grow beyond their initial capacity, can lead to a scenario where backing arrays which have grown to MB capacities are returned by the pool to be used for a few KB, and the KB buffers are returned to high memory jobs which in turn grow the backing arrays to MB and return to the pool.
If that's the case, it's usually better to have non-global pools, pool ranges, drop things after a certain capacity, etc.:
What do you mean? Pool content can't be GCed , because there are references to it: pool itself.
What people do is what this article suggested, pool.Get/pool.Put, which makes it only grow in size even if load profile changes. App literally accumulated now unwanted garbage in pool and no app I have seen made and attempt to GC it.
sync.Pool uses weak references for this purpose. The pool does delay GC, and if your pooled objects have pointers, those are real and can be a problem. If your app never decreases the pool size, you've probably reached a stable equilibrium with usage, or your usage fits a pattern that GC has trouble with. If Go truly cannot GC your pooled objects, you probably have a memory leak. E.g. if you have Nodes in a graph with pointers to each other in the pool, and some root pointer to anything in the pool, that's a memory leak
It's rather something that sits between WebAssembly and containers, combining the sandboxing guarantees of the former with the performance of the latter. From a security perspective, the composition is also really good (WebAssembly enforces memory limits, but doesn't have memory protection, NULL pointers are writable, etc. and this is solved here). But unlike WebAssembly, it is Linux-only. So, not something that can run in Web browsers.
gRPC is widely understood as an API tool for Microservices.
Microservices solve an organizational problem, not a technical one.
Ironically, gRPC doesn't really help to solve the organizational problem.
However, GraphQL in combination with Federation, also known as GraphQL Federation actually DOES help organizations to scale APIs across teams and services.
So, even though the typical popular opinion suggests that gRPC is better for Microservices than GraphQL, the reality looks different.
There were times when it was popular to generate a GraphQL API from your database but that's not how the query language is used today.
GraphQL is a query language to implement query style APIs.
These days, it's most widely used as a "Federation" layer to expose a single query-able graph on top of a (micro-) service architecture.
I absolutely love Postgres, but please allow me to say that you absolutely don't want to expose an API generated from a database to people outside of your team. This limits you a lot in changing the way you store your data.
What exactly is the problem with tight coupling? You're going to insert an entire layer that basically translates format A to format B, just so you can later change a column name in the database and not have to change it in the API or something?
1. You don’t want or need to expose lots of implementation details. Many of your data structures should be private, and many should be partly private.
2. Your data structures should not dictate the shape of your api, usage patterns should (e.g. the user always needs records a,b,c together or they have a and want c but don’t care about b)
3. It stops you changing any implementation details later and/or means any change is definitely a breaking change for someone.
There's a few issues; one is that if you have the DB do everything, all of your business logic lives there too, instead of just the data. This is still fine if you have a single use case, but what if in addition to your main application, you also need to use it for things like BI, customer service, analytics / predictions, etc? It then quickly becomes better to use it as a datastore and have another layer decide what to do with it.
And in 30 odd years, everything will be different again, but your company's main data store will not have moved as fast.
Normalization is one of those typical issues where you might be fine with having everything normalized when you start off, but then once performance gets bad you end up denormalizing tables that are typically joined.
The extremely obvious problem is that how you store data is an implementation detail and those change when requirements (or the market) evolve. I'll give you an API and will make triple sure it's as fast as a machine can even serve it and you let me worry about how it's all stored.
To additionally answer you with an analogy: when you have a problem with a company, you call the call center, not Jenny from accounting in particular. Jenny might have helped you twice or thrice but she might leave the company next year and now you have no idea how to solve your problem. Have call centers to dispatch your requests wherever it's applicable in the given day and leave Jenny alone.
> * What exactly is the problem with tight coupling?*
As Joel Spolsky put it: ”the cost of software is the cost of its coupling”.
More specifically the cost of making changes when “if I change this thing I have to change that thing”. But if there’s no attention paid to coupling, then it’s not just the two things you gave to change, but “if I change this thing I have to change those 40 things”.
Looks cool and I understand what it does. Can you explain the production use case for this solution? It's not clear to me how this would be used in real life.
Not OP and not at all related to the project, but I imagine in production you would just replace the container runtime of your Kubernetes/Nomad/Swarm/homemade orchestrator, and run Docker container as usual (but each one is a separate VM). The advantages would be that you have actual isolation between the containers, so it's great for higher security contexts (e.g. banks) or multi tenant setups.
That's about right. RunCVM should be compatible with orchestrators today that use the Docker command line or API. As an experimental project, Kubernetes compatibility is beyond our current scope but it would be an interesting exercise.
