I kept on reading expecting to see motion prediction, multiversion, or similar given the name and focus on games, but no. This is a totally normal database, designed for low latency and with support for WASM stored procedures. You can host your own server or they will rent you one.
Don't get me wrong, this looks very nice. It looks like a solid building block for persistent worlds in multiplayer games. You'll just have to do your lag masking netcode yourself.
I'm Tyler (guy in the video). In BitCraft we currently implement client-side prediction outside of SpacetimeDB.
However, we DO plan to add automatic client-side prediction as a feature for SpacetimeDB in the near-ish future! Because all your server-side logic is in Wasm modules, we plan to run an embedded version of SpacetimeDB to execute the server logic on the client. As long as the server and client agree on the changes to the data we can reconcile the transactions, otherwise we'll rollback. Notably, we can do this with only partial state on the client!
We can also do deterministic simulation if you have total knowledge of the game state with this solution as well.
How would you simulate physics, pathfinding, animation state on the server using this tech? Do you have to ditch your engine (do I have to reimplement what unity/unreal gives me for free?) ?
Cool, jumping in as a noob to multiplayer dev, so currently for BitCraft MMORPG simulation, e.g. to prevent "god-mode flying", would be done outside the server.
Q How is that coordinated with the SpacetimeDB? Is there a penalty, or perhaps a correction applied to the malicious client (and replicated to listening clients) in the case a hacker modifies their movement illegally?
Since SpacetimeDB would be your backend (game client <-> SpacetimeDB) you would need to verify the inputs to your reducers to prevent cheating. Not really different from anything else.
But that's completely different from how Unreal typically does it. Movement is client-side predicted and server authoritative out of the box. How does SpacetimeDB even know the collisions of my level?
If I were to cheat and disable all wall collisions on the client. An Unreal server would roll me back, to all other people it would look like I'm walking into the wall. How do you even get SpacetimeDB to run Unreal's runtime (because movement kind of depends on deterministic results for frames). How does SpacetimeDB get hold of the assets (collisions)?
I am also skeptical of SpacetimeDB for the same reason. There is no game engine helping you do any simulation and that makes development harder. Your (server-side) assets would somehow need to be synced to SpacetimeDB which possibly means you can't use your editor tools to design them anymore.
As a service backend+database the concept is cool and probably useful. But I don't see it being good at being a game server.
Reading/hearing talks from people doing server backends there is many half insane things being done to keep servers and persistence in check, and everyone still a different custom solution.
A common scenarios:
- "Game-server" using "regular" server+DB frameworks, often works but seldom in real-time scenarios.
- Realtime Game-servers (with in-memory state, sometimes supporting rollback,etc) that then has persistence more or less as an afterthought (duplicating state management between client<->server<->persistence)
What spacetime tries to do (and my current experiment) is to merge the server and DB to move persistence _performance_ concerns into the backend process.
We _rightfully_ moved databases out of server processes for easier management and safety in the 90s, but since f.ex. SQLite has been a bit of a resurgence because the usercode was in a managed language and wasn't as likely to cause a corruption issue.
WASM is a great enabler here because it allows the server/DB to sandbox the logic-code even in "unsafe"/realtime languages from the more precarious DB parts of the process so we don't have to risk data-safety even as we move back to co-locating server-state and persistence data within the same process.
I have also experimented in this space but came to the conclusion that it's not good for games. Not necessarily because it's a bad idea but because it's not how games are made. To make a game with something like SpacetimeDB you would need to build a game engine. You'd also need to build out tooling for managing assets for the server.
One thing about SpacetimeDB that I'm not sure about is if reducers can run concurrently or not. I would hope so but it's very possible they don't and that can be a big regression vs. current game servers.
Agreed, going for a custom engine anyhow in my scenario so that's fine.
It seems that they're built to run serially to achieve reproducible simulations so my guess would be no, this is one thing my experiment does differently although formally it might just be a naming difference.
It's not a major issue though I think, the stuff I work on could shard work by area so single areas are single-threaded but world-wise it's parallel (Eve online has done this for ages).
Serial execution makes the most sense from a database perspective because they work off a transaction log. Parallel execution may require separate transaction logs (basically durable queues) to accomplish sharding but that introduces a whole new set of issues because tables can only be mutated from a single transaction log. Code that needs to access multiple suddenly gets a lot more complex. There was actually a post on HN the other day about this: https://news.ycombinator.com/item?id=43661181
It's not meant as a regular SQL database with full ACID semantics (even if that is a interesting stretch target).
Rather the main thing is mainly about coupling storage and sharding decisions with the simulation, since both a sharded simulator and sharded storage system will need logic for sharding and time-versioning, it's silly to duplicate work (or worse have out of sync semantics). In addition cohabiting simulation and storage will lessen latency and other issues w.r.t. to separating them.
Now, it's entirely possible that I'm barking up the wrong tree but considering that many are exploring this design-space means that there is interest and merits even if only the Spacetime and a few others have taken the leap yet.
Prediction can definitively be argued that it belongs in a layer above it, multi-version/rollback/reconciliation should be integrated however and I don't really see any mention of it from a quick skimming. (really need to take a closer look later today).
Things I have never thought as a professional multiplayer game developer:
"Phew! I've just finished coding all the complex lag masking netcode myself, you know what I really need next? An in-memory SQL-like database to store my world state!"
Hey, can you please make your substantive points without shallow dismissals or snark? This is in the site guidelines: https://news.ycombinator.com/newsguidelines.html. Unfortunately you posted a few comments in this thread that have crossed that line.
If you know more than others as a professional multiplayer game developer, that's great, but in that case the thing to do is to share some of what you know, so others can learn. If you don't want to do that, that's fine, but in that case please don't post. Putdowns and swipes only degrade the discussion.
Tom I'm afraid you're misreading the tone of my posts and their character.
I could post anything objectively true in this thread like "the sky is blue", and some idiot will post a one sentence response that in fact it's not. This idiot likes to be the last person getting snark in responding to a completely reasonable statement of fact or opinion by an expert in the space.
What I do differently is that yes, I do reply to the snark with a rebuttal. Is this rebuttal snarky? Potentially, but is it earned? Definitely.
You're moderator so let me speak to you that way. If you don't let people rebut bullshit in this comment section, you'll just end up with a bunch of uninformed discourse, where the final word is made by some idiot, who doesn't even make multiplayer games and everyone ends up thinking the sky is fucking green. This has real world consequence when future game developers try to build games on poorly made technology, which ruins game studios and people's lives.
Then again, it's hacker news, and what do a bunch of wannabe startup guys know about making games in the first place. Unskilled and unaware of it. Moderate away sir!
You can "rebut bullshit" without being aggressive or nasty. If someone else is wrong, it's enough to provide correct information respectfully. Putting others down, or slamming their work in a hostile way, doesn't help and degrades this community badly.
Btw, it's quite common for commenters to underestimate the provocation in their own posts and overestimate the provocation in other people's posts. Those two factors can multiply into quite a skew in perception (https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...). I wonder whether this might be a bit of a factor here, if you really don't see how aggressively your comments were coming across.
In my experience providing detailed information in a response in the comment section on this particular topic (game development and networking) in hacker news comments simply creates more surface area for nitpicking and arguing in response, so I keep to short responses and rebuttals.
In the future, I don't feel that writing anything more here on this topic is a worthwhile use of my time.
Yes yes, everybody should be the Dalai Lama. But I'm not.
Each comment that I post, kind or not, when it's longer creates a larger attack surface area for people to nitpick and respond to. This in turn creates larger and larger posts I need to make in response.
By the end I need to write whole essays in the comment section, nobody wants this, so I just keep short, and curt so people stop responding.
For example:
> Ah you mean like implementing UDP and supporting complex lag masking netcode before we have to?
I could respond with an entire essay about how lag masking netcode and UDP-based protocols (both accepted and very standard best practices in the game industry) informs the design of data structures on the game server and how you store game state in order to support delta-encoding in a cache efficient way.
I could then explain how this most likely completely invalidates the current design of SpacetimeDB which is surely based around TCP synchronization using events when properties change, which doesn't scale when lots of objects change at the same time, and causes hitching when people play over a combination of latency and some packet loss because of head of line blocking.
But it's just so above the heads of the people reading this comment section, that it will spin off stupid responses like "Hurrrrr. But doesn't World of Warcraft use TCP?".
It's much easier to just realize that the game they're working on will most likely fail (because 99% of multiplayer games released these days do fail, if you haven't noticed, the game industry is in serious crisis), and if the game has no players, does it really matter if it's networked using UDP or TCP? End of discussion.
ps. The last game team that argued UDP vs. TCP with me was the team behind Stormgate backed by Hathora game server hosting. In fact, Hathora even spun up a ridiculous architecture astronaut architecture for this game based around TCP because they didn't know any better.
This team was adamant that they didn't need to use UDP, because they had implemented rollback netcode in an RTS, and TCP would be just fine with rollback netcode and edge termination with AWS global accelerator. This is objectively wrong, but it ended up not mattering, because their average CCU these days on steam is around 10.
Yes yes, everybody should be the Dalai Lama. But I'm not.
You're just being asked not to be a jerk to people on a messageboard. It's the same thing being asked of everyone else so that we can have a decent messageboard and it's not difficult.
Just want to chime in that although this comment is a bit harsh, it does hold a lot of truth. As a game netcode engineer, most of my time is spent on latency hiding techniques and its consequences (for example, we use rollback netcode and lots of efforts are spent on minimizing visual/sound glitches when mispredict, rollback and reconcile). There are lots of middleware that help with this (eg. Photon, SnapNet, etc) but in general there's no silver bullet - it's highly gameplay specific. Even for the same game, the solution can vary depending on different trade-offs (i.e. budget, correctness/anticheat).
As to how to store the game state in memory, it's usually not something that needs much thinking: it's simply done the same way (or similar) as the game's client code. After all, netcode is mostly about efficient state replication and this saves CPU time when replicating it across clients - and gives more CPU time for minimizing bandwidth (like delta encoding, quantization, etc). If you want, you can utilize some techniques like ECS to optimize for CPU cache locality, but it affect gameplay code a lot and would need to get the whole team onboard.
Also, I just noticed the username "gafferongames" - Glenn's blog[1] has been a must-read for netcode engineers. It helped me a lot when I started working on netcode in the 2010s
Well, I'm not a professional gamedev, nor did I ever do much multiplayer code, but:
1) AFAIR, the ECS patterns family[0] originally came from MMORPG world, and boiled down to designing your game state to work well with relational databases;
2) Around 6 years ago, while toying with yet another (see [0]) take on ECS for a simple game, and wondering how to query stuff efficiently, it dawned on me that the code I'm writing is basically implementing spacial indices and other such concepts from relational DB world. At that point, I thought to myself, "why continue wasting time Greenspunning[1] a database for the internal state, when I could use a real one?", and followed by "you know what I really need next? An in-memory SQLite database to store all my runtime game state!".
I didn't get very far with my experiment due to constraints of adult life, but I did prove that I can build a basic Roguelike skeleton (2D tile map, entities, walls, collisions) while storing all state in SQLite, and have it run at 60 FPS with half of the frame time to spare for other things, on moderately powerful machine (as of 6 years ago), all without too much optimization (Common Lisp with FFI bindings to libsqlite3 + a simple RLU cache in front of queries).
So it's not at all a stupid idea, IMO :). I would test it further if I had the time.
--
[0] - The term means different things to different people. I've seen at least three distinct gamedev design patterns claiming the name ECS for themselves, all conceptually incompatible with each other. More, if you also consider "whatever Unity/Unreal/Godot/... calls ECS".
We can and plan to handle the complex lag masking netcode as well. We've just focused so far on games that don't need very sophisticated lag compensation
TODO does not imply no plan or that we haven't thought deeply about the problem. Prioritization is imperative when you're building an MMORPG and a new database with a small team for both.
How can you think deeply and plan for something when you don't know what don't know. You can only really understand these things when you've taken a game using these latency hiding techniques into production, and then after this, you'll probably realize that some of your original ideas were wrong. Where does this leave SpacetimeDB then?
This company focused all it's effort in reinventing the part of multi-player games that did not need reinventing.
40 engineers for 5 years. I'm guessing at least 20 million dollars spent. 1 million trailer views... Let's say 100k wishlists, 10,000 sales at 30 LTV. 300k in revenue
This company will need to pivot if the game is not successful.
I would say the db could be pivoted for multi-player Figma style apps rather than games, but the coupling of server and db means you can't scale the db layer the same way you could do with literally every other db solution available s your company scales
I mean, they said they'd like to get your expert input on the matter. Why not give them a chance? Maybe you can help them do some much needed course correction—which is what I understood you're staying by reading a bit between the lines.
Maybe you can even negotiate a consultancy fee with them
You can stay sceptical, but try giving them a chance. You might end up surprising eachother!
Aside: I'm getting into indie game dev recently and love your site!
I mean, they are dogfooding their own database. If they’re making a MMO game with it presumably the game has latency masking netcode, even if it isn’t in part of the database.
"Tyler was a Sr. Data Science Engineer at MZ (the company behind Game of War and Mobile Strike). Prior to working at MZ he was an engineer at Apple. Tyler holds a Masters in Distributed Systems and Machine Learning from Johns Hopkins University."
"Alessandro was a Sr. Software Engineer at Bloomberg, LP where he worked on high throughput distributed pricing systems. He holds a Computer Science and a Biomedical Engineering degree from Johns Hopkins University."
Looks like they've built stuff before.
I don't like this database product because it lacks SDK:s in languages I would touch without getting paid, but I think the idea is sound and rather attractive. For many years I've used Picolisp in a similar way, it bundles an object database with a logic engine and a small Lisp-like programming language, which makes it easy to throw in some data and relations and slap on a web page with standard library widgets.
The BEAM is similar, you push some modules into it and store stuff in ETS and don't bother with RDBMS until you need it. If your data is small or trivially shardable you can get persistence too, with Mnesia.
I'm not so sure they'll be making a lot from that game of theirs, but if their database turns out to be robust it could become competitive with things like Supabase/Firebase and such, because there will be modules for serving web and auth and dashboards and whatnot you can just push in there and you're good to go. One-click-install for applications, similar to cPanel and Wordpress rigs, but less messy on the ops side.
Game of War and Mobile Strike look like games to me. Casino-like games for handheld devices rather than God of War, but still games, and likely quite latency sensitive due to the gambling-reminiscent aspects. It also seems like that company had the same idea you have, that selling dedicated networks for latency sensitive systems might be profitable.
If you compare the commit messages on your public repos with those on the SpacetimeDB repo, do you notice any differences?
Seems too biased and harsh. The best lag masking and clientside prediction systems I've experienced as a player are those that are closely integrated with the idiosyncratic gameplay mechanics. I would never expect SpacetimeDB to have a one-size-fits-all solution to this, nor for it to be the central focus of their product.
You will implement terrible lag masking and clientside prediction if you aren't looking at it from the UX perspective, because that is literally the only reason these features exist, to trick players into thinking the game signals are moving faster and with greater frequency through the network than is physically possible. Perhaps you would make better games if you considered the user perspective.
I kept on reading expecting to see motion prediction, multiversion, or similar given the name and focus on games, but no. This is a totally normal database, designed for low latency and with support for WASM stored procedures. You can host your own server or they will rent you one. Don't get me wrong, this looks very nice. It looks like a solid building block for persistent worlds in multiplayer games. You'll just have to do your lag masking netcode yourself.
https://news.ycombinator.com/item?id=43593913