Hacker News new | past | comments | ask | show | jobs | submit login

This reminds me of a very particular problem that we tried to solve when I worked at VKontakte. It was about instant messaging and flaky mobile data connections.

The problem: you're on a subway train and you send a message as it departs a station. The request does get to the server, but by the time the response arrives, the train is already in the tunnel and you don't have a signal any more. So the client thinks that the message failed to send, but it was, in fact, sent successfully. The client would retry when it's back online, and would send another copy of that message.

The solution was to send a client-generated "random ID" with each request. I much later learned that this is conventionally called an "idempotency token". This worked, except there was now another problem: you sometimes receive your own message over the long-polling thing before the response to the request that sent it. You don't know for sure whether it's the message you just sent, or something else sent by a different client on the same account, because you don't know the ID of your message yet. This was solved by me delaying the processing of outgoing messages on the client side until all outstanding messages are fully sent and their IDs are known.

Telegram solved this much more elegantly: when the client reconnects to the server, the server sends it all the responses that were not acknowledged during the previous connection. MTProto has its own acknowledgement mechanism in addition to TCP's.

So yeah, instant messaging seems trivial at the first glance, but it turns out that TCP is a leaky enough abstraction that you need to somehow plug those leaks at the application level.




I had to deal with the second problem in a file synchronization app. The solution was to propagate a “device id” through the request and poll/push, so the originating device could ignore changes that it originated.


Honest question: Why didn't you send the message's idempotency token back to the client? Then you're playing the familiar local database locking game.


You can't imagine how desperately I asked for that. Backend guys came up with all sorts of excuses to not do it. As far as I know, they did do it sometime after I quit.


> because you don't know the ID of your message yet

Wouldn't the client know the ID if they randomly generated it? (honest question)


I meant the permanent unique ID assigned by the server.


Ive learned that just about nothing is trivial when it comes to distributed systems




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: