Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

There really isn't enough emphasis on the downsides of server side platforms.

So many of these are only deployed in US and so if you're say in country Australia not only do you have all your traffic going to the US but it will be via slow and intermittent cellular connections.

It makes using services like LLMs unusably slow.

I miss the 90s and having applications and data reside locally.



Unusably slow? It's like 0.3 seconds to first token and then pretty much all of the tokens can follow within a second.

I find it hard to understand the edge usecase for text-based models.


Even in Australia is the LLM lag to a server noticable?

Generally an LLM seems to take about 3s or more to respond, and the network delay to the US is a couple of hundred milliseconds.

The network delay seems minimal compared to the actual delay of the LLM.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: