> but it's hard to see situations where the local approach beats out the cloud approach.
I think the most glaring situation where this is true is simply one of trust and privacy.
Cloud solutions involve trusting 3rd parties with data. Sometimes that fine, sometimes it's really not.
Personally - LLMs start to feel more like they're sitting in the confidant/peer space in many ways. I behave differently when I know I'm hitting a remote resource for LLMs in the same way that I behave differently when I know I'm on camera in person: Less genuinely.
And beyond merely trusting that a company won't abuse or leak my data, there are other trust issues as well. If I use an LLM as a digital assistant - I need to know that it's looking out for me (or at least acting neutrally) and not being influenced by a 3rd party to give me responses that are weighted to benefit that 3rd party.
I don't think it'll be too long before we see someone try to create an LLM that has advertising baked into it, and we have very little insight into how weights are generated and used. If I'm hitting a remote resource - the model I'm actually running can change out from underneath me at any time, jarring at best and utterly unacceptable at worst.
From my end - I'd rather pay and run it locally, even if it's slower or more expensive.
People have trusted search engines with their most intimate questions for nearly 30 years and there has been what ... one? ... leak of query data during this time, and that was from AOL back when people didn't realize that you could sometimes de-anonymize anonymized datasets. It hasn't happened since.
LLMs will require more than privacy to move locally. Latency, flexibility and cost seem more likely drivers.
You're still focused on trusting that my data is safe. And while I think that matters - I don't really think that's the trust I care most about.
I care more about the trust I have to place in the response from the model.
Hell - since you mentioned search... Just look at the backlash right now happening to google. They've sold out search (a while back, really) and people hate it. Ads used to be clearly delimited from search results, and the top results used to be organic instead of paid promos. At some point, that stopped being true.
At least with google search I could still tell that it was showing me ads. You won't have any fucking clue that OpenAI has entered into a partnering agreement with "company [whatever]" and has retrained the model that users on plans x/y/z interact with to make it more likely to push them towards their new partner [whatever]'s products when prompted with certain relevant contexts.
> Hell - since you mentioned search... Just look at the backlash right now happening to google. They've sold out search (a while back, really) and people hate it. Ads used to be clearly delimited from search results, and the top results used to be organic instead of paid promos. At some point, that stopped being true.
Only people in HN-like communities care about this stuff. Most people find the SEO spam in their results more annoying.
> At least with google search I could still tell that it was showing me ads. You won't have any fucking clue that OpenAI has entered into a partnering agreement with "company [whatever]" and has retrained the model that users on plans x/y/z interact with to make it more likely to push them towards their new partner [whatever]'s products when prompted with certain relevant contexts.
> You won't know this for any local models either.
But you will know the model hasn't changed, and you can always continue using the version you currently have.
> Most people find the SEO spam in their results more annoying.
This is the same problem. These models will degrade from research quality to mass market quality as there's incentive to change what results they surface. Whether that's intentional (paid ads) versus adversarial (SEO) doesn't matter all that much - In either case the goals will become commercial and profit motivated.
People really don't like "commercial and profit motivated" in the spaces that some of these LLMs stepping into. Just like you don't like SEO in your recipe results.
> But you will know the model hasn't changed, and you can always continue using the version you currently have.
Will you? What happens when an OS update silently changes the model? Again this is one of those things only HN-types really care/rant about. I've never met a non-technical person care about regular updates beyond being slow or breaking an existing workflow. Most technical folks I know don't care either.
> This is the same problem. These models will degrade from research quality to mass market quality as there's incentive to change what results they surface. Whether that's intentional (paid ads) versus adversarial (SEO) doesn't matter all that much - In either case the goals will become commercial and profit motivated.
Not at all. Search providers have an incentive to fight adversarial actors. They don't have any incentive to fight intentional collaboration.
> People really don't like "commercial and profit motivated" in the spaces that some of these LLMs stepping into. Just like you don't like SEO in your recipe results.
I disagree. When a new, local business pops up and pays for search ads, is this "commercial and profit motivated?" How about advertising a new community space opening? I work with a couple businesses like this (not for SEO, just because I like the space they're in and know the staff) and using ads for outreach is a pretty core part of their strategy. There's no neat and clean definition of "commercial and profit motivated" out there.
Two issues though: leak of data from one party to another, and misuse of data by the party you gave it to. Most big companies don’t leak this type of data, but they sure as hell misuse it and have the fines to prove it.
Almost everyone is willing to trust 3rd parties with data, including enterprise and government customers. I find it hard to believe that there are enough people willing to pay a large premium to run these locally to make it worth the R&D cost.
Having done a lot of Bank/Gov related work... I can tell you this
> Almost everyone is willing to trust 3rd parties with data, including enterprise and government customers.
Is absolutely not true. In it's most basic sense - sure... some data is trusted to some 3rd parties. Usually it's not the data that would be most useful for these models to work with.
We're already getting tons of "don't put our code into chatGPT/Copilot" warnings across tech companies - I can't imagine not getting fired if I throw private financial docs for my company in there, or ask it for summaries of our high level product strategy documents.
Yes, just like you might get fired for transacting sensitive company business on a personal gmail account, even if that company uses enterprise gmail.
Saying that cloud models will win over local models is not the same as saying it will be a free-for-all where workers can just use whatever cloud offering they want. It will take time to enterprisify cloud LLM offerings to satisfy business/government data security needs, but I'm sure it will happen.
I think the most glaring situation where this is true is simply one of trust and privacy.
Cloud solutions involve trusting 3rd parties with data. Sometimes that fine, sometimes it's really not.
Personally - LLMs start to feel more like they're sitting in the confidant/peer space in many ways. I behave differently when I know I'm hitting a remote resource for LLMs in the same way that I behave differently when I know I'm on camera in person: Less genuinely.
And beyond merely trusting that a company won't abuse or leak my data, there are other trust issues as well. If I use an LLM as a digital assistant - I need to know that it's looking out for me (or at least acting neutrally) and not being influenced by a 3rd party to give me responses that are weighted to benefit that 3rd party.
I don't think it'll be too long before we see someone try to create an LLM that has advertising baked into it, and we have very little insight into how weights are generated and used. If I'm hitting a remote resource - the model I'm actually running can change out from underneath me at any time, jarring at best and utterly unacceptable at worst.
From my end - I'd rather pay and run it locally, even if it's slower or more expensive.