My biggest pet peeve with all these articles on local AI is the only thing they talk about is tokens per second. No one mentions the quality of the answers. No one. I don't mind waiting a little longer if the quality is better. Quickly serving me slop doesn't make it more useful. Are people really only looking at tokens per second?
The model already has its own quality benchmarks elsewhere. The article is just about running the model on X hardware, so the remaining question is then how fast it is. Or does the output quality somehow depend on the hardware too?
Local model as such will give you "autocomplete on steroids" but it is not going to run away and implement cross project feature like frontier model in let's say Cursor.
So there is no value in testing quality of answers, but there is value in testing token speed.
Is autocomplete using LLMs really useful? Even with frontier models I found it to be about 50% right, I turned it of and prefer to use IntelliJ built-in, it is way more reliable.
For me local models is all about quality, and how to achieve that - e.g. by providing guardrails that test the job done.
That's fair. There are even many dimensions to define 'quality' which include use case (coding? writing? multimedia?) and prompt. I suppose if you ask testers to provide benchmarks with their analysis, that might hamper their desire to share.
My aunt had the same disease you mention and was on medications since the 90s. She lead a healthy life with no real side effects from her medication and she passed away last year in her 80s. To be perfectly honest, she did die of the disease, because her medication stopped working and her bone marrow was all scarred. But up until a year before she passed away she was very active and healthy. Once the medication stopped working, she went steadily downhill until she passed away.
Hopefully you get great progress on your research but I just wanted to reassure you that the name sounds scary but the current treatment appears to work well and hopefully gives you enough runway to find your cure.
> I just wanted to reassure you that the name sounds scary but the current treatment appears to work well
Even a charitable read is condescending. This person just wrote they are in the innermost research circles. I think they are beyond the "scary name" - needs reassurance phase.
The insane level of flopping with no dignity or shame, and the insane level of allowing this to happen without any penalties is one of the biggest reasons why I don’t watch soccer. Those in charge WANT soccer players to flop but I don’t understand why. It’s dishonorable and weak but the sport does nothing to stop it.
Another reason is that the best American athletes will go to the sport that pays the most and soccer is on the bottom of that list.
If all you see in soccer is fake flopping, that means you do not really appreciate nor understand the game, you do not understand what hard and what's easy, the incredible acrobatic feats the players do, the intricate positional maneuvers, etc
The fake flopping happens sometimes, but overall it hardly detracts from the game. It would be like me saying false starts is why I don't watch the 100 m dash.
And I'd be wary of thinking a fall is fake when the referee and the linesmen who are actually on the field think otherwise. Note that soccer is mostly not supposed to be a physical contact game. It was much more like that up to the 1070s. In fact, the infamous and relentless fouling of Pelé in the 1966 World Cup was a major catalyst for the creation of the red and yellow card system.
Gamesmanship in soccer is not more egregious than in say basketball. If you look at the nonsense ronnie coleman did in his day you wouldn't be complaining about soccer players
ZIRP created a level of absurd wealth such that the ultra wealthy can buy large swathes of things that they never could before, and they’re doing it. And societal norms and laws can’t keep up with it to protect us from them.
Now they are buying fire stations, dentist offices, ski resorts, whatever the fuck they can think of and then raise the prices. Something needs to be done to stop this.
Another bury-your-head-in-the-sand article. It sucks but AI replacing coders is very real. I haven’t coded since last year but I’ve pushed more features at a rate faster than I ever could in my entire career. I pushed new code while chatting on WhatsApp with my friends yesterday using 2 prompts.
“Remove this old feature.”
“Are you sure you didn’t break anything?”
That was it. Then I manually tested it to make sure nothing was broken. Then I did a brief review before posting it for code review and then pushed it. What would have taken me probably about 1 day to go through and figure out code changes and then actually change the code took me about 30 seconds.
To think that we need to maintain the post-Covid hiring bloat is nonsense. I’m not so arrogant to think that someone with an llm can’t replace me, if I survive a few more years in this industry I’ll be amazed and grateful.
They’re not competing in the same domain - if you look at their business model it actually is much closer to ML consulting for companies (CMA CGM, ASML, Airbus…). The big three are trying to capture B2C mainly while Mistral is full focused B2B
I don't know about France, but here in Germany I think there's more room for B2B than B2C because of the desire for stability, itself leading to bureaucracy in everyday life that sets expectations for a much slower everything (like, buying a house took 7 months, many contracts have 3 month notice periods).
But that's just my best guess, and I'm saying this as one who migrated here rather than growing up here. I've also actually noticed the literal anarchists here, whereas the ones in the UK I only knew once they told me, before anyone makes a planet-of-hats kind of mistake on this.
Another aspect is that selling to all of Europe as a B2C business is hard. Until recently you ended up having to register for VAT all over the place, god only knows how many different specificities, bureaus, and rules, with most payment solutions not helping you in any way, and most accountants (in my experience) being at best unable to help you in any significant fashion, at worst being very confident in their ability to help you.
It is done, but by very few. And the EU has made progress on uniformizing and simplifying it, but it seems to have done more progress on the B2B side than on B2C.
While the US is a much bigger consumer market than any single EU country, with significant differences in disposable income and spending power. 18% to 20% of full-time workers in the US make over $100.000. That's nearly half the entire population of France. A third that of Germany.
And even if there are differences and administrative hurdles when selling across US states borders, that road has mostly (or seems to have, from here) been paved.
It brings more money. Europe in general has much more industry (hence China shock might be bigger) so Industrial AI makes more sense. And also the amount of Consumers needing industrial consulting is not high.
An hypothesis from my own consulting: firing an employee can be somewhat difficult, and their salary has quite some taxes on top. At my previous startup we’d rule-of-thumb calculate someone as costing us 140% of their pre-tax salary.
Put these two together and a freelancer becomes an interesting proposition for some tasks. In Germany I could bill roughly 2.5x my after-tax salary. You also incur no other costs such as equipment and illness-as—a-loss. And… you’re really easy to fire when COVID comes around.
I check almost every mistral model. But to be honest after agentic coding has become a thing, I only test their models in a chat app. I tried their coding TUI a few months back and found that it was way behind. It didn't even support skills back then. I wish they go strong just because they are a big AI player in Europe and I have personal connection to France.
mistral was never competitive and is getting less so, but that doesn't matter they cant be allowed to fail and have a long time to find their lane. They're smart and have an audience of like 600m people and the largest governments by spending who would use them if they were good enough.
I would speculate that the purpose is not just to have a domestically owned model that’s as good as the Chinese one, but for the investment to be used to build a domestic industry of people that know how to do it themselves, so as not to rely on foreign parties, so it doesn’t have to be better, just has to be good enough.
If you're a European AI lab and you see you can't compete with the Amerikans on compute or spending, and the Chinese are open sourcing their LLM work and keeping the Amerikans honest, then there's no real need for you to focus on LLMs
This is why Elon’s appealing. I thought they surely had the talent to be considered a fourth at this point. (Oh someone mentioned they’re politically unpopular at work.)
No they really don't because Grok is not a competitor. The big 3 are the big 3 because they have historically traded the top place for model intelligence. Grok has never crested that high. People like to think Grok isnt as popular because of elon and politics, but if Grok was the best coding model, nobody would give a fuck. Google has also not led since gemini 2.5 pro. It's really the big 2 at this point.
I'm statisfied by vibe coding-wise, but i found their TUI UX to be abysmal (not that CC is great, but definitely better). I'm talking to the extent of some characters from the numpad not being typable in the prompt. Have you had a similar experience?
I don't use the numpad characters, but I have tried Vibe, Goose (GUI and CLI), Dirac and the built in agent in Zed for vibe coding. I keep coming back to Mistral's Vibe. I actually find the ergonomics of it nicer than the others I've used so far. I really wanted to like Goose, and their GUI offering is OK for chat, but I thought their CLI was poor. Dirac was OK and I should try it again to be fair. Zed was just overkill and complex for what I needed. Vibe CLI seems to hit the sweet spot, although it's not perfect. The challenges I encounter are mostly down to API errors though and sometime bash tooling. I could configure it better for that, if I took the time (which I should).
The are in Europe and in France. The worst possible situation to be successful as a startup. I mean the extreme hiarchy, lack of salaries and taxation is not really gonna help.
And they obviously don't hire the right people. Reasons can be many-fold. One possible explanation is that there's not many talent left on the market, and most have been already picked up by other AI labs paying more $$$ while offering more exciting work and more exciting trajectory at the same time. Another possible explanation is that there is enough talent on the market left but their recruitment process doesn't allow them to recognize those people, hence it is broken. When I look into their job postings, I tend to give higher chances for the latter.
It's probably never going to happen because neither party cares about protecting Americans rights, but we need to have some sort of law that creates a Chinese firewall between these mass surveillance data and the government, or whoever else.
I don't know if you could ever collect this data and never have foreign entities or NSA moles infiltrate into it by sending their agents to work at that company and steal the data whenever they want. But I can see how this would be good at fighting crime but also a completely and absolute destruction of privacy.
We need politicians that actually care about Americans and their rights but no one who cares is dumb enough to want to go into politics, which is the sad thing.
"The third-party doctrine is a United States legal doctrine that holds that people who voluntarily give information to third parties, such as banks and phone companies, generally have "no reasonable expectation of privacy" in that information."
The fact that has been a routine practice is egregious, but the bigger insult is the fact that this loophole has been known for quite some time. Yet, our legislators and judiciary have allowed the practices to continue. There’s nothing but foxes in the henhouse.
Just don't collect the data. If it's too dangerous for the government to have then private companies shouldn't have it either. The entire purpose of license plate readers is to assist law enforcement; if we decide as a society that we don't want to do it then just ban it completely.
> The entire purpose of license plate readers is to assist law enforcement
It was the repossession companies that deployed them first. The police, as a general rule, are about 10 years behind on technology almost everywhere, so when new stuff drops, it's actually profit driven industries that deploy it.
Our company cut deals with several large business in the area, like malls, and we deployed the cameras at the entrances to their lots. If a car on the "hot list" pulled in, we'd get an alert, then dispatch a truck to go collect the vehicle.
It’d be hard to keep individuals from doing this. But individuals aren’t running networks of cameras. Companies are. Those companies probably couldn’t fly under the radar selling LPR data if the practice was banned.
Most individuals I know only have access to a handful of places they could put ALPR devices. Their home, their work, a couple of really really close friends who trust them enough to let them setup a camera on their property. Individuals could pay people to host their cameras but then that starts looking like a business, so while it's theoretically possible for an individual to have a network of APLRs just for the fun of it, that just seems like weird enough hobby that, I don't have any evidence of this, but I don't think anyone is doing that.
You can ban the commercialization and mass scaling of the technology. Just because you can't prevent something at a small scale doesn't mean you can't prevent corporations and government agencies from doing it without exposing themselves to unacceptable legal risk.
There is little chance I could just post up cameras wherever my ex travels and note all the time she arrives and leaves at all intersections and get away with that without at least a restraining order ordering me to stop. What they are actually doing is stalking by method of a network of cameras deliberately installed to follow people from place to place. It isn't generalized observation in pursuit of speech, it arguably isn't even speech, but rather mass individualized stalking. Maybe 1A allows that but that doesn't seem to be the law on the books for anyone else trying to stalk people in such a way.
Personally I don't have a huge problem with 1A being broad enough to including recording literally everything in public and meticulously cataloging and following everyone, but only if the rest of the amendments are read in the same broad and literal manner. Meaning I can own nukes, I don't have to display a plate, the 10th amendment would stop the feds from outlawing intrastate weed, etc. What it looks like what happens is the feds cherry pick interpretations of the bill of rights to trump up their powers and then give the least charitable interpretations to the plebs.
Americans (the majority anyway) are still far too comfortable to care about any of this. Unemployment would need to be like 50% before most people even stopped to think "hey, maybe I should pay attention and vote".
"The Great Firewall (GFW; simplified Chinese: 防火长城; traditional Chinese: 防火長城; pinyin: Fánghuǒ Chángchéng) is the combination of legislative actions and technologies enforced by the People's Republic of China to regulate the Internet domestically" [1].
Only america can think there no harm in mass collection of data, and actively is against any attempts to limit it (gdpr for example) because it’s “anti growth”
reply