These token numbers look off for a 5090. For comparison, an rtx 5000 Blackwell SFF with just 470gb/s bandwidth gets me 30 tok/s on gemma4-31B-QAT. Almost 60 tok/s with MTP enabled. A 5090 should get you much more than that!
Thanks for flagging, this is on my local rig and it's driving my display too. I'm curious now, will take a closer look. These are the tok/s as reported by LMStudio.
EDIT: Updating llama.ccp gets me 58 tok/s on Gemma 31b
The Qwen-35B-A3B numbers are even weirder, did you drop a digit? I get half of that speed on a AMD Radeon RX 5500 XT (RADV NAVI14) (8192 MiB) (unsloth/Qwen3.6-35B-A3B-MTP-GGUF:Q6_K with q8_0 context).
bahaha, LMStudio's badge looks a lot like a B but the tooltip says it is indeed an 8 (I just assumed it was B for "byte", but that's a poor initialism considering "bit").
"Interestingly, Kimi K3 got rid of all RoPE layers and uses NoPE (No Positional Embeddings) everywhere instead."
It just baffles me that this even works at all. Doesn't it just become a token soup? Is attention that precise that a second token can tell its the second token just because it learns to accumulate something in the embedding space without any sort of inductive bias?
As a sibling comment points out you don't strictly need positional embeddings for decoder-only causal transformers. You definitely need it for non-causal ones (e.g. the encoder of the original transformer paper!).
And yes accumulation is a good intuition for what's going on. You could imagine a part of the attention head that just kept writing to the same part of the residual stream causing that to keep accumulating (simply via attention summation) as more input tokens come in thereby functioning as a kind of index without the need for any positional encoding.
And adding to that, there is also the recurrent state in the Kimi Delta Attention. I wouldn't call it position information but more sth like "position sensitivity"
"Kimi K3 uses no explicit positional embedding (NoPE), and instead encodes positional information implicitly through the recurrent gating and decay mechanism of KDA."
Linear layers use decays (like IIR filters) that naturally provide relative positions. Full attention layers can then be free to develop concepts that attend to each other regardless of distance.
When you have recurrent blocks in your model, you implicitly have a timestep T(amount of recurrent steps). Similar to Diffusion Transformers, it then becomes valuable to encode the knowledge of where you are in this chain somehow. NoPE is more flexible than RoPE for this.
I'm no expert analyst on this topic, but I'm worried this time might be different for them. China finally has the technological capability to challenge this time around and they will gobble up the unmet demand, allowing them to get more experience and more money to catch up.
"Your plan pricing is unchanged: Copilot Pro remains $10/month and Pro+ remains $39/month, and each includes $10 and $39 in monthly AI Credits, respectively."
If there's no discount on credits (in terms of tokens per dollar) over other providers, I'm going to switch to a PAYG provider. If there's a month where there's little to no coding I can pocket the 10$. What incentive do they give to stay with this plan?
Or if you're a business with multiple seats, these plans may be more inefficient than raw API usage billing. Since if anyone at your organization fails to utilize their full $19/39 allotment each month, that's wasting money, whereas with API credits it is 100% utilized.
I don't think they've thought through the implications of this. Everyone should cancel and go usage-based billing with caps.
They do address this in the doc, Orgs can now (although it was vague as to whether it was an option or just the new standard, probably option due to business contracts) 'pool' the Usage billing across all users.
I'm guessing they did that (and the 'temporary bonus credits') to make the pill easier to swallow for that side of customers.
It forces you to pay at least $20 in tokens per user even for people who use less (they probably have stats on how many people use just autocomplete, which doesn’t count against the quota. or have a seat and don’t use the service at all).
You can get at least at baseline vague stats but from what I have seen it is more account than user focused, i.e. 'X number of users used %feature%' and/or '%model% was X percent of requests per day'.
That said it's worth noting, I don't see how anything they expose will reliably help orgs plan costing from what AFAIK is in fact a big shift for billing/costing planning.
> It forces you to pay at least $20
For better or worse that's public pricing, i.e. if you are coming in also negotiating VS for devs, windows/office licenses for the rest of the business and stuff like Azure Devops... a lot of their stuff gets cheaper if your company's IT procurement group is vaguely competent at negotiating. Not even talking bigcorp here I'm talking 500-1000 employee range.
Of course, very small orgs will suffer, but it does tie in with the theme over the last two weeks; anyone with a personal account is basically subsidizing the credits for the business accounts during the transition period.
This was my first thought too. "Oh cool, I should be seeing lower prices" as I don't use Co-pilot that often anymore. But no, that's not the case. It rather served to remind me that I should probably just cancel.
I think you are spot on and a lot of other comments sharing "I'm also so precise, and people don't get it and it's frustrating" are in fact the problem. It's arrogant to think you're that eloquent that there is not other interpretation to your words, and the problem must be with the reader. It only results in more inefficiency if you stick to that mindset.
These are probably the same people that say "everyone else's code smells" and think only they write the perfect code.
