It's a signal. It's not a strong signal, and you certainly should not base your entire perf on it, but if the number is unusually high or low, it's a signal that could warrant further investigation.
(I once worked with an engineer that had two PRs, both fairly small bug fixes, in a given calendar year, and when I looked more carefully, they did not have any other obvious output or impact.)
Strongly agreed. It is a signal. I did an analysis once at the end of the year. Work group of about 45 engineers. The CM system had a lot of steps, and work could get bounced around, but there was a step where some one "resolved" a software activity. Bug fix or new requirements, it did not matter. This step was when someone actually completed work and put into into the dev stream.
A quick DB query and the variance was substantial. A couple of people had over a hundred. About 10 had 2. For the year. The ramp up was slow, average was 8 to 10 a year.
Dig a little deeper. Those at the top were 'group leads' not only did they do IC work, they also got stuck with all 'paperwork' on the problem work packages. They had 'power', so they could override various things. So, they were doing a lot of work, and taking care of things. Good signal, matches what one would expect.
Those at the bottom. One of them had effectively been a 'systems engineer'; all of their time was working on requirements with the customer, making powerpoint, etc. Important work, so that signal was inverse of what it originally showed.
A couple were in the middle that had great reputations for technical expertise. They were spending almost full time in training / mentoring / very hard problems mode. Highly valuable, but not shown by looking at these numbers.
All the rest? 80% of the work was being done by 20% of the people. We could have dropped about 12 heads and barely noticed.
The problem is, you could not take action on this measure. It gave you a place to start, but you needed to know more about what was going on day to day.
Let's measure "executive performance" by counting how many "answered phone calls" per hour they have. If they don't answer enough calls, that's a signal, that they aren't doing anything useful, and should be depreciated as a result.
Trying to parse your sentence, which is ambiguous...
You're saying that the manager-of-managers would argue that the number of PRs should affect perf ratings? Or the MoM would push back against the line managers who were giving ratings based on # of PRs?
Yes, I have been listening to Gruber complain about them for a year now. His critique is that "Apple's HIG used to say not to," not anything about what actually makes them bad.
Curiously, I haven't heard him talk about Apple Intelligence in Shortcuts, or any of the cool new features in Tahoe. Design is how it looks, I guess.
GitHub didn’t embrace, extend, extinguish git. You can git push to a different company (e.g. Gitlab) and you’ve migrated. The biggest problems with GitHub are scaling and availability, not lock-in.
Microsoft today is nothing like it was 30 or 20 years ago.
Fifteen years ago we were writing HTML and JavaScript specifically for Internet Explorer. Edge is built on Chromium.
VScode might be open source (to a degree), but the levers of control are on a different location. See https://ghuntley.com/fracture/ for an excellent blog on it.
Good luck migrating the ”forge” part with the ”git” part. Your github org settings, pull requests, rulesets, CI/CD pipelines, containers, copilot...
The lock-in always comes from the ”forge” part, never the ”git” part.
Open core at best. It's proprietary software built on top of an open source base. The remote coding feature is proprietary and you need to run proprietary software on the remote server / container to use it. People maintaining forks (like Codium and the Theia IDE) are not allowed to use VS Code's marketplace. Many of their flagship VS Code extensions are proprietary. Why would they do this if they believed in open source?
The distinction is quite important. VS Code aims to get control of the development process of those who are not using Visual Studio. That's the only reason why VS Code exists. VS Code is not a gift no strings attached.
By the way the title of https://code.visualstudio.com/ is a lie that says "The open source AI code editor". Three lines under, there's "By using VS Code, you agree to its license and privacy statement.". The license is https://code.visualstudio.com/license, which is very much like your usual horrible Microsoft EULA, including tracking and forbidden reverse engineering, decompiling or disassembling. Really, the only thing missing there is the license key field at first run.
GitHub is still proprietary SaaS also aiming to control the whole open source ecosystem. With GitHub, a big chunk of the open source (and free software! Which is even sadder) world relies on proprietary infra. That's as close as Extinguish as you can get (it's just that git is not the thing that's Extinguished). GitHub is actually a pretty good example of lock-in, see what other commenters wrote on this.
30 years later, Microsoft, still the same lying company trying to control its users and the world with proprietary software. With the twist that they try a bit harder to look cool and open source (since the moment they realized open source wasn't going to disappear, not before). They really are not, especially for end-user facing software, including when the end-users are developers.
The only thing that dramatically changed is that they don't publicly claim Linux is cancer anymore, and that's probably because they are coerced into dealing with Linux. Exactly like the Web against their failed attempt to privatize it with MSN (MicroSoft Network) (the current MSN news frontpage and the memory of their messenger are only shadows of the original ambitions behind MSN).
At least the stability and consistency is comforting… or not.
Don't fall for their open washing. They just play along and attempt to get control on what they didn't manage to extinguish. Only forced changes happened, the spirit seems intact.
People would revolt if there were auctions, because it would make it impossible for many people to attend live events. Right now there is the hope that you can, even if that’s most not true.
For example, I did the Amex presale for US Open tickets. There were 22k people in front of me for regular day passes during the earliest rounds. So we end up with an auction-ish situation anyway, via the resale market, but can blame scalpers.
The thin illusion keeps it palatable enough, if only barely. Ticketmaster is wants to extract the most profit. Being hated is fine. Being regulated because you’re _too_ hated is not.
For some artists in Japan we do lotteries. There's an application period and then the opportunity to purchase is doled out randomly among the applicants. You can often apply for multiple tickets together if you're a group, but ID is checked at the venue for the purchaser so you have to arrive together, making resale more awkward. Usually there are multiple rounds to sell the tickets when nobody paid - some times an early round for fanclub members.
If they stop releasing their larger models because they want to monetize, would we expect them to release better small models that can outcompete those?
there's pros and cons to it for them. Clearly, they get good branding (at least in enthusiast circles). Perhaps more important is they get community work on optimization. There have been significant performance uplifts on the Qwen3.6 models from the open-source community since they were launched (at a minimum, multi-token prediction is now working with them. It is almost a 2x token generation speedup)
Those are not GPUs available on iPhones. Will we get there eventually? Maybe! Maybe we end up with GPU clusters built on the edge (e.g. cell towers) for offloading, maybe it’s never economical, maybe a different model architecture makes it simpler, who knows.
But it doesn’t seem anywhere imminent with our current world state.
My computer is 15,000 times faster and costs in inflation adjusted dollars half that of my computer in 1995. There's zero reason to think that won't happen over the next 30 years again.
For whatever reason every generations thinks they are the peak. Naw man. You're just a blip at the bottom of the logarithmic chart.
- was the pause in model scaling a result of the benefits of RL & SFT being easier to access and quicker than scaling, or was it genuinely the result of scaling being low ROI now?
- are power densities necessary to provide high quality on device inference possible? Can the best, technically feasible, architectures accomodate T scale models and run them off batteries that fit in your hand?
- will thing slow down enough to allow edge depoloyments to realise value vs. centralised deployments.
- do edge use cases drive enough revenue to get this to happen?
- can local inference make up for model scale? Does that make sense in a latency/power race with the central infrastructure? Is there a sweet spot here?
It has slowed down massively for CPUs at least. e.g. modern CPUs are hardly more than 3-5x faster than those from 10 years ago. There is zero reason to think won’t happen over the next 10 years again.
This isn't an crazy statement (cpu performance metrics have mostly stalled their meteoric rise from prior to the 2000s)
But it also doesn't capture the entire picture.
CPU metrics mostly stalled for two reasons.
1. There wasn't much demand for the extra capacity. Even low end cpus from a decade ago are plenty capable for just browsing the web and typing up documents. It takes a novel use-case to drive demand again (or a desire to do things like play new games).
2. The interest in CPU development shifted in response to mobile. Given point #1 and the state of battery development.... the blocker wasn't "performance". It was "performance per watt". And on that metric you couldn't be more wrong.
Since ~2005, MIPS per watt has improved 15x to 30x.
Also - fun news is that the traditional CPU pipeline really isn't the bottleneck for AI workloads. So we're going to see incredible interest in things like memory bandwidth and other inference related hardware bottlenecks, which haven't already been optimized.
> There wasn't much demand for the extra capacity. Even low end cpus from a decade ago are plenty capable for just browsing the web and typing up documents.
It stalled before the rise of PC-as-Internet-portal.
I bought a high end PC in 2003, and 5 years later the PCs were not much faster - probably not even 2x. Around 2008-2010 was when most people started using PCs as a way to connect to the Internet.
It stalled because scaling got a lot more challenging. Not because of lack of demand.
Yes, but it only stalled along a single dimension - Single core clock speed.
I was building gaming machines in the early 2000s, I absolutely remember the 4ghz wall that cpus hit.
But it wasn't a real wall... because we then got one of the arguably most influential processors ever in the Core 2 duo. Which... blew the limit away by giving you two processors clocked at 2.93 GHz each.
And honestly, even then - it was lack of demand (we could go to 4+ghz, but we didn't want to pay the power bill for the rest of the system - the planned pentium 5 was 7-10ghz on paper, but they canceled the project because keeping it fed and cool was too hard for personal desktop machines).
Of Note - we did reach these speeds on consumer hardware (ex - in 2012, Andre Yang hit 8.794Ghz on an AMD FX-8350)
So it was never "impossible" to keep scaling. It just wasn't worth it compared to going multi-core.
---
And maybe it's because I was in my formative years at this time, but you're off by 5+ years with this:
> Around 2008-2010 was when most people started using PCs as a way to connect to the Internet.
Gmail was a web only email client released in 2004. Wikipedia was released in 2001. Web browsing was very much one of the "killer" apps for computers by the 2000s. What do you think the damn 2000s dot-com bubble crash was?
at the risk of aging myself - I was born in '89, and I literally do not remember a time where we didn't have DSL speeds and above (friends houses often still had dial-up until ~2005, though).
> Gmail was a web only email client released in 2004
Well, Gmail was actually one of the last web based email clients people used :-) Yahoo mail, Hotmail, and so many others predate Gmail by years.
> Web browsing was very much one of the "killer" apps for computers by the 2000s.
One of them. People still used non-browser apps for all kinds of things: Media consumption (people didn't watch movies on Youtube), Office (Google Docs was very much a niche thing for many years), photo-editing (lots of pirated versions of Photoshop/Lightroom years after the iPhone release), etc.
Most non-mail, non-social media, non-shopping stuff people do on the web these days was a dedicated SW from the vendor in those days. Want to make a photobook? Download this Windows binary and set it up there. It will then communicate with the server for the order (no browser utilized).
> at the risk of aging myself - I was born in '89, and I literally do not remember a time where we didn't have DSL speeds and above (friends houses often still had dial-up until ~2005, though).
Spring chicken! My first online experience was on a 340 baud modem :-)
> There are three ways to make a living:
> 1) Lie to people who want to be lied to, and you’ll get rich.
> 2) Tell the truth to those who want the truth, and you’ll make a living.
> 3) Tell the truth to those who want to be lied to, and you’ll go broke.
reply