If you have independent copies of the network learning gradients, then you’re effectively making the batch size smaller— unless you’re doing an all collect and making them sync, in which case there’s a lot of overhead
When you take a batch and calculate gradients, you’re effectively calculating a direction the weights should move in, and then taking a step in that direction. You can do more steps at once by doing what you say, but they might not all be exactly in the right direction, so overall efficiency is hard to compare
I am not an expert, but if I understand correctly I think this is the answer.
I want to also mention that the previous model was 3.7. 3.7 to 4 is not an entire increment, it’s theoretically the same as 3 -> 3.3, which is actually modest compared to the capability jump I’ve observed. I do think Anthropic wants more frequent, continuous releases, and using a numeric version number rather than a software version number is their intent. Gradual releases give society more time to react.
The numbers are branding, not metrics on anything. You can't do math to, say, determine the capability jump between GPT-4 and GPT-4o. Trying to do math to determine capability gaps between "3.7" and "4.0" doesn't actually make more sense.
He's not saying that all non fiction is bad, just that the incentives are misaligned, and to be fair at least in my experience, there are a lot of popular non-fiction books where each chapter is repetitive, and I feel the whole thing could have been written in 2-3 chapters, if publishing a 30-page nonfiction book wasn't taboo
I assume it’s easier to find an engineer who went to engineering school to learn how to build airplanes that are safe than it is to find an MBA who went to business school to learn how to build planes that are safe. (It’s not about the knowledge but about the root desire)
Similarly, I assume it’s harder to find an engineer who went into the field purely for money.
I do think on average engineers will prioritize safety (since they likely understand failure modes and production and long tail statistics better. We literally have to take engineering ethics classes), at the cost of doing a worse job at running the business. But when the business requires this level of safety, that IS doing a good job.
You need to be able to steer a large and complex organisation - being an engineer has nothing to do with that. And, yes, incentives matter, but those can be set.
No. This just says some engineers can (like some MBAs). There is no intrinsic link between being an engineer and being a great CEO. Most engineers are not executives.
This argument relies on false equivalence and isn’t even rational.
Except top executives are engineering degree holders. They are, and that’s a fact. The majority of top performing companies are headed by engineering degree holding CEOs.
But you keep making the same wrong points and trying to play devils advocate on positions that you don’t back up.
If you want to claim an intrinsic link between being an engineer and being a top performing CEO, you need to show something different anyway. For example, that the proportion of engineers that are great CEOs is higher than the proportion of MBAs or lawyers or chemist or ... that are great CEOs. Maybe that is true, but I haven't seen it.
Edit: we could also look at a narrower problem, for example: is the performance of engineer CEOs in "engineering companies" better on average than that of non-engineer CEOs in that sector?
I am not saying engineers cannot be good CEOs, just that the link between being an engineer and a good CEO is (probably) not intrinsic.
"If you have a great executive ticking all the boxes - splendid.
But there is no intrinsic link between being an engineer and being a good CEO (same holds for other disciplines, btw.). You could have engineers that qualify, lawyers, MBAs, mathematicians, physicists, ...
I think planes can still fly with the rudder loose? If the bolt falls out and it loses control, wind will push it into the neutral position and then flying will still be possible with other control surfaces? But I guess if the pilots don't know and it happens suddenly at a critical moment or if the bolt causes the rudder to get jammed, then that would be really bad. But I assume it falling out would result in the rudder loosely returning to neutral...
Planes can fly with the rudder inoperable, although with some restrictions -- you wouldn't want to do a serious crosswind landing, and you wouldn't want to stack it up with other failures, especially asymmetric engine failures.
However, that doesn't mean that planes can fly with the rudder /loose/. A significant risk in higher-speed airplane designs is that of aerodynamic flutter, where aerodynamic forces excite a vibration mode in the airframe, or a subset of it. You can find some impressive video of e.g. bending modes in sailplane wings being excited, with increasing magnitude bending until the wings are destroyed (or the excitation is reduced dramatically, or shifted to a different frequency). While aeroelastic modes get a lot of attention in flutter analysis, loose control surfaces can be much, much worse, because movement of the surface within the lash provided by the loose connections is effectively undamped.
The rudder is necessary for directional control specifically turns, and for flying straight in a crosswind.
I've heard of a few cases where applying more/less power on the right/left engines can sort of crudely achieve the same thing, and you might get lucky and get on the groud without crashing, but loss of the rudder would be a serious emergency indeed.
> The rudder is necessary for directional control specifically turns,
Only if you add a secondary constraint of coordinated turns, which are important for passenger comfort and efficiency, but not directly a safety concern. (You still need directional stability, but that's provided by the fixed portion of the vertical tail, not the rudder.)
> and for flying straight in a crosswind.
Only if you add a secondary constraint of alignment between body angle and flight path. This constraint is totally absent in normal flight -- it only comes up during takeoff and landing, where it's useful to have the plane lined up with the runway to avoid side-loading the landing gear. In the case of a known rudder failure, you'd head to an alternate where there's not much crosswind, to avoid this issue; but you wouldn't expect many issues getting there.
The third case where the rudder is actually critical is when combined with other failures, especially asymmetric engine failures. There are parts of the flight envelope where a single engine failure combined with a rudder failure would not be expected to be survivable.
In fact, the rudder does not do what new pilots think it does (it is NOT like a boat rudder at all, really, because the plane banks) that instructors will often make you practice flying without using the pedals at all.
If we have intelligent AI that can automate programming, then making really good robots will not be a problem. While not trivial, actuators and power systems are not the reason why we don’t have robots that can do all manual labor for us. Software is the reason, and the same kind of software that’s learning to code (machine learning) can also be adapted to washing dishes, folding clothing, doing craft labor or previously human manufacturing jobs.
Accelerating programming and information jobs also means accelerating the creation of robots that can do these trade jobs
Totally - but is the AI development of robots included in the timeline for what you're calling AGI? Can we get to AGI without having those robots, and then the AGI designs them?
I think they'll ultimately go hand in hand - this is more just a question of what we're defining AGI to be and whether robotics should be mandatory as part of the stated definition around doing 95% of work.
The AGI will have to run the robots, problem solving when things go wrong is what allows humans to run large organized endeavors without getting stuck, you need AGI to do that for generalized work. Before AGI robots will only be able to handle tasks with very simple error scenarios, and will still need humans to look after them for the rare cases where things go more wrong.
Recommend passing the speech-to-text narration through a round of GPT4 API to correct for any transcription errors (use some prompt giving context that it's speech to text)
Suggests there's other variables involved, like time of day taken, other supplements taken simultaneously, metabolic processes, diet, and maybe even the placebo effect.
Well, assuming they likely have some PFAS already in their system, you might just be giving them blood with the same concentration of PFAS that they already have.
When you take a batch and calculate gradients, you’re effectively calculating a direction the weights should move in, and then taking a step in that direction. You can do more steps at once by doing what you say, but they might not all be exactly in the right direction, so overall efficiency is hard to compare
I am not an expert, but if I understand correctly I think this is the answer.