> "OpenBrain (the leading US AI project) builds AI agents that are good enough to dramatically accelerate their research. The humans, who up until very recently had been the best AI researchers on the planet, sit back and watch the AIs do their jobs, making better and better AI systems."
I'm not sure what gives the authors the confidence to predict such statements. Wishful thinking? Worst-case paranoia? I agree that such an outcome is possible, but on 2--3 year timelines? This would imply that the approach everyone is taking right now is the right approach and that there are no hidden conceptual roadblocks to achieving AGI/superintelligence from DFS-ing down this path.
All of the predictions seem to ignore the possibility of such barriers, or at most acknowledge the possibility but wave it away by appealing to the army of AI researchers and industry funding being allocated to this problem. IMO it is the onus of the proposers of such timelines to argue why there are no such barriers and that we will see predictable scaling in the 2--3 year horizon.
It's my belief (and I'm far from the only person who thinks this) that many AI optimists are motivated by an essentially religious belief that you could call Singularitarianism. So "wishful thinking" would be one answer. This document would then be the rough equivalent of a Christian fundamentalist outlining, on the basis of tangentially related news stories, how the Second Coming will come to pass in the next few years.
Crackpot millenarians have always been a thing. This crop of them is just particularly lame and hellbent on boiling the oceans to get their eschatological outcome.
This is a letter signed by the most lauded AI researchers on Earth, along with CEOs from the biggest AI companies and many other very credible professors of computer science and engineering:
"Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war."
https://www.safe.ai/work/statement-on-ai-risk
Laughing it off as the same as the Second Coming CANNOT work. Unless you think yourself cleverer and more capable of estimating the risk than all of these experts in the field.
Especially since many of them have incentives that should prevent them from penning such a letter.
Troubling that these eminent great leaders does not cite climate change among societal-scale risks, a bigger and more certain societal-scale risk than a pandemy.
Would be a shame to have energy consumption by datacenters regulated, am I right ?
Eh, not sure if the second coming is a great analogy. That wholly depends on the whims of a fictional entity performing some unlikely actions.
Instead think of them saying a crusade occurring in the next few years. When the group saying the crusade is coming is spending billions of dollars to trying to make just that occur you no longer have the ability to say it's not going to happen. You are now forced to examine the risks of their actions.
Reminds me of Fallout's Children of Atom "Church of the Children of Atom"
Maybe we'll see "Church of the Children of Altman" /s
It seems without a framework of ethics/morality (insert XYZ religion), us humans find one to grasp onto. Be it a cult, a set of not-so-fleshed-out ideas/philosophies etc.
People who say they aren't religious per-se, seem to have some set of beliefs that amount to religion. Just depends who or what you look towards for those beliefs, many of which seem to be half-hazard.
People I may disagree with the most, many times at least have a realization of what ideas/beliefs are unifying their structure of reality, with others just not aware.
A small minority of people can rely on schools of philosophical thought, and 'try on' or play with different ideas, but have a self-reflection that allows them to see when they transgress from ABC philosophy or when the philosophy doesn't match with their identity to a degree.
It also ignores the possibility of plateau... maybe there's a maximum amount of intelligence that matter can support, and it doesn't scale up with copies or speed.
Or scales sub-linearly with hardware. When you're in the rising portion of an S-curve[1] you can't tell how much longer it will go on before plateauing.
A lot of this resembles post-war futurism that assumed we would all be flying around in spaceships and personal flying cars within a decade. Unfortunately the rapid pace of transportation innovation slowed due to physical and cost constraints and we've made little progress (beyond cost optimization) since.
The fact that it scales sub linearly with hardware is well known and in fact foundational to the scaling laws on which modern LLMs are built, ie performance scales remarkably closely to log(compute+data), over many orders of magnitude.
Eh, these mathematics still don't work out in humans favor...
Lets say intelligence caps out at the maximum smartest person that's ever lived. Well, the first thing we'd attempt to do is build machines up to that limit that 99.99999 percent of us would never get close to. Moreso the thinking parts of humans is only around 2 pounds of mush in side of our heads. On top of that you don't have to grow them for 18 years first before they start outputting something useful. That and they won't need sleep. Oh and you can feed them with solar panels. And they won't be getting distracted by that super sleek server rack across the aisle.
We do know 'hive' or societal intelligence does scale over time especially with integration with tooling. The amount of knowledge we have and the means of which we can apply it simply dwarf previous generations.
I would assume this comes from having faith in the overall exponential trend rather than getting that much into the weeds of how this will come about. I can sort of see why you might think that way - everyone was talking about hitting a wall with brute force scaling and then inference time scaling comes along to keep things progressing. I wouldn't be quite as confident personally and as have many have said before, a sigmoid looks like an exponential in it's initial phase.
Global warming's worst effects aren't in 2-3 years, but we all (I hope) still care very much about it.
Perhaps the article is wrong about the timescale, but given how much AI has improved in the last 5 years, can you agree that it's likely to reach 'sit back and watch' levels in the next 5-10 years?
FWIW, as someone who had multiple friends who worked closely with Andy over 5+ years and at all stages of their career (both BS / PhD) those comments reek of someone with an axe to grind. All of the many anecdotes I have about Andy paint a picture of a great advisor and mentor. I suppose I should say "shenanigans aside", but if you can't separate his jokes from his academic side you need to develop a sense of humor.
Your analysis sounds reasonable to the non-expert, but recent work on purely-functional trees suggests that the gap is smaller than you suggest ("orders of magnitude slower").
E.g., see the nice work on the PAM library (https://arxiv.org/abs/1612.05665). Ideas from this work were used to build lots of cool things (immutable graph data structures, segment trees, databases) that are very fast, and all immutable.
This shit is all lies dude. In measurements like this, they constrain mutable structures to the same memory layouts and ideas as immutable ones and say “see! Sometimes kinda close!”
> bog standard FP community lies
> constrain mutable structures to the same memory layouts and ideas as immutable ones
What are you even talking about, "dude"? I don't think you have the background or knowledge you seem to think you have to argue about this space. It's OK, as you blithely pointed out in your earlier post, there is a place called medium where you'll find likeminded folks that will eat up your drivel.
Interestingly, medium is the place pushing idiotic “always runtime immutable or ur dumb” nonsense, resulting in this blog post because people such as yourself are eating it up on the basis of moronic bullshit like “beautiful”.
>what are you even talking about
I am talking about your own posts, perpetuating more moronic FP lies
>background
Nobody needs background to see that needlessly copying data for no reason is more expensive than not doing that.
Unfortunately for FP programmers, the moronic levels of doing this is actually worse than it sounds, as needlessly copying data is basically completely antithetical to how every modern CPU wants to optimally work.
It isn’t “me” declaring that runtime immutability is moronic. It is your CPU telling you that.
Given the age of your account my guess is that you're still young with too strong opinions and too little experience. That's ok, we all start there. But I can tell you from experience, that you'll need to outgrow that mindset in order to grow intellectually.
The difference between mutability and immutability is not as clear cut as you think, and might just be the difference between a `& mut` and a `&`.
Unfortunately none of these news articles or papers discuss whether consuming ~2--3g of xylitol/day in the form of gum (e.g., Pur) is OK. I'm guessing it's fine, given that the warning is for products that are artificially sweetned with xylitol with a lot more xylitol, and they say that toothpaste/mouthwash with xylitol should be OK.
That is a bit tricky at least in the US, I'm not sure how it's done elsewhere. The FDA has a habit of defaulting of assuming foods and additives are safe unless proven dangerous. They also frequently default to safety recommendations from manufacturers.
It's very possible that xylitol in toothpaste or mouthwash is considered safe because no one has identified those levels as dangerous. That isn't necessarily an indicator that we know it's safe, just that we haven't looked.
My assumption is that when the FDA was created there were already a large number of foods and food additives and testing all of those would have been impossible (and would have struck the public as absurd).
You also have to deal with the fact that food processors are fairly powerful and that the number of new foods/additives created a year would swamp the agency and be virtually untestable.
That's a reasonable guess, and on the more benign end of the spectrum for reasons it works the way ths title does. We really just have to guess at intent though, the best we can do is look at their output and see if we're okay with the quality of regulation they put in place.
I mean it appears the explicit answer is due to the 1938 FDC act. Whereby congress explicitly empowered the FDA to enforce the fact that new drugs must be shown safe before selling and advertising can begin. However no such rule was passed for food or supplements as such the regulatory authority in that space is oriented towards preventing "adulterated or misbranded or poisonous or deleterious food, drugs, medications, and liquor"
Having the legislative mandate definitely helps. Interestingly, the Chevron case would have left the FDA well within their rights to decide that a similar rule was important for food and supplements, they chose not to do that.
Our food supply was totally different in 1938, I can understand why the original legislation didn't include this. Processed foods weren't common, seed oils were even less common, and I assume supplements like we gave today didn't exist though I'm less sure about that one.
Well two of the major reasons for the original act were because of highly unsanitary conditions at slaughter houses (Upton Sinclair's The Jungle) and the use of known unsafe preservatives (e.g formaldehyde, spurred on heavily by Harvey Washington Wiley).
While I do think some oversight of food additives is warranted, given the way the FDA currently operates, having them have oversight over them would essentially be tantamount to banning new additives. If the cost was even 1/100th as high as it is to get drug approved it seems likely that there would be no economic payoff worth the hassle.
>If I wasn't generally opposed to an ever-growing scope of federal powers, I'd probably support an effective ban or extremely high bar on new additives. If food companies aren't willing to test it they shouldn't bother trying to sell it.
I think you are underestimating the cost of testing a new additive. It's also the case that the risk on additives is not nearly as high as with a drug, which paradoxically both makes the testing less important and also vastly more expensive (since the effect size is smaller). Something like xylitol for instance seems completely safe and we didn't have a huge obvious reason for thinking it was likely to be problematic. In fact previous studies were unable to pick up on the signal of increased stroke risk because they were confounded by the fact that sugar free consumers were vastly more likely to have those anyway.
As an additional point I'll just note that regulating this stuff would then have to deal with the fact that many things (like red meat) are already associated with known health risks. Putting principles aside, even if you want to only ban "additives", you'll start running into issues like we see with nitrates/nitrites, where organic manufacturers add celery juice (extremely high in nitrates), to avoid the label of added nitrates.
The vast majority of the cost to get a new drug approved is a Phase 3 clinical trial, which is all about proving efficacy. Just running phase 1 & 2 trials would be expensive, probably still prohibitively so, but it's no where near the cost of a full drug approval with phase 3 trials.
Yep that's a totally fair point. I was actually just talking to my father-in-law about that earlier today. We were processing a couple roosters and he was mentioning stories from his mother about how bad meat processing plants used to be.
If I wasn't generally opposed to an ever-growing scope of federal powers, I'd probably support an effective ban or extremely high bar on new additives. If food companies aren't willing to test it they shouldn't bother trying to sell it.
The same rationale for the Chevron defense (attacking the root of its power) - there's a large base of republicans who don't want to increase regulation on much of anything, and especially big business.
Mel Gibson even famously was part of an ad campaign to make sure that nobody COULD regulate supplements and to this day there's basically just about anything in those things you can buy at any convenience store.
This isn't a party problem, executive agencies have run the same way for decades regardless of who is in charge of the White House or Congress.
Republicans aren't against regulation more than Democrats, each side just wants a different set of regulations. Both parties want some level of restrictions on abortion for example, they just disagree on what the regulation should be.
Try to explain it any way you want, the last republican presidents have all made it part of their platform to slash regulation and let big business do what it wants, and the democrats have not made that part of their platform.
What regulations are you thinking about that were so one sided? I may very well just be misremembering here, or have a blind spot as I try not to pay too much attention to politics unless something big is happening.
It has seemed to me thst both parties cut regulations selectively. The Democrats, for example, fully embrace big pharma and is happy to let them effectively buy the NIH and FDA. This may fit for the Republicans as well, but that woulf be another item on my list of how the parties seem way to similar in my opinion.
Both parties also attempted to ban TikTok, whether that pans out or not.
Republicans aren’t republicans anymore, they are populist right wingers. The business dudes who just want to make money are in the back seat. It’s fanatics and reactionaries now.
“Starve the beast” in the context of chevron means that regulatory adjudication moves from administrative law judges in agencies to Federal courts. Supposedly this is to align with constitutional principles, but the reality is the except for the richest companies and individuals, litigation in federal court is prohibitively expensive.
Party platforms change over time. They're Republicans because that's the name of the party.
Its totally fair to say they aren't conservative anymore though, much like the Democratic party isn't liberal (at least classic liberal, I'm not a fan of the term having been co-opted and redefined).
More importantly to me, I just don't see much light between the base goals of the Republican and Democratic parties. They have done a hell of pushing voters into diametrically opposed corners, but I don't see the same with the parties. They debate details of most topics, but at the end of the day they both seem to be focused on growing the federal government, locking down our borders, and completely ignoring the concept of a budget.
If anything, both parties appear more to be run by neocons putting on a show of division to keep people distracted from what they're actually doing.
Here's a simple one, one party has attempted to enshrine voters rights and the other has done its damndest to make sure the smallest number of people vote in elections.
This idea that "they are all the same" is just an entirely bogus farce, the last republican tried to remain a king of america less than four years ago, this argument has no weight anymore.
What policies specifically do you consider the Democrats having tried to enshrine voter rights? Or do you mean access to voting, rather than the right to vote itself?
For Trump, do you actually think he attempted to make himself a king? Or is that embellishment when your actual view is that he attempted to vircumvent the electoral process and "win" a second term? I can't stand the man personally and would vote for an empty chair before him, but I also didn't read his attempt to flip the electoral result as an attempted annointment or coup.
I view them as effectively the same due to the huge overlap in policy. Both parties are pro-war, both parties want to lock down our borders, both parties want some kind of gun control, both want to limit access to abortion at some level, and both view the supreme court as a place that needs political appointees rather than the most qualified judges.
They absolutely debate the details, but neither party is for a balanced budget. Neither party supports real privacy laws. Neither party values individual freedoms over federal power. Neither party wants to limit federal powers in favor of state-level powers, though both are happy to make that claim only when it suits their specific law they can't get through at the federal level.
I think it's the US beloved default of non-regulation. While it can have its positives, it also creates monstrosity such as the FDA.
I'd personally rather go to the store and assume that most (if not all) the products available are proven to be safe rather than one which can sell anything that no one has yet proven to be unsafe.
I'd personally rather go to a farmers market or local store and buy food that was all grown and produced locally.
Where the US really got off the rails, in my opinion, was holding onto our distaste for regulation while simultaneously providing legal cover for corporstions to become massive and centralize industry so massively.
The FDA shouldn't be necessary at all. I don't need an organization to tell me raw milk from a local dairy is safe. The FDA is only needed for milk because we have a handful of massive, industrial dairies raising animals in terrible conditions and selling their product to everyone through national infrastructure balanced like a house of cards.
Eh, I wish that was always the case but it really depends on the individual farmer: I know many small and medium-scale farmers which use pesticides and chemicals. Even many people in their own small allotment don't shy away from using them willy nilly.
And raw milk is one of those things that it's all good and dandy until it's not. I still prefer buying raw milk and doing a LTLT pasteurisation to have the best of both worlds.
Totally agree on both counts. The benefit of local is that you can know the farmer at all. You'll just never get that with industrial ag. You'll never know how the animals were raised, how they were processed, or who all was involved as the animal likely was shipped all over the country or even overseas before you ate it.
I definitely get people's apprehension with raw milk. I very much need to know where it came from, and have a good idea for the health of the animal. We hand milk when our cows are lactating, and occasionallu buy from a local family dairy that we've worked with for years.
Additives and supplements grew into the loophole between food processing and drugs. Those lobbies grew as the US developed a political movement against regulation.
A lot of people make a lot of money selling ground up bark and food flavoring.
> In studies performed by Dr Hazen’s team, healthy volunteers were given a drink sweetened with 30 grams of xylitol. That’s similar to the amount found in a single scoop of keto-friendly ice cream or several cookies marketed for people with diabetes.
>
> In every volunteer studied, platelets were significantly more prone to clot after consuming xylitol.
"No serious health risk exists in most humans for normal levels of consumption. The European Food Safety Authority has not set a limit on daily intake of xylitol. Due to the adverse laxative effect that all polyols have on the digestive system in high doses, xylitol is banned from soft drinks in the European Union. Similarly due to a 1985 report, by the E.U. Scientific Committee on Food, stating that "ingesting 50 g a day of xylitol can cause diarrhea", tabletop sweeteners, as well as other products containing xylitol are required to display the warning: "Excessive consumption may induce laxative effects"
Xylitol gum exists in the EU too. It would almost certainly be banned if consuming even a slightly high amount of gum was linked to health risks.
reply