I think that formatting code is necessary to maintain a codebase that's used by multiple people and keep some consistency. It's very confusing to have different standards in different parts of the same code.
Code should be generally written so it's easy to read.
I like short lines in general, as having a bunch of short lines (which tend to be the norm in code) and suddenly a very long line is terrible for readability. But all has exemptions. It's also very dependent on the programming language.
I see some of this, from the point of view that it's going to be cheaper to create bespoke solutions for problems. And perhaps a "neoSaaS" company is one that, from a very bare bones idea, can create your own implementation.
But, at the same time, there are two issues:
- Companies can be really complex. The "create a system and parametrise it" idea has been done before, and those parametrisation processes are pretty intensive and expensive. And the resulting project is not always to be guaranteed to be correct. Software development is a discovery process. The expensive part is way more in the discovery than in the writing the code.
- The best software around is the one that's opinionated. It doesn't fit all the use cases, but it presents you a way to operate that's consistent and forces you to think and operate in certain way. It guides you how to work and, once going downstream, they are a joy to work with.
This requires a consistent product view and enforcing, knowing when to say "no" and what use cases not to cover, as they'll be detrimental from the experience.
It's very difficult to create software like that, and trying to fit your use case I'll guarantee it won't happen.
These two things tension any creation of software, and I don't think they'll go away just because we have a magical tool that can code fast.
The best software around is Emacs. Does that count as "opinionated" in your view?
In some ways it is—Emacs does a lot of things its own way, completely unbothered by mainstream conventions—but, at the same time, it's also totally malleable in the sense of this article. What makes Emacs great is a consistent and coherent conceptual foundation coupled with remarkably flexible code, letting you adjust Emacs to your needs rather than adjusting your needs to Emacs.
Or maybe the best software around is situated software. Software that's built by and for a specific set of people in a specific social context. Situated software is qualitatively different from product software, and it works so well because, again, it gives its users real agency and control. Instead of trying to create software that knows better than its users, we can create software that supports its users in whatever ways works for me. The result is still opinionated, but it's opinionated in a categorically different way from what you're describing.
So perhaps the best mainstream software is Excel.
And, while I don't think they're there now, it seems like LLMs are likely to be the foundation for the next Excel.
You can either go with simple primitives and a way to combine them (emacs, excel, unix) or simple program that just works (notepad, sumatra,…). Anything else is going to be restrictive in one way or another.
As a vim user I agree with all this. Same is true about why I am terminally terminal. I'm able to adapt the tools to me so that I may get the most use out of them. Sane defaults are great, but there are no settings which are universal. The only solution to this is to let people adjust as needed.
I think the article presents a bit of an odd premise - I can make a mini app in ChatGPT today so by 2035 I can create an entire suite of software needed for a given business. What is the requisite change between what I can do now and in 2035? Presumably it is AGI.
OK, so we are in a digital super intelligence world in 2035. The HR department can now just have a conversation with a chatbot and create software to make them more productive. No more configuring SAP widgets or whatever they do today. The chatbot will be like "hey bro, the process that you want to automate doesn't make any sense: here is a better way. And, by the way, I'm terminating your entire department. I'll take care of it from now on". I mean, get real, in a post DGI world there will be exactly zero office jobs and no SaaS software at all.
It doesn't need to be AGI to build complex software. A human software developer can build a complex software system and perform other complex tasks with the same body (play an instrument, fly an aircraft, etc.). Doing all of that with the same resources is what AGI is needed for. Just software, well I'm sure an LLM can eventually become an expert just like it learnt how to play Go.
AGI usually means "equivalent to human" while digital super intelligence generally means "smarter than all humans put together". In any case I agree that once we reach "equivalent to human" naturally it can do anything we do. That should be enough to end office jobs imo.
A machine, that is capable of performing human intelligence in every paradigm, according to a mathematical model, and scalable by increasing the frequency, power or duplicating it, because it is reproducible, is both "equivalent to human" and "smarter than all humans put together". When humans were capable of producing this, then this will be capable of improving itself and optimizing until the limit of information density. The only limit will be money as a proxy of available resources.
> scalable by increasing the frequency, power or duplicating it
Well there's your problem. Very few things scale like that. Two people are not twice as smart as one person, nor are two instances of ChatGPT are twice as smart as one. One instance of ChatGPT running twice as fast isn't significantly smarter, and in fact, ChatGPT can never outrun its own hallucinations no matter how fast you overclock it.
Intelligence is the most complex phenomenon in the universe. Why would it ever scale geometrically with anything?
> When humans were capable of producing this, then this will be capable of improving itself and optimizing until the limit of information density.
This doesn't follow. After all, humans are as smart as humans, and we can't really optimize ourselves beyond a superficial level (good nutrition, education, etc). Increasingly, AI is a black box. Assuming we do create a machine as smart as we are, why would it understand itself any better than we understand ourselves?
And why wouldn't we hit some sort of technical roadblock at (arbitrarily) 1.5x human intelligence? Why do we assume that every problem becomes tractable once a computer is solving it? Imagine we applied this reasoning to cars: Over a matter of a century, cars went from 10 km/h to 100km/h to 500km/h to (in special vehicles) 1000km/h. Can we expect to see a 5000km/h car within the next century? No, that's unlikely; at such high speeds, you begin to hit intractable technical limits. Why should scaling intelligence just be smooth sailing forever?
I wasn't talking about two instances for scaling smartness, I meant applying two instances to different problems. That very much scales.
> This doesn't follow. After all, humans are as smart as humans ...
In the hypothetical case of humans capable of producing the one true AI system (real AI or AGI or however its called, because marketing has taken the previous term), then this system is capable of producing another system by definition. Humans are capable of following Moores law, so this system will as well. So this chain of system will explore the set of all possible intelligent systems restricted only by resources. It isn't bound by inner problems like "(good nutrition, education, etc)", because it is a mathematical model, its physical representation does only matter as so far as it needs to exist in this hypothetical case.
> AI is a black box
In this case, the black box "humans" was able to produce another thing reproducing their intelligence. So we have understood ourselves better than we currently do.
Note, that every intelligent system is completely able to be simulated by a large enough non intelligent statistical system, so intelligence isn't inferable from a set of inputs -> outputs. It's really the same as with consciousness.
> And why wouldn't we hit some sort of technical roadblock? Can we expect to see a 5000km/h car?
Yes. We are capable of accelerating "objects" to 0.99..c. It's not impossible for us to accelerate a "car" to nearly light speed, we "just" need enough energy (meaning matter as energy).
> technical roadblock at (arbitrarily) 1.5x human intelligence
I wrote "until the limit of information density". Whatever this may be.
I intended to point out, why a system "equivalent to human" is actually equivalent to "digital super intelligence meaning 'smarter than all humans put together'".
---
You don't need to tell me you don't think this system will exist. I think this will end the same as the attempts to build a machine creating energy. My personal understanding is this: A system (humans) can never completely "understand" itself, because it's "information size" is as large as itself, but to contain something, it needs to be larger then this. In addition that "understanding" needs to be also included in its "information size" so the size to understand has at least doubled then. This means that the largest system capable of "understanding" itself has the size of 0.
In other words understanding something means knowing the whole thing and abstracting to a higher level then the abstractness of the system to be understood. But when the system tries to understand itself, it's always looking for yet another higher abstraction to infinity, as each abstraction it finds is not yet enough.
This idea comes from the fact, that you can't prove that every implementation of a mathematical model has some behaviour, without formalizing every possible model, in other words inventing another higher model, in other words abstracting.
> I meant applying two instances to different problems. That very much scales.
You can't double the speed at which you solve a problem by splitting it in two and assigning one person to each half. Fred Brooks wrote a whole book about how this doesn't scale.
> this system is capable of producing another system by definition
Yeah, humans can produce other humans too. We're talking about whether that system can produce an improved system, which isn't necessarily true. The design could easily be a local maximum with no room for improvement.
> Humans are capable of following Moores law
Not indefinitely. Technical limitations eventually cause us to hit a point of diminishing returns. Technological progress follows a sigmoid curve, not an exponential curve.
> It isn't bound by inner problems like "(good nutrition, education, etc)", because it is a mathematical model
It's an engineering problem, not a math problem. Transistors only get so small, memory access only gets so fast. There are practical limits to what we can do with information.
> We are capable of accelerating "objects" to 0.99..c.
Are we? In practice? Because it's one thing to say, "the laws of physics don't prohibit it," and quite another to do it with real machines in the real world.
> > technical roadblock at (arbitrarily) 1.5x human intelligence
> I wrote "until the limit of information density".
Yeah, I know: That's wildly optimistic, because it assumes technological progress goes on forever without ever getting stuck at local maxima. Who's to say that it doesn't require at least 300IQ of intelligence to come up with the paradigm shift required to build a 200IQ brain? That would mean machines are capped at 200IQ forever.
> Note, that every intelligent system is completely able to be simulated by a large enough non intelligent statistical system, so intelligence isn't inferable from a set of inputs -> outputs.
This is circular. If a non-intelligent statistical system is simulating intelligence, then it is an intelligent system. Intelligence is a thing that can be done, and it is doing it.
> A system (humans) can never completely "understand" itself, because it's "information size" is as large as itself, but to contain something, it needs to be larger then this.
I don't think this logic checks out. You can fit all the textbooks and documentation describing how a 1TB hard drive works on a 1TB hard drive with plenty of room to spare. Your idea feels intuitively true, but I don't see any reason why it should necessarily be true.
I only need two instances to be faster then a single one. This means the human having the resources to run the system is unbound to do anything an infinite number of humans can do regarding his own time and energy.
> Yeah, humans can produce other humans too
In this hypothetical scenario humans were able to build "AI" (including formalized, deterministic and reproducible). A system as capable as a human (=AI) is then able to produce many such systems.
> There are practical limits to what we can do with information.
Yes, but we are nowhere near this limits yet.
> Are we? In practice?
Yes. We are able to build a particle accelerator. Given enough resources, we can have enough particle generators as there are particles in a car.
> That would mean machines are capped at 200IQ forever.
Except when the 300IQ thing is found by chance. When the system is reproducible and you aren't bound by resources, then a small chance means nothing.
> This is circular.
No it just means intelligence is not attributable to a black box. We don't think other humans are intelligent solely by their behaviour, we conclude that they are similar then us and we have introspection into us.
> You can fit all the textbooks and documentation describing how a 1TB hard drive works on a 1TB hard drive with plenty of room to spare.
It's not about encoding the result of having understood. A human is very much capable of computing according to the nature of a human. It's about the process of understanding itself. The harddrive can store this, it can't create it. Try to build a machine that makes predictions about itself including the lowest level of itself. You won't get faster then time.
> Given enough resources, we can have enough particle generators as there are particles in a car.
Given by whom? I said in practice—you can't just assume limitless resources.
> Except when the 300IQ thing is found by chance. When the system is reproducible and you aren't bound by resources, then a small chance means nothing.
We're bound by resources! Highly so! Stop trying to turn practical questions about what humans can actually accomplish into infinite-monkey-infinite-typewriter thought experiments.
> We don't think other humans are intelligent solely by their behaviour
I wouldn't say that, haha
> It's not about encoding the result of having understood. It's about the process of understanding itself.
A process can be encoded into data. Let's assume it takes X gigabytes to encode comprehension of how a hard drive array works. Since data storage does not grow significantly more complex with size (only physically larger), it stands to reason that an X-GB hard drive array can handily store the process for its own comprehension.
Because I think we haven't even started. Where is the proof based system able to invent every possible thought paradigm of humans a priori? I think we are so far away from anything like this, we can't even describe the limits. Maybe we will never have and never do.
> you can't just assume limitless resources
I assumed that, because the resource limits of a very rich human (meaning for whom money is never the limit) and the one true AI system are not different in my opinion.
> comprehension
Comprehension is already the result. But I don't think this is a sound definable concept, so maybe I should stop defending this.
> Where is the proof based system able to invent every possible thought paradigm of humans a priori?
Beyond the realm of feasibility, I'd imagine. The gulf between what is theoretically possible and what is realistically doable is gargantuan.
> I assumed that, because the resource limits of a very rich human (meaning for whom money is never the limit)
The resources of a very rich human are extremely limited, in the grand scheme of things. They can only mobilize so much of the global economy, and even the entire global economy is only capable of doing so much. That's what I'm getting at: Just because there's some theoretical configuration of matter that would constitute a superintelligence, does not guarantee that humanity, collectively, is capable of producing it. Some things are just beyond us.
I'd say it might scale like whatever your mathematical model is telling you, but it might not. I don't think we have a reasonable model for how human intelligence scales as the number of brains increases. Sometimes it feels more like attenuation than scaling in many meetings.
> The best software around is the one that's opinionated.
This. And it isn't going to change.
The post avoids trying to answer "Why are opinionated tools popular and effective?"
The answer is that a standardized process that they encourage is often more efficient than whatever bullshit {random company} came up with in-house.
Malleable software needs to produce two equivalently good outcomes to beat opinionated:
1. Improve the underlying process at the customer's business (in terms of effectiveness)
2. Avoid a customization maintenance burden
The seductiveness of "just for you" bespoke solutions is they avoid (1) by telling the customer what they want to hear: you're so brilliant, your process is actually better, our product is a custom fit for your exact process, etc. That's bullshit -- a lot of customer processes are half-baked dumpster fires, and their companies would be better served by following standards.
To (2), I am incredibly skeptical on the long-term tech debt that malleable solutions will impose. What happens when there's a bug in the version only you use? Is that going to be the vendor's priority? Oh, you're supposed to fix it yourself? Congrats... we've just added a requirement that these tools are capable of making random mid-level in-house practitioners as competent as focused dev teams. That's a tall order.
Exhibit A that I'd want a follow-up post to address: SAP.
The above are the reason they realized they were trending in the wrong direction and have been dragging their customer base back to Clean Core.
Walk me through how malleable software would work better for SAP as a product, and I'll begin to believe...
In my first job I had to work with healthcare software and it horrified me. There is a standard for interop, HL7, but every system implements HL7 in its own special way so there are "integration engines" to massage the data so that they all conform to the same standard.
The history of HL7 is kind of nuts. It was originally developed for copper wire communication in 1979. Formalization was ongoing until maybe the early 1990s and lots of proprietary usage arose, because back in the 1990s none of these systems really inter-operated and everything eventually ended up on paper. It wasn't until after the ACA that a lot of interoperability pushes really got going at scale. Before that you had a few Health Information Exchanges at state levels so there was usually a local standard if there was an HIE. HL7 FHIR is much more standardized now.
I wouldn't call any of it a grift. It's just old tech built for a fragmented archipelago of systems that didn't communicate. Also you can write a pretty good HL7v2 parser in an afternoon, I've written maybe 5 of them.
The koan that unlocked why healthcare technology is the way it is for me:
I was working on automating health insurance claims processing on a mainframe system.
In their key interface, a key form had 8 blanks for ICD codes. If more than 8 codes were needed, a child claim was created and linked to the parent claim.
This was a long project, so I was staring at this interface for months, as linked child claims made automation more complex than it needed to be. (E.g. if a parent claim had aged, been archived, and needed to be reloaded to active overnight before processing the child claim)
Finally, I started asking around. "This is a computer system. Why are there a finite number of fields for something that might need more?"
Nobody knew. Project continued. I continued asking different people.
Finally, I asked a guy who had been working in the industry since the 1960s...
"Oh, because that's how many fields there were on the paper version of the form that preceded the mainframe app."
Which seems insane, until you think it through. There were innumerable downstream processes of that paper form.
Changing the number of fields on the digital version would have cascaded that change downstream to all those processes. In the interest of rapid implementation, the optimal approach was to preserve everything about the form.
And then nobody had a reason to go to the bother to change it for the next 50 years. (And that was a process within a single company!)
But you can split these claims into child claims upon printing. That's the thing with good software, the user model and the internal implementation are completely orthogonal. I think a good example of this is postfix.
100%; customization maintenance burden is underrated - it simply costs a lot of time and energy to customize things; often there are better uses of this time, especially in the business context
Your arguments are totally valid, niche tools will be alive and well. I think my take is that even in niche tools we will see a lot of generalization and more flexible niche tools will eventually win.
The problem is that software can be too flexible. A great example is companies ending up using Excel as a load-bearing database, relying on a bunch of incomprehensible macros to execute critical business logic.
Sure, it's flexible, but are they really better off than a competitor using properly-engineered one-off software? In the end, is there really a difference between software development and flexible-tool-configuration?
I think this is a great argument for flexible code, though it was unclear to me that the author of that post was talking about that.
> The best software around is the one that's opinionated.
I think I might be on the same page as you but I would say that the best software is written to be an environment more than a specific tool. You're absolutely right that you can't solve all problems.
tikhonj jokingly suggests emacs but even as a vim user I fully agree. Like they say, the beauty of it is that the complexity draws from simpler foundations. It is written as an environment rather than just as a text editor. Being written that way lets it adapt to many different situations and is what has kept both vim and emacs alive and popular after all these years. There's a constant in software development: requirements change with time. The point of writing an environment is that you're able to adapt to these changes. So any time you write a tool that tool is built out of that environment. Anything short of that means the tool won't be able to adapt as time marches on.
I definitely agree that writing software like this is hard but I'm not sure if it is harder. It takes more work up front but I'd argue it takes less work in the long run. It's just that in the long run many efforts are distributed across different people and time. But hey, good flexible code also tends to be much easier to read and that's big short term benefit to anyone coming into a mature project.
I don't understand why we would ever want an agent to buy stuff for us.
I understand, for example, search with intent to buy "I want to decorate a room. Find me a drawer, a table and four chairs that can fit in this space in matching colours for less than X dollars"
But I want to do the final step to buy. In fact, I want to do the final SELECTION of stuff.
How is agent buying groceries superior to have a grocery list set as a recurring purchase? Sure an agent may help in shaping the list, but I don't see how allowing the agent to do purchases directly on your end is way more convenient, so I'm fine with taking the risk of doing something really silly.
"Hey agent, find me and compare insurance for my car for my use case. Oh, good. I'll pick insurance A and finish the purchase"
And many of the purchases that we do are probably enjoyable and we don't want really to remove ourselves from the process.
When Amazon came out with the "dash" button and then the "Alexa" speakers, I figured they must have expected they'd get some unintended purchases, and that they'd make more profit from those than they'd lose in the people going through the refund process. (That, or they'd learn whether it was profitable, and eat it as an R&D cost if it turned out to be unprofitable.)
I think this might be similar. In short, it's not consumers who want robots to buy for them, it's producers who want robots to buy from them using consumers dollars.
I think more money comes from offering this value to every online storefront, so long as they pay a fee. "People will accidentally buy your coffee with our cool new robot. Research says only 1% of people will file a return, while 6% of new customers will turn into recurring customers. And we only ask for a 3% cut."
Well, that would be $215.40 since there are 12 months in a year *(taxes, title, fees, regulations, donations to our internal charity system, mandatory 17.7% internet utility fee, and tips for our servers are not included in that total)
That only really follows if you look at "producers" as a homogenous unit, but the companies hyping up their AI browser agents aren't really in the business of running online goods stores
The real answer here is the same as every other "why is this AI shit being pushed?" question: they want more VC funding.
> In short, it's not consumers who want robots to buy for them, it's producers who want robots to buy from them using consumers dollars.
This. Humans are lazy and often don’t provide enough data on exactly what they are looking for when shopping online. In contrast, Agents can ask follow up questions and provide a lot more contextual data to the producers, along with the history of past purchases, derived personal info, and more. I’d not be surprised if this info is consumed to offer dynamic pricing in e-commerce. We already see dynamic pricing being employed by travel apps (airfare/uber).
I suspect part of this is rich people coming up with use cases. If you're rich enough money means nothing but product selection feels like a burden so you have an assistant who does purchasing on your behalf. You want your house stocked with high quality items without having to think of it.
For the rest of us, the idea of a robot spending money on our behalf is kinda terrifying.
> "I suspect part of this is rich people coming up with use cases."
Yes. Having been in the room for some of these demos and pitches, this is absolutely where it's coming from. More accurately though, it's wealthy people (i.e., tech workers) coming up with use cases that get mega-wealthy people (i.e., tech execs) excited about it.
So you have the myopia that's already present in being a wealthy person in the SFBA (which is an even narrower myopia than being a wealthy American generally), and matmul that with the myopia of being a mega-wealthy individual living in the SFBA.
I honestly see this as a major problem with our industry. Sure, this has always been true to some extent - but the level of wealth in the Bay Area has gotten so out-of-hand that on a basic level the mission of "can we produce products that the world at large needs and wants" is compromised, and increasingly severely so.
It's almost like every recent silicon valley product is designed by multi-millionaires whose problems are so out of touch with regular-people problems. "I have so much money and don't know what to spend it on, it would be great if AI could shop for me!" Or "I have so little time, it would be great if an app could chauffeur me around and deliver food for me." Or "It's Christmas again, I need to write heartfelt, personalized letters to 1,000 important clients, partners, friends, and relatives. Why not have an AI write them?"
It's like the endless examples around finding restaurants and making reservations, seemingly as common a problem in AI demos as stain removal is in daytime TV ads. But it's a problem that even Toast, which makes restaurant software, says most people just don't regularly have (https://pos.toasttab.com/blog/data/restaurant-wait-times-and...).
Most people either never make restaurant reservations, or do so infrequently for special occasions, in which case they probably already know where they want to go and how to book it.
> Most people either never make restaurant reservations, or do so infrequently for special occasions, in which case they probably already know where they want to go and how to book it.
And even if they don't know, they likely either live in a small place, in which case there's not going to be a huge amount of choice, or a big place, in which case there will be actual guides written by people whose actual job it is to review restaurants. It really seems like a solution in desperate need of a problem.
This is fascinating to me, as I basically never go to a restaurant without a booking.
But I think you're underestimating this use case, as the data you linked shows that Google is the top referral used by people to find the restaurant/booking website, and once SEO is overtaken by ChatGPT-like experiences it would make sense that "book this for me" would be a one-click (or one-word) logical next step that Google never had.
Thinking about grocery shopping makes me think this need is real, but for poor people.
The amount of time that goes into "what food do we need for this week" is really high. An AI tool that connected "food I have" with "food that I want" would be huge.
Also, consider what enshittification in this area will look like: First year, all the choices are good, second year, it starts picking worse price/value items, then it goes downhill until you finally do it yourself again. Nope thanks
Correct: as soon as you start using an AI to buy things for you, influence over the choices that AI makes becomes an incredibly tantalizing fruit to be auctioned off. And you don't control that AI, a for-profit entity does. It doesn't matter whether it's working well and acting in your best interest now, it's abundantly clear that it won't be very long before it's conspiring against you. You are the product.
The ultimate problem is the incentives. Web stores are already forcing us to use their proprietary (web)apps, where they define all of the software's capabilities.
For example, subscription purchases could be a great thing if they were at a predictable trustable price, or paused/canceled themselves if the price has gone up. But look at the way Amazon has implemented them: you can buy it once at the competitive price, but then there is a good chance the listing will have been jacked up after a few months goes by. This is obviously set up to benefit Amazon at the expense of the user. And then Amazon leans into the dynamic even harder by constantly playing games with their prices.
Working in the interest of the user would mean the repeating purchase was made by software that compared prices across many stores, analyzed all the quantity break / sale games, and then purchased the best option. That is obviously a pipe dream, even with the talk of "agentic" "AI". Not because of any technical reason, but because it is in the stores' interest to computationally disenfranchise us by making us use their proprietary (web)apps - instead of an effortless comparison across 12 different vendors, we're left spending lots of valuable human effort on a mere few and consider that enough diligence.
So yes, there is no doubt the quiet part is that these "agents" will mostly not be representing the user, but rather representing the retailers to drive more sales. Especially non-diligent high-margin sales.
> then it goes downhill until you finally do it yourself again
Good news, usually by the time you reach this point in the cycle, the do-it-yourself option has become super-niche and the stores themselves might not even make that available.
Definitely true, but basic goods and services we have today (e.g. every song nd every movie ever made in your pocket) were unimaginable luxuries even 50 years ago.
Uber started as a chauffeur service, but is now available to everyone and is (mostly) a huge improvement over taxis.
Agent, I need some vitamin D, can you find me the best deal for some rated in the top 5? Agent deployed. Ok we found a bottle with a 30 day supply of Nature’s Own from a well respected merchant. It can be here in 2 days and it is $12. Should I buy? Yes.
Or you could add some other parameters and tell it to buy now if under $15.
Agent, I need a regular order for my groceries, but I also need to make a pumpkin pie so can you get me what I need for that? Also, let’s double the fruit this time and order from the store that can get it to me today.
Most purchases for me are not enjoyable. Only the big ones are.
> Agent, I need some vitamin D, can you find me the best deal for some rated in the top 5?
Ok we found a bottle with a 30 day supply of <producer that paid us money to shill to you>, a Well-Known Highly Rated and Respected Awesome Producer Who Everyone Loves and Is Very Trustworthy™, from <supplier that paid us money to shill to you>, a Well Respected And Totally Trustworthy And Very Good-Looking Merchant™. <suppressing reports of lead poisoning, as directed by prompt>
Supposing I accept that's a likely outcome, it's exactly the same thing that would have happened if a typical human shopper searched for Vitamin D and picked the top result, right?
The cynicism on these topics is getting exhausting.
> Supposing I accept that's a likely outcome, it's exactly the same thing that would have happened if a typical human shopper searched for Vitamin D and picked the top result, right?
Yeah sure, but humans (normally) only fall for a particular scam once. Because LLMs have no memory, they can scale these scams much more effectively!
Not to out myself as, like, a total communist, or something, but I think there should be government regulations preventing lead-laced Vitamin D pills with no Vitamin D in them from being sold.
Does anyone actually buy this way? For anything that isn’t groceries, I check, particularly now that Amazon has roughly the same trust as temu.
Vitamin d? I’m going to check the brand, that it’s actually a good quality type. It’s a 4.9 but do reviews look bought ? How many people complain of the pills smelling? Is Amazon the actual seller?
As for the groceries, my chain of choice already has a fill order with last purchases button, I don’t see any big convenience that justifies a hallucination prone ai having the ability to make purchases on my behalf.
You will have a 3rd party agent, in your home, that you get your news and information from, controlled by a for-profit entity, literally conspiring against you, the product, to squeeze you for every cent in your bank account, to put you in debt, to funnel your money directly to its masters. A Grima Wormtongue at your shoulder at all times, making your decisions for you, controlling your access to information, a slave to a company whose entire goal is to capture your attention and money and prevent you from ever learning anything negative about anyone who pays them money, and ever learning anything positive about anyone who they don't like. And you're going to make completely rational decisions?
Why do we all keep making the same obvious mistakes over and over? Once you are the product, thousands of highly paid experts will spend 40+ hours per week thinking of new ways to covertly exploit you for profit. They will be much better at it than you're giving them credit for.
It is even the same people pushing this. The people who made money spying on and manipulating our social lives and then selling that ability to advertisers are exactly the same people now pumping billions into AI.
How so? Search was way less capital intensive than AI to develop. We started with dozens of search engines back in the 90s and we still ended up with a near monopoly.
Edit: All major AI companies have millions if not billions of funding either from VCs or parent companies. You can't start an AI company "in your garage" and be "ramen profitable".
Edit 2: You don't even need to monopolize anything. All major search engines are ad-driven and insert sponsored content above "organic" search results because it's such an obvious way to make money from search. So even if there wasn't a product monopoly, there's still a business model "monopoly". Why would the same pattern not repeat for "sponsored" purchases for agentic shopping?
> but I also need to make a pumpkin pie so can you get me what I need for that
Have you actually baked a pumpkin pie? There are numerous versions, and the distinction between them is cultural. There is zero chance an AI will understand what kind of pumpkin pie you want, unless you are talking about the most general case in your region. In this case why even bother doing it yourself?
Yes, you can teach it the recipe beforehand, but I think it is too complex to tech the AI the details of every task you want it to perform. Most likely what will happen is AI will buy you whatever is more profitable for corporations to sell.
And there will be number of ways (and huge amount of money to make) to ensure that your open-weights self-hosted model will make the right choices for the shareholders as well.
Yeah, agree most daily purchases are humdrum and shouldn’t command all of my attention.
Incidentally, my last project is about buying by unit price. Shameless plug, but for vitmain D the best price per serving here (https://popgot.com/vitamin-d3)
I think the main driving force is it’s a way to monetize an LLM. If the LLM is doing the buying then a “buyer fee” can be tacked on to the purchase and paid to the LLM provider. That is probably an easier sell than an ongoing monthly subscription.
Also, sellers can offer a payment to the LLM provider to favor their products over competitors.
That's kinda backwards, tho. You don't, or at least shouldn't, say "we have this thing, and we need to make people use it, so let's make up use cases even if they make no sense and will fail". (People do this all the time, of course; it's more or less the sunk-cost fallacy.)
"[Our model] can make mistakes. Double-check the answers"
footer to
"[Our model] can make mistakes. Double-check the answers. May insert affiliate links which pay us commission."
And I'm speaking as someone building an AI agent. Zero plans of doing this because we have a completely different business model, but still, it wouldn't be the biggest most expensive business decision to make.
Let's say even if I always buy "Deodorant X", I might instruct my agent every month to go out and buy it from the cheapest place. So I wouldn't do it for "any chairs" but the usual purchase from a certain brand, I can see myself automating this. In fact, I have because I use Subscribe & Save from Amazon, but sometimes things are cheaper on the brand's website or some other marketplace.
The issue of course is that true-deodorant-x.com is not a trustworthy website and will sell you knock-offs, "Deodorant X-Killer-Y" is a separate brand that LLMs might mistake for Deodorant X upon purchase, and Amazon mixes the supply of Deodorant X with fake products anyway.
You're basically rolling the dice with every single refill.
Agreed: If I was working with a human interior designer I would still want them to provide me a curated list of options on what decor to buy. Blindly trusting a person seems risky, a robot even more so.
Exactly. If we really wanted AI to help us, we would find a way to fix the inscrutable problem of why we have to enter our ID number over a phone, then do it a second time when we connect to a human. No company in the world has solved this riddle.
I’ve had two cases for this in the last month. Not that I have access to an agentic browser.
* We decided to buy a robot vacuum, again. And we decided on a particular model that yo-yo‘s up and down in price by about $200 every month. We ended up buying it off of Amazon because of camelcamelcamel, but if I could have easily tracked prices and bought elsewhere, I would’ve. And I would’ve considered using an antigenic browser to do that for me – if I could trust them at all. One model number and I know the price I wanna pay, I just don’t want to check a bunch of storefronts everyday
* kids going back to school – and he has a school supply list. He’s up for a new backpack and a new lunchbox, and a bunch of back to school clothes - so those we’ve actually been shopping for all summer. But the wooden ruler, the three sheafs of college rule paper, etc. I don’t wanna shop for. I actually had chatGPT scan the paper list, and then get me either direct links, or links to searches on walmart.com (they are more than an hours drive from us, but they do deliver to my wife’s work). Then I created a cart and had them deliver. ChatGPT solutions were not bad, I only switched one or two items for a version my kid should have versus a version I should buy. In the moment, I probably would have trusted a bot to do this, though retrospectively I’m glad it went the way it did
That's because you don't trust the agent, and for good reasons considering the article.
But if you trust the agent, why not let it do the final step? You will accept anyways. Imagine you have a car mechanic you trust, you can just ask him "hey, fix my car" and let him buy whatever parts he needs on your behalf. If he quoted you first, you would say "yes" anyways, so skip that step and get your car repaired as soon as possible. Only if you don't trust him you will ask for a quote and review it, which, if the mechanic is trustworthy is a hassle for both of you.
Some purchases are enjoyable, most of them are not. I don't enjoying doing the groceries. And to continue with the car mechanic theme, I don't really enjoy buying new tires, though I know some people do. So I just ask my mechanic: "if the tires are worn, change them, give me the ones you think are the best". I will probably end up with the most boring option: the same model as before, which is the one recommended by the manufacturer, and that's perfect for me.
I don't personally have an issue with this use case. It just has to work as well as if you told a trusted assistant or friend to do it for you. Needs discrimination and needs to intelligently include or exclude you from the loop based on the circumstances
Imagine an agent being a roommate. They see that toilet paper is running out, they go to the supermarket, they buy more, they charge you money. All without you saying a word. Sure, it might not be your favorite brand, or the price might not be optimal, but realistically, the convenience of not having to think about buying toilet paper is definitely worth the price of having your roommate choose the details. After all, it's unlikely they'll make a catastrophically bad decision.
This idea has been tried before and it failed not because the core concept is bad (it isn't), but because implementation details were wrong, and now we have better tools to execute it.
The idea has been tried before and it failed because people don’t actually want this product at the scale the inventors thought. Amazon has never stopped doing this. Adding an element of indeterminism to the mix doesn’t make this a better product. Imagine what the LLM is going to hallucinate with your credit card attached.
The second someone has to deal with returning an item that an LLM hallucinated they needed is the second they cancel this service. The hassle of returning something vastly outweighs any perceived convenience of not having to buy toilet paper or whatever the next time you're at the grocery store. Hell, you can buy toilet paper in 10 seconds on your phone while sitting on the toilet from Amazon, and that's been possible for probably 15 years. What problem is this actually solving?
Sure, but why would you use a nondeterministic LLM for that?
LLMs can do things that we cant reasonably do with deterministic software. But everything that can be done deterministically should be done deterministically.
Because for vast majority of people, LLM is a superior interface over Raspberry Pi with crontab running a Python script with headless Firefox. Hard to believe, I know.
if my roommate charged me for toilet paper they picked out I would want to talk to them about the brand they go for and other details, at which point a lot of the overhead is back isn't it?
That's sort of the tradeoff, though. You get the convenience of having tp show up without having to to through the steps of shopping for it. Except in extreme cases, it seems likely your roommate will pick something that is effectively a commodity at a reasonable price. If you want granular control over brand, features, and pricing, you'll have to pay for it in time and/or money.
I recently bought a blood pressure monitor from Amazon, and it took two days because I couldn't buy it at work, can't buy it while driving, couldn't get it at the gym, and was too tired to look into it when I was at home.
The idea of an agent, not owned by the store (who may try to upsell me) that could look into the product and buy one sounds great. Instead of waiting two days I could have just told the AI to run the errand for me while I was at work. I don't know anything about blood pressure monitors and I don't want to learn, so as long as it's <$50 any choice is fine.
> How is agent buying groceries superior to have a grocery list set as a recurring purchase?
I could see an interesting use case for something like "Check my calendar and a plan meals for all but one dinners I have free this week. One night, choose a new-to-me recipe, for the others select from my 15 most commonly made dishes. Include at least one but at most 3 pasta dishes. Consider the contents of my pantry, trying to use ingredients I have on hand. Place an order for pickup from my usual grocery store for any ingredients necessary that are not already in the pantry"
This has been the dream driving smart refrigerators for literally decades: if you know what food they have, you could sell them ingredient li so they could take their existing theta and digeut and make dish sha. Advertisers have wanted this for a long time. But no one has found a use case that is actually compelling to customers to get them to buy such a refrigerator. This is actually similar to the Alexa: Amazon invested in the project expecting there to be a lot of purchases through it, but mostly it gets used as a timer or to play music and not much purchase volume goes through it.
Maybe people will accept ubiquitous digital surveillance enough that they accept someone else knowing what they have in their pantry and refrigerator, but so far it isn't a thing.
Who follows recipes to produce every meal that they eat, anyway? I just look in the fridge, select some vegetables and a protein, and bang that together into something edible using spices or condiments most nights. But I'm not going to outsource that to my fridge because I'm a lot faster at thinking through all of that then my fridge would be. I do it entirely without thinking.
This seems like something that won't ever work because there's like 10 decisions that the computer has to make that it can't possibly know unless it's either a mind reader or has a crazy level of surveillance on your life. How does it know what's in your fridge or pantry and the quantities of each item? How does it know how many people you're cooking for? What if your kids or spouse aren't going to be home a given night -- do they all have their own calendars that are impeccably maintained and synced to yours? What's your budget and do you really trust it spending your money? What if there are several options for each ingredient, how does it know your preference? Perhaps you prefer to buy certain ingredients at Costco and were planning on making a trip tomorrow, how does it know not to order stuff you buy from Costco in your grocery order?
Even if it could figure everything out, is this a problem that people actually have? I'm not even being facetious, but you're describing someone who cares enough to spend time cooking and clearly has a preference on what they want to make, but doesn't care enough to actually select the specific dishes.
Yeah, it's a very odd proposition. Even if it were guaranteed not to screw it up (and of course in reality LLMs are experts in screwing stuff up), this still doesn't seem like an appealing idea.
I could see some charm to something to go through intentionally annoying and confusing checkout processes, booking a Ryanair flight, say. I'm fairly sure that an LLM would end up falling for their car hire/insurance/whatever upsells, tho. There's a reason that that checkout process is annoying.
The proper way to do this is to first perfect AI at spotting all the dark patterns and ads that exist today, and automatically filter all of that shit (including never buying anything ever from companies that have done SPAM a single time in their entire history).
Then figure out what the real human wants ahead of time, and it can go out and find the best deal / best value / best long term reliable company / whatever the HUMAN wants...
If you were a lawyer, you’d think something slightly different when you heard the word agent than you would if you were a computer guy. The delta is the fact that under the law of agency, an agent has the power to bind the principal to a contract.
If the lawyers didn’t have this definition in their head there would be no drive to make the software agent a purchaser, because it’s a stupid idea.
I believe half of the comments here are just dumping on AI-related ideas because they see it as their duty to counter the hyperbolic claims about capabilities being tossed around.
I enjoy reading both sides of the argument when the arguments make sense. This is something else.
I think it has more to do with the various new meanings that have been attached to the word "agent" and the concept of "agency" by software and some parts of west coast culture. Those concepts do not really have much to do with the law of agency.
Lawyers don't come up with good ideas; their role is to explain why your good ideas are illegal. There's a good argument that AI agents cannot exercise legal agency. At the end of the day, corporations and partnerships are just piles of "natural persons" (you know, the type that mostly has two hands, two feet, a head, etc.).
The fact that corporate persons can have agency relationships does not necessarily mean that hypothetical computer persons can have agency relationships for this reason.
> I think it has more to do with the various new meanings that have been attached to the word "agent" and the concept of "agency" by software and some parts of west coast culture.
Indeed, agency (the capability to act) and autonomy (the freedom to choose those actions) are separate things.
BTW, attorneys' autonomy varies, depending on the circumstances and what you hired them to do. For example, they can be trustees of a trust you establish.
> I don't understand why we would ever want an agent to buy stuff for us.
Why not? Offload the entire task, not just one half of it. It's why many well-off people have accountants, assistants, or servants. And no one says "you know, I'm glad you prepared my taxes, but let me file the paperwork myself".
I think what you're saying isn't that you like going through checkout flows, just that you don't trust the computer to do it. But the approach the AI industry is "build it today and hope the underlying tech improves soon". It's not always wrong. But "be dependable enough to trust it with money" appears to be a harder problem than "generate images of people with the right number of fingers".
No doubt that some customers are going to get burned. But I have no doubt that down the line, most people will be using their phones as AI shoppers.
Comparing regular people's shopping to the super-wealthy is absurd. Regular people care, possibly quite a lot, about costs and cost/benefit ratios. To the super wealthy the cost of most regular goods is entirely irrelevant. Whether their yogurt supply is 10 dollars a month or 200 dollars a month makes no difference to them. But it makes a huge difference to the vast majority of people. Even people who would be happy to pay the premium for very good yogurt will want a very good experience from this.
You can say that about 99% of the tech that people use today. Windows and MacOS don't serve you. Your browser doesn't serve you. Heck, Hacker News doesn't serve you - it serves a bunch of VCs!
But the reality is that most of the time, this is not an adversarial relationship; and when it is, we see it as an acceptable trade-off ("ok, so I get all this stuff for free, and in exchange, maybe I buy socks from a different company because of the ads").
I'm not saying it's an ideal state or that there are no hidden and more serious trade-offs, but I don't think that what you're saying is a particularly compelling point for the average user.
Many people on this forum might agree with the statements that Windows (increasing ads, tracking and bloat) and your browser not serving you (Chrome Manifest V3, etc.)
Adversarial relationships can and will happen given the leverage and benefits; one only need to look at streaming services where some companies have introduced low-tier plans that is paid for but also has ads.
I think it's also more a generic wish to have agents do things without review, this would open up a lot bigger window of possibilities. If it fails at easy shopping, then more crucial decision making is out of the order.
Limited time to buy would be one reason. Another one would be dynamic nature of certain merchendize. Recurring purchase is static, but if I want tomato of specific kind, there can be endless array of options to choose from.
It's not looking good so far. When OpenAI introduced product searches back in April I tried running one of their own example queries from the announcement post, and it obliviously cited "reviews" and "recommendations" from LLM-generated affiliate link farms. I just tried it again and it still falls into the same trap.
One other ancillary benefit is no more “impulse” buying. Unless of course the AI gets incentivized to do it, it will then bubble that impulse buy up to the consumers UI.
I imagine the exact opposite is far more likely - there will be a button for "get your AI agent to consider us!" that will be even easier to just click, since you know it won't just lead to an immediate purchase - but they know very well it will lead to a purchase down the line.
Lots of senior executives, celebrities, etc have other people buy stuff for them all the time - flights, gifts, lunch, etc. The problem is this is very expensive so not available to most people. If agents reduce the cost and are mostly reliable, there will be a significantly large market for this service.
I agree. I'm confused that this idea is even controversial. I would absolutely use it in the way you're describing. I've wanted something like this since around when Alexa/Echo launched.
You're looking at individual generations. These tools aren't for casual users expecting to 1-shot things.
The value is in having a director, editor, VFX compositor pick and choose from amongst the outputs. Each generation is a single take or simulation, and you're going to do hundreds or thousands. You sift through that and explore the latent space, and that's where you find your 5-person Pixar.
Human curated AI is an exoskeleton that enables small teams to replace huge studios.
Is there any example of an AI generated film like this that is actually coherent? I've seen a couple short ones that are basically just vibe based non-linear things.
Some of the festival winners purposely stay away from talking since AI voices and lipsync are terrible, eg. "Poof" by the infamous "Pizza Later" (who is responsible for "Pepperoni Hug Spot") :
Stock images. I’ve already seen trining courses (for compliance reasons) using AI videos. A bit cringey, but I imagine cheaper than shooting real people.
Very confusing to read the article labelled as 1998 and have references for newer stuff (e.g. Ratatouile).
The biggest one for me is to recommend a bunch of 98-propiate languages (C++) and then recommend Go!
I guess that the article has been slightly updated, but it felt weird. In another language I checked the references are older.
Interviewing is a skill. I think that he got really good at it, both in tailor his CV as well as the process itself. It probably worked on the process and make adjustments. Based on the what I’ve read, it seems that he was interviewing all the time.
So, IMHO, he focused in the interview process over everything else, including understanding and exploiting the blind spots. He iterated and refined it, until, he became a master of it.
He really seemed to be not great at the work itself, though. He was being fired after a couple of days, which I don’t think is common. You really have to do it very badly to be fired so quick after being hired.
This is what happens at top consulting firms. Top 5% interview skills, especially as smoothing your way through a case study. But when they actually come consult for your business, it’s just not what you pay for given what you’ve been led to believe with those case studies.
Code should be generally written so it's easy to read.
reply