I worked in automated customer support, and I agree with you. By default, we automated 40% of all requests. It becomes harder after that, but not because the problems the next 40% face are any different, but because they are unnecessarily complex.
A customer who wants to track the status of their order will tell you a story about how their niece is visiting from Vermont and they wanted to surprise her for her 16th birthday. It's hard because her parents don't get along as they used to after the divorce, but they are hoping that this will at the very least put a smile on her face.
The AI will classify the message as order tracking correctly, and provide all the tracking info and timeline. But because of the quick response, the customer will write back to say they'd rather talk to a human and ask for a phone number they can call.
The remaining 20% can't be resolved by neither human nor robot.
Between the lines, you highlight a tangental issue: execs like Zuckerberg think easy/automatable stuff is 90%. People with skin in the game know it is much less (40% per your estimate).This isn't unique to LLMs. Overestimating the benefit of automation is a time-honored pastime.
I’ve noticed this when trying to book a flight with American Airlines earlier this year. Their website booking was essentially broken, insisting that one of my flight segments was fully booked but giving no indication of which one and attempting alternate bookings which replaced each of the segments in turn still failed. They’d replaced most of their phone booking people with an AI system that also was nonfunctional and wanted to direct me to the website to book. After a great deal of effort, I managed to finally reach a human being who was able to place the booking in a couple minutes (and, it turned out, at a lower price than the website had been quoting).
This reminds me how Klarna fired their a large part of their customer support department to replace it with ai, only to eventually realize they couldn't do the job primarily using ai and had to rehire a ton of people.
OT: just googled that name, info panel on the right in my language settings categorizes it as "金融の連鎖", or "cascading of finances". am not sure how to take that.
In fairness to Klarna, the interest rates they charge are typically low or even zero. The problem is more that they're encouraging poor people to waste money on things they probably shouldn't buy in the first place, like expensive concert tickets or consumer electronics.
I used to work at a competitor to klarna so take this with a grain of salt, but the zero interest rates aren't really zero. They finance either by klarna eating into their runway, or by the business paying the interest up front. Which usually leads to higher prices for everyone, regardless of you using klarna or not
Their business model is an online payment provider (like e.g. PayPal/apple pay) that splits the payment into 3, 6 or 12 monthly payments, usually at 0% interest
The idea being that for the business the loss in revenue from an interest free loan is worth it if it causes an increase in sales
Yeah I think I do already see this happening in my work. It's clearly very beneficial, but its benefit is also overestimated. This can lead to some disenchantment and even backlash where people conclude it's all useless.
But it isn't! It's very useful. Even if it isn't eliminating 90% of work, eliminating 40% is a huge benefit!
I never call a customer service line unless the website doesn't work, but customer service robots try very hard to get me to hang up and go to the website.
It's super frustrating. These robots need to have an option like "I am technically savvy and I tried the website and it's broken."
Do you know why your isp asks you to unplug and plug your modem back in while on call, even if you insist you did that already? A surprising large number of people don’t even realize their modem isn’t even plugged in at all.
> But because of the quick response, the customer will write back to say they'd rather talk to a human
Is this implying it's because they want to wag their chins?
My experience recently with moving house was that most services I had to call had some problem that the robots didn't address. Fibre was listed as available on the website but then it crashed when I tried "I'm moving home" - turns out it's available in the general area but not available for the specific row of houses (had to talk to a human to figure it out). Water company, I had an account at house N-2, but at N-1 it was included, so the system could not move me from my N-1 address (no water bills) to house N (water bill). Pretty sure there was something about power and council tax too. With the last one I just stopped bothering, figuring that it's the one thing that they would always find me when they're ready (they got in touch eventually).
The world is imperfect and we are pretty good at spotting the actual needle in the haystack of imperfection. We are also good at utilizing a whole range of disparate signals + past experience to make reasonably accurate decisions. It'll take some working for AI to be able to successfully handle such things at a large scale - this is all still frontier days of AI.
They don’t care about you. You are a number on a screen that happens to pay their company money sometimes. But by using recorded voices, the company hopes to tap into the empathetic part of your human brain to subconsciously make excuses for their crappy service.
When I get stellar customer service these days, I’m happy and try to call it out, but i don’t expect it anymore. My first expectation is always AI slop or a shitty phone tree. When I reframed it for myself, it was a lot easier not to get frustrated about something that I can’t control and not blame a person who doesn’t exist.
> They don’t care about you. You are a number on a screen that happens to pay their company money sometimes.
Actually that reminds me, I couldn't figure out how to cancel my old insurance online and couldn't get to a person on the phone - I just deleted the direct debit, and waited until they called me to sort it out.
> A customer who wants to track the status of their order will tell you a story about how
I build NPCs for an online game. A non trivial percentage of people are more than happy to tell these stories to anything that will listen, including an LLM. Some people will insist on a human, but an LLM that can handle small talk is going to satisfy more people than you might think.
A customer who wants to track the status of their order will tell you a story about how their niece is visiting from Vermont and they wanted to surprise her for her 16th birthday. It's hard because her parents don't get along as they used to after the divorce, but they are hoping that this will at the very least put a smile on her face.
The AI will classify the message as order tracking correctly, and provide all the tracking info and timeline. But because of the quick response, the customer will write back to say they'd rather talk to a human and ask for a phone number they can call.
The remaining 20% can't be resolved by neither human nor robot.