Just because the citation exists, what the LLM says it stands for and what it actually stands for are not the same.
For testing, I've asked (admittedly last-gen) LLMs to generate legal opinions regarding issues in commercial English civil litigation, and I received back cases where the citation is real, but the area of law (family law) is not relevant as family courts apply a very different set of procedural rules.
(If you squint a bit, they sometimes might be relevant... and could be useful for a particularly creative litigator to make a novel argument on behalf of a very risk tolerant client. But you would very much want to go read those cases and think quite hard about them.)
Right, I know what you mean. If the parties are only breezing over the motion then it looks great and 95% of the time you'll get away with it, even though really it's ethically dubious. And that's a super hard one for a human to catch when reviewing LLM output. Especially because (certainly for me) you tend to get lazier and lazier reviewing the LLM output as they get "smarter."
I'm assuming you've just used some off-the-shelf ones like Claude or GPT? All the lawyers I know are just using those. I'd love to know what Lexis and Westlaw and other companies are serving that might mitigate some of these issues with better custom tuning or a better harness.
I have tested Lexis AI once for a legal research point. I wasn't particularly keen on putting the exact details of an actual problem in, but I gave it a summary version.
It didn't feel drastically different from using ChatGPT with the ability to search the web, except it was searching material on Lexis, both statute/case law and commentary. It dug out some commentary that confirmed my prior hunches, but also pointed to some cases that weren't in any way relevant.
Otherwise, all the experimentation I've done is with non-confidential material using public LLMs.
OK, sounds like they've basically done nothing more than attach an MCP with all the case law. I was hoping all that cash they make from monetizing the commons would be used to at least create a decent legal LLM.
It'll take another company eating their lunch for them to wake up.
The answer is: the market will work it out eventually. Clients will push for more work to be fixed-fee/outcome-based rather than billed hourly. There'll be some small firms who'll successfully grab lots of lower value clients who are willing to use digital tools to handle their work and don't particularly care about having a big fancy office in London or New York if it means lower bills (and they can then basically use the relationship they've had providing the supervised online service to be the first point of call when said client wants something that's less off-the-shelf and needs more work).
Also, an interesting example: in English litigation (where, broadly, loser pays unlike America where each side pays), maximising billable hours is not always a viable strategy for anybody if those costs aren't recoverable on success. Someone involved in large-scale commercial litigation involving disclosure of millions of documents who doesn't use algorithmic document classification (now pretty broadly accepted as normal) potentially runs the risk of a judge determining that the costs of going through all the documents by hand isn't recoverable. Insurers/litigation funders aren't going to want to risk padding the costs so much that the judge prevents them from recovering their stake in the litigation.
Customers using their own LLMs: yep, they might do that. I think the pitch from the legal LLM providers is "we've got legally trained people doing RLHF to make it more accurate" mixed in with "also we've got a partnership with Lexis/Westlaw/etc. so we can do legal research that's better than what's on the open web", with a little bit of "if you get sued for professional negligence, 'I used the legal AI thing that's built into Westlaw' is gonna be more convincing to a judge and jury (and your insurance company) than 'I used ChatGPT, yes, like the app you've got on your phone'...".
You then get lots of interesting exceptions and difficulties in applying that rule: who pays when you have multiple potential defendants (but some of them have died or gone out of business or are in a different jurisdiction), or there are fiddly causation issues, or the defendant is an organisation that's vicariously liable for their employee/contractor/authorised religious leader/any number of other relationships, or there's insurance involved, or there's some third party interest, or the purportedly tortious act was mandated by some law or other obligation, or the government decides to set up some kind of alternative process of compensation which may or may not indemnify the tortfeasor, and so on and so on.
When dealing with the not-so-straightforward cases, appellate courts do look at questions like "is this a fair and equitable distribution of the cost?" and legal scholars compare the pros and cons of tort liability to other mechanisms (compensation by the government, industry self-regulation, no-fault/strict liability, mandatory insurance etc).
Cloud is probably the better comparison, since crypto never had the sort of mainstream management buy-in that the other two got. Microsoft's handling of OneDrive in particular foreshadows how AI is being pushed out.
i dont like onedrive very much. i get it its useful as a pigeonhole, what i really dont like is how it is used. its the thing that moves files to onedrive and destroys local copies, that i hate, and onedrive is something that enables that. so i dont hate onedrive, i just dont like it.
I have never received a Crypto spam email from any place where I opted out from it. Same for cloud. It feels different. With crypto it was everyone wanting to ride the hype train. With AI they spent a bunch of money up front and are desperate to see ROI.
Yep. At one point I expected the software I needed to work for a reasonable time range, possibly up to a decade. Best if you could buy it once and use it from then on.
Now crap has turned into revenue sucking subscriptions, at most yearly licensing, feature flutter. And the worst is being bought up by VC/PE and milked for anything useful and thrown away.
> Now crap has turned into revenue sucking subscriptions
So much this. Each subscription is literally a small percentage of your revenue. You can't reinvest it ... it's just gone. Hopefully it enables more productivity ... but most likely, it is only marginal.
Votes close at 10pm. Might be a few stragglers left in the queue, so call it 10:15pm. (Exit poll results are embargoed until 10pm.)
Ballot boxes are transferred from individual polling station to the location of the count. The postal votes have been pre-checked (but the actual ballot envelope has not been opened or counted) and are there to be counted alongside the ballots from the polling stations.
Then a small army of vote counters go through the ballots and count them and stack together ballots by vote. There are observers - both independent and appointed by the candidates. The returning officer counts the batches up, adjudicates any unclear or challenged ballot, then declares the result.
The early results come out usually about 1 or 2. The bulk of the results come out about 4 or 5. Some constituencies might take a bit longer - it's a lot less effort to get ballot boxes a mile or two down the road in a city centre constituency than getting them from Scottish islands etc. - but it'll be clear who has the majority by 6 or 7 the next day.
I can appreciate that the US is significantly larger than the UK, but pencil-and-paper voting with prompt manual counts is eminently possible.
The actual data is being held by GPs, hospitals, other secondary care providers, and pharmacies. Enough of those providers use systems that all conform to a bunch of common standards and APIs that the NHS app can get that data (and the idea is it puts pressure on the remainder to switch to systems that are accessible).
The capabilities the NHS app offers will depend on what subset of the functionality the GP practice has implemented (on, in reality, the commercial vendor that makes the software they use).
For testing, I've asked (admittedly last-gen) LLMs to generate legal opinions regarding issues in commercial English civil litigation, and I received back cases where the citation is real, but the area of law (family law) is not relevant as family courts apply a very different set of procedural rules.
(If you squint a bit, they sometimes might be relevant... and could be useful for a particularly creative litigator to make a novel argument on behalf of a very risk tolerant client. But you would very much want to go read those cases and think quite hard about them.)
reply