> There are many things wrong with Canada. It has a go for bronze mentality, the smartest of us keep going to the US because there is not enough opportunity here, much of its public infrastructure is crumbling and the housing prices are frightful. The nation is very obviously sick.
Is anyone currently moving from Canada to the US?
If so, are they the "smartest", or do they simply have different priorities than a lot of equally smart people?
Virtually all of the top performers at my school left for the USA immediately after graduation.
I think somewhere between 70-90% of Waterloo graduates in CS leave every year.
Turns out doubling or tripling your take home compensation is absolutely worth it.
You can buy a house instead of renting an apartment with roommates. You can afford to marry and have children. You can buy all the things the government would've provided you had it not been dysfunctional.
Plus, there are just more jobs in SWE in the USA. Many of my classmates graduating last year in June are still unemployed since you have to be exceptional to get a job here.
Pretty much anyone who can get TN1/H1B/L1B does, unless you were born wealthy, have an extreme sense of patriotism, or have a very strong attachment to family.
That's definitely true, though probably only limited to CS (and maybe the top 5% of people going to investment banking). The vast majority of the most intelligent people at any Canadian university will stay in Canada, and given the current political situation, I think that's probably only going to become more true.
Also, anecdotally, I would wager that schools with significant numbers of Americans (e.g., McGill) probably have more US students staying in Canada than vice versa at this point (with perhaps the exception of CS).
> Virtually all of the top performers at my school left for the USA immediately after graduation. [...] Turns out doubling or tripling your take home compensation is absolutely worth it. [...] You can buy a house instead of renting an apartment with roommates. You can afford to marry and have children. You can buy all the things the government would've provided you had it not been dysfunctional.
And how does the "dysfunction" of the current Canadian government compare to what is happening in the US, in your eyes?
> Plus, there are just more jobs in SWE in the USA.
There is the rational answer... for graduates in software.
I took issue with this too, but chose to interpret it charitably. It’s true that a lot of our most qualified people move to the US because of the money.
That questionable-sounding stunt by the media outlet wasn't comparable: Google/Alphabet knows much more about individuals than addresses, salary, and political donations.
Google/Alphabet knows quite a lot about your sentiments, what information you've seen, your relationships, who can get to you, who you can get to, your hopes and fears, your economic situation, your health conditions, assorted kompromat, your movements, etc.
Schmidt is actually from OG Internet circles where many people were aware of privacy issues, and who were vigilant against incursions.
But perhaps he had a different philosophical position. Or perhaps it was his job to downplay the risks. Or perhaps he was going to have enough money and power that he wasn't personally threatened by private info that would threaten the less-wealthy.
We might learn this year, how well Google/Alphabet protects this treasure trove of surveillance state data, when that matters most.
It was probably a decade ago and I recall using something within Google that would tell you about who they thought you were. It profiled me as a middle eastern middle aged man or something like that which was… way off.
If I were extremely cynical, I would suspect they might have intentionally falsified that response to make it seem like they were more naive than they actually were.
I suspect the more likely scenario is they don't actually care how accurate these nominal categorizations are. The information they're ultimately trying to extract is, given your history, how likely you are to click through a particular ad and engage in the way the advertiser wants (typically buying a product), and I would be surprised if the way they calculate that was human interpretable. In the Facebook incident where they were called out for intentionally targeting ads at young girls who were emotionally vulnerable, Facebook clarified that they were merely pointing out to customers that this data was available to Facebook, and that advertisers couldn't intentionally use it.[0] Of course, the result is the same, the culpability is just laundered through software, and nobody can prove it's happening. The winks and nudges from Facebook to its clients are all just marketing copy, they don't know whether these features are invisibly determined any more than we do. Similarly, your Google labels may be, to our eyes, entirely inaccurate, but the underlying data that populates them is going to be effective all the same.
I think its their currently targeted ad demographic or whatever. Its probably a "meaningless" label to humans, but to the computer it makes more sense, he probably watches the same content / googles the same things as some random person who got that label originally, and then anyone else who matched it.
male, lives in this region, has an income between X to X+40000, and has used the following terms in chat or email, regardless of context, in the last 6 months: touchdown, home run, punt, etc. etc.
the ad game is not about profiling you specifically, it's about how many people in a group are likely to click and convert to a sale; they're targeting 6 million people, not you specifically, and that's balanced by how much the people who want the ads are willing to pay.
palantir or chinese social credit, etc., is targeting you specifically, and they don't care about costs if it means they can control the system, forever.
The idea that Google’s lack of knowledge of you a decade ago is somehow related to what they know today is naive. Dangerously naive, I would say. Ad targeting technology (= knowledge about you) is shocking good now.
Color me unconvinced. Google can't even figure what language I speak even though I voluntarily provide them the information in several different ways. I can't understand half the ads they serve me.
Google doesn't choose what ad to show you. Google serves up a platter of details and auctions the ad placement off to the highest bidder.
That platter of details is not shown to you, the consumer.
What you are experiencing is that your ad profile isn't valuable to most bidders, ie you don't buy stuff as much as other people do, or your ad profile is somehow super attractive to stupid companies that suck at running ads who are overpaying for bad matches.
It is not evidence that google knows nothing about you.
Google is pleased that you think they don't know you. It helps keep the pressure down when people mistake this system for "Perfectly target ads". The system is designed to make google money regardless of how good or bad their profile of you is.
It's not just the ads though. Am I to think that Youtube helpfully replacing a video title (whose original text I understand) by a half-assed translation into a language that I don't speak is actually Alphabet playing 5D chess ? If so, hats off to you, Google. I totally fell for it.
If it was his job to downplay the risk's then he absolutely deserved at least this.
Google or any other US company will not be defending your's or anyone's else's data. It's not only that they doesn't want to(which they dont) but they simply can't.
You must comply with the law and you do not want to currently piss off anyone's at the top.
> Schmidt is actually from OG Internet circles where many people were aware of privacy issues, and who were vigilant against incursions.
> But perhaps he had a different philosophical position. Or perhaps it was his job to downplay the risks
I feel that as the consumer surveillance industry took off, everyone from those OG Internet circles was presented with a choice - stick with the individualist hacker spirit, or turncoat and build systems of corporate control. The people who chose power ended up incredibly rich, while the people who chose freedom got to watch the world burn while saying I told you so.
(There were also a lot of fence sitters in the middle who chose power but assuaged their own egos with slogans like "Don't be evil" and whatnot)
Yeah, being unaffected by social pressure when philosophizing about what is moral and liberating is strongly related to being unaffected by social pressure regarding personal hygiene and social norms, unfortunately. Still I'd rather have the weirdos, especially this one particular weirdo, than not! Stallman has blazed the trail for us slightly-more-socially-aware types to follow, while we look/act just a little more reasonable.
There's a popular video on YouTube of him eating skin peeled from his foot during a lecture at a college. Not AI, very old, repellant to normal people.
I'm a bit awestruck. Was there any discussion about it among your peers? We might be a generation or two apart, I saw that video when I was not yet an adult and it might have been literally part of my introduction to the person that is Richard Stallman. It definitely wasn't a good first impression.
Yes, I remember that period of conscious choice, and the fence-sitting or rationalizing.
The thing about "Don't Be Evil" at the time, is that (my impression was) everyone thought they knew what that meant, because it was a popular sentiment.
The OG Internet people I'm talking about aren't only the Levy-style hackers, with strong individualist bents, but there was also a lot of collectivism.
And the individualists and collectivists mostly cooperated, or at least coexisted.
And all were pretty universally united in their skepticism of MBAs (halfwits who only care about near-term money and personal incentives), Wall Street bros (evil, coming off of '80s greed-is-good pillaging), and politicians (in the old "their lips are moving" way, not like the modern threats).
Of course it wasn't just the OG people choosing. That period of choice coincided with an influx of people who previously would've gone to Wall Street, as well as a ton of non-ruthless people who would just adapt to what culture they were shown. The money then determined the culture.
Sorry, I didn't mean to write out the hacker collectivists. I said "individualist" because to me hacking is a pretty individualist activity, even if one's ultimate goal is to contribute to some kind of collective. Or maybe I just don't truly understand collectives, I don't know.
But yes, individualists and collectivists mostly cooperated and coexisted. I'd say this is because they were merely different takes on the same liberating ground truths. Or at least liberating-seeming perceptions of ground truths...
Having met him one time he seemed like just a really intense dude who embodied the chestnut “the CEO is the guy who walks in and says ‘I’m CEO’.” I dunno if there’s more to it than that.
It wasn’t a stunt and there was nothing questionable about it. I’m amazed by how easily people shit all over journalists - it really has to end because it is precisely how truth dies.
Here’s a question - since you have such strong feelings did you write the editor of the piece for their explanation?
PSA for any grad student in this situation: get a lawyer, ASAP, to protect your own career.
Universities care about money and reputation. Individuals at universities care about their careers.
With exceptions of some saintly individual faculty members, a university is like a big for-profit corporation, only with less accountability.
Faculty bring in money, are strongly linked to reputation (scandal news articles may even say the university name in headlines rather than the person's name), and faculty are hard to get rid of.
Students are completely disposable, there will always be undamaged replacements standing by, and turnover means that soon hardly anyone at the university will even have heard of the student or internal scandal.
Unless you're really lucky, the university's position will be to suppress the messenger.
But if you go in with a lawyer, the lawyer may help your whistleblowing to be taken more seriously, and may also help you negotiate a deal to save your career. (For example of help, you need the university's/department's help in switching advisors gracefully, with funding, even as the uni/dept is trying to minimize the number of people who know about the scandal.)
I found mistakes in the spreadsheet backing up 2 published articles (corporate governance). The (tenured Ivy) professor responded by paying me (after I’d graduated) to write a comprehensive working paper that relied on a fixed spreadsheet and rebutted the articles.
Will this be deployed mostly by those with the worst judgment?
For example, that video is implied to be of some open source self-driving project, run on an active public road, at 42mph. A lot of sensible people would say that's irresponsible or unsafe, and not do it. Move-fast-and-break-things bros and narcissists, however, wouldn't see a problem.
> Typically just enough goes towards the share purchase to make investors happy, and the rest is structured as incentives for founders and key execs with milestone payouts.
So they're getting the employees' shares without compensating the employees?
And there's incentives paid to the people who approved the deal, separate from their shares?
(I've heard of liquidation preferences, but never by the person making a job offer with stock options. Bribery also never came up.)
Yes, and yes. The sibling comment here about liquidation preferences is correct, and these separate incentives are usually structured as retention incentives — eg, compensation for future work with the acquiring company.
Shareholders are of course free to sue the board for acting outside of the interests of the shareholders overall, but this happens very rarely because typically the company would otherwise be shutting down and it’s very hard to make the argument that the deal undervalues common shareholders’ shares.
Because “shares” are not all the same. Preferred vs common, so unless you negotiated some kind of preferred share terms, assume your shares are worthless. For a non publicly listed company. For a publicly listed company, the details are all publicly available, so the different types of shares will have their different prices be easily available to see.
If that's true, when a startup is making you an offer for ISOs of common shares, and explaining it... how likely are they to know that, in event of a successful exit for the startup, your shares would be diluted and preferenced to 0 value?
(The two most recent offer equity components I accepted were "2%" and "a million shares". On the latter, an upper exec did a kind of deal-closer meeting for their offer, showing me a spreadsheet, estimating how much the options would be worth if there were an exit in X years at $Y valuation.)
> If that's true, when a startup is making you an offer for ISOs of common shares, and explaining it... how likely are they to know that, in event of a successful exit for the startup, your shares would be diluted and preferenced to 0 value?
If they have any experience, or even just browse a forum like this, they should be 100% likely to know. The person on the opposite side of the negotiating table has a goal of giving you as little as possible in exchange for your work (and vice versa).
> If these are the only errors, we are not troubled. However: we do not know if these are the only errors, they are merely a signature that the paper was submitted without being thoroughly checked for hallucinations. They are a signature that some LLM was used to generate parts of the paper and the responsible authors used this LLM without care.
I am troubled by people using an LLM at all to write academic research papers.
It's a shoddy, irresponsible way to work. And also plagiarism, when you claim authorship of it.
I'd see a failure of the 'author' to catch hallucinations, to be more like a failure to hide evidence of misconduct.
If academic venues are saying that using an LLM to write your papers is OK ("so long as you look it over for hallucinations"?), then those academic venues deserve every bit of operational pain and damaged reputation that will result.
I would argue that an LLM is a perfectly sensible tool for structure-preserving machine translation from another language to English. (Where by "another language", you could also also substitute "very poor/non-fluent English." Though IMHO that's a bit silly, even though it's possible; there's little sense in writing in a language you only half know, when you'd get a less-lossy result from just writing in your native tongue, and then having it translate from that.)
Google Translate et al were never good enough at this task to actually allow people to use the results for anything professional. Previous tools were limited to getting a rough gloss of what words in another language mean.
But LLMs can be used in this way, and are being used in this way; and this is increasingly allowing non-English-fluent academics to publish papers in English-language journals (thus engaging with the English-language academic community), where previously those academics they may have felt "stuck" publishing in what few journals exist for their discipline in their own language.
Would you call the use of LLMs for translation "shoddy" or "irresponsible"? To me, it'd be no more and no less "shoddy" or "irresponsible" than it would be to hire a freelance human translator to translate the paper for you. (In fact, the human translator might be a worse idea, as LLMs are more likely to understand how to translate the specific academic jargon of your discipline than a randomly-selected human translator would be.)
Autotranslating technical texts is very hard. After the translation, you muct check that all the technical words were translated correctly, instead of a fancy synonym that does not make sense.
(A friend has an old book translated a long time ago (by a human) from Russian to Spanish. Instead of "complex numbers", the book calls them "complicated numbers". :) )
I remember one time when I had written a bunch of user facing text for an imaging app and was reviewing our French translation. I don't speak French but I was pretty sure "plane" (as in geometry) shouldn't be translated as "avion". And this was human translated!
You'd be surprised how shoddy human translations can be, and it's not necessarily because of the translators themselves.
Typically what happens is that translators are given an Excel sheet with the original text in a column, and the translated text must be put into the next column. Because there's no context, it's not necessarily clear to the translator whether the translation for plane should be avion (airplane) or plan (geometric plane). The translator might not ever see the actual software with their translated text.
The convenient thing in this case (verification of translation of academic papers from the speaker's native language to English) is that the authors of the paper likely already 1. can read English to some degree, and 2. are highly likely to be familiar specifically with the jargon terms of their field in both their own language and in English.
This is because, even in countries with a different primary spoken language, many academic subjects, especially at a graduate level (masters/PhD programs — i.e. when publishing starts to matter), are still taught at universities at least partly in English. The best textbooks are usually written in English (with acceptably-faithful translations of these texts being rarer than you'd think); all the seminal papers one might reference are likely to be in English; etc. For many programs, the ability to read English to some degree is a requirement for attendance.
And yet these same programs are also likely to provide lectures (and TA assistance) in the country's own native language, with the native-language versions of the jargon terms used. And any collaborative work is likely to also occur in the native language. So attendees of such programs end up exposed to both the native-language and English-language terms within their field.
This means that academics in these places often have very little trouble in verifying the fidelity of translation of the jargon in their papers. It's usually all the other stuff in the translation that they aren't sure is correct. But this can be cheaply verified by handing the paper to any fluently-multilingual non-academic and asking them to check the translation, with the instruction to just ignore the jargon terms because they were already verified.
To that point I think it's lovely how LLMs democratize science. At ICLR a few years ago I spoke with a few Korean researchers that were delighted that their relative inability to write in English was no being held against them during the review process. I think until then I underestimated how pivotal this technology was in lowering the barrier to entry for the non-English speaking scientific community.
If they can write a whole draft in their first language, they can easily read the translated English version and correct it. The errors described by gp/op were generated when authors directly required LLM to generate a full paragraph of text. Look at my terrible English; I really have the experience of the full process from draft to English version before :)
We still do not have a standardized way to represent Machine Learning concepts. For example in vision model, I see lots of papers confused about the "skip connections" and "residual connection" and when they concatenate channels they call them "residual connection" while it shows that they haven't understood why we call them "residual" in the first place. In my humble opinion, each conference, and better be a confederation of conferences, work together to provide a glossary, a technical guideline, and also a special machine translation tool, to correct a non-clear-with-lots-of-grammatical-error-English like mine!
I'm surprised by these results. I agree that LLMs are a great tool for offsetting the English-speaking world's advantage. I would have expected non-Anglo-American universities to rank at the top of the list. One of the most valuable features of LLMs from the beginning has been their ability to improve written language.
Why is their use more intense in English-speaking universities?
Good point. There may be a place for LLMs for science writing translation (hopefully not adding nor subtracting anything) when you're not fluent in the language of a venue.
You need a way to validate the correctness of the translation, and to be able to stand behind whatever the translation says. And the translation should be disclosed on the paper.
There are legitimate, non-cheating ways to use LLMs for writing. I often use the wrong verb forms ("They synthesizes the ..."), write "though" when it should be "although", and forget to comma-separate clauses. LLMs are perfect for that. Generating text from scratch, however, is wrong.
I agree, but I don't think any of the broadly acceptable uses would result in easily identifiable flaws like those in the post, especially hallucinated URLs.
>I am troubled by people using an LLM at all to write academic research papers.
I'm an outsider to the academic system. I have cool projects that I feel push some niche application to SOTA in my tiny little domain, which is publishable based on many of the papers I've read.
If I can build a system that does a thing, I can benchmark and prove it's better than previous papers, my main blocker is getting all my work and information into the "Arxiv PDF" format and tone. Seems like a good use of LLMs to me.
> And also plagiarism, when you claim authorship of it.
I don't actually mind putting Claude as a co-author on my github commits.
But for papers there are usually so many tools involved. It would be crowded to include each of Claude, Gemini, Codex, Mathematica, Grammarly, Translate etc. as co-authors, even though I used all of them for some parts.
Maybe just having a "tools used" section could work?
> It's a shoddy, irresponsible way to work. And also plagiarism, when you claim authorship of it.
It reminds me of kids these days and their fancy calculators! Those new fangled doohickeys just aren't reliable, and the kids never realize that they won't always have a calculator on them! Everyone should just do it the good old fashioned way with slide rules!
Or these darn kids and their unreliable sources like Wikipedia! Everyone knows that you need a nice solid reliable source that's made out of dead trees and fact checked but up to 3 paid professionals!
I doubt that it's common for anyone to read a research paper and then question whether the researcher's calculator was working reliably.
Sure, maybe someday LLMs will be able to report facts in a mostly reliable fashion (like a typical calculator), but we're definitely not even close to that yet, so until we are the skepticism is very much warranted. Especially when the details really do matter, as in scientific research.
> I doubt that it's common for anyone to read a research paper and then question whether the researcher's calculator was working reliably
Reproducibility and repeatability in the sciences?
Replication crisis > Causes > Problems with the publication system in science > Mathematical errors; Causes > Questionable research practices > In AI research, Remedies > [..., open science, reproducible workflows, disclosure, ]
https://en.wikipedia.org/wiki/Replication_crisis#Mathematica...
Already verifiable proofs are too impossibly many pages for human review
There are "verify each Premise" and "verify the logical form of the Argument" (P therefore Q) steps that still the model doesn't do for the user.
For your domain, how insufficient is the output given process as a prompt like:
Identify hallucinations from models prior to (date in the future)
Check each sentence of this: ```{...}```
Research ScholarlyArticles (and then their Datasets) which support and which reject your conclusions. Critically review findings and controls.
Suggest code to write to apply data science principles to proving correlative and causative relations given already-collected observations.
Design experiment(s) given the scientific method to statistically prove causative (and also correlative) relations
Identify a meta-analytic workflow (process, tools, schema, and maybe code) for proving what is suggested by this chat
> whether the researcher's calculator was working reliably.
LLM's do not work reliably, that's not their purpose.
If you use them that way it's akin to using a butter knife as a screwdriver. You might get away with it once or twice, but then you slip and stab yourself. Better to go find screwdriver if you need reliable.
Im really not motivated by this argument; it seems a false equivalence. Its not merely a spell checker or removing some tedium.
As a professional mathematician I used wikipedia all the time to lookup quick facts before verifying it myself or elsewhere. A calculator well; I can use an actual programming language.
Up until this point neither of those tools were asvertised or used by people to entirely replace human input.
There are some interesting possibilities for LLMs in math, especially in terms of generating machine-checked proofs using languages like Lean. But this is a supplement to the actual result, where the LLM would actually be adding a more rigorous version of a human's argument with all the boring steps included.
In a few cases, I see Terrance Tao has pointed out examples LLMs actually finding proofs of open problems unassisted. Not necessarily problems anyone cared deeply about. But there's still the fact that if the proof holds, then it's valid no matter who or what came up with it.
AI People: "AI is a completely unprecedented technology where its introduction is unlike the introduction of any other transformative technology in history! We must treat it totally differently!"
Also AI People: "You're worried about nothing, this is just like when people were worried about the internet."
The internet analogy is apt because it was in fact a massive bubble, but that bubble popping didn't mean the tech went away. Same will happen again, which is a point both extremes miss. One would have you believe there is no bubble and you should dump all your money into this industry, while the other would have us believe that once the bubble pops all this AI stuff will be debunked and discarded as useless scamware.
Well the internet has definitely changed things; but also it wasnt initially controlled by a bunch of megacorps with the same level of power and centralisation today.
> Those new fangled doohickeys just aren't reliable
Except they are (unlike a chatbot, a calculator is perfectly deterministic), and the unreliability of LLMs is one of their most, if not the most, widespread target of criticism.
Low effort doesn't even begin to describe your comment.
> Except they are (unlike a chatbot, a calculator is perfectly deterministic)
LLM's are supposed to be stochastic. That is not a bug, I can see why you find that disappointing but it's just the reality of the tool.
However, as I mentioned elsewhere calculators also have bugs and those bugs make their way into scientific research all the time. Floating point errors are particularly common, as are order of operations problems because physical devices get it wrong frequently and are hard to patch. Worse, they are not SUPPOSED TO BE stochastic so when they fail nobody notices until it's far too late. [0 - PDF]
Further, spreadsheets are no better, for example a scan of ~3,600 genomics papers found that about 1 in 5 had gene‑name errors (e.g., SEPT2 → “2‑Sep”) because that's how Excel likes to format things.[1] Again, this is much worse than a stochastic machine doing it's stochastic job... because it's not SUPPOSED to be random, it's just broken and on a truly massive scale.
That’s a strange argument. There are plenty of stochastic processes that have perfectly acceptable guarantees. A good example is Karger’s min-cut algorithm. You might not know what you get on any given single run, but you know EXACTLY what you’re going to get when you crank up the number of trials.
Nobody can tell you what you are going to get when you run an LLM once. Nobody can tell you what you’re going to get when you run it N times. There are, in fact, no guarantees at all. Nobody even really knows why it can solve some problems and why it can’t solve other except maybe it memorized the answer at some point. But this is not how they are marketed.
They are marketed as wondrous inventions that can SOLVE EVERYTHING. This is obviously not true. You can verify it yourself, with a simple deterministic problem: generate an arithmetic expression of length N. As you increase N, the probability that an LLM can solve it drops to zero.
Ok, fine. This kind of problem is not a good fit for an LLM. But which is? And after you’ve found a problem that seems like a good fit, how do you know? Did you test it systematically? The big LLM vendors are fudging the numbers. They’re testing on the training set, they’re using ad hoc measurements and so on. But don’t take my word for it. There’s lots of great literature out there that probes the eccentricities of these models; for some reason this work rarely makes its way into the HN echo chamber.
Now I’m not saying these things are broken and useless. Far from it. I use them every day. But I don’t trust anything they produce, because there are no guarantees, and I have been burned many times. If you have not been burned, you’re either exceptionally lucky, you are asking it to solve homework assignments, or you are ignoring the pain.
Excel bugs are not the same thing. Most of those problems can be found trivially. You can find them because Excel is a language with clear rules (just not clear to those particular users). The problem with Excel is that people aren’t looking for bugs.
> But I don’t trust anything they produce, because there are no guarantees
> Did you test it systematically?
Yes! That is exactly the right way to use them. For example, when I'm vibe coding I don't ask it to write code. I ask it to write unit tests. THEN I verify that the test is actually testing for the right things with my own eyeballs. THEN I ask it to write code that passes the unit tests.
Same with even text formatting. Sometimes I ask it to write a pydantic script to validate text inputs of "x" format. Often writing the text to specify the format is itself a major undertaking. Then once the script is working I ask for the text, and tell it to use the script to validate it. After that I can know that I can expect deterministic results, though it often takes a few tries for it to pass the validator.
You CAN get deterministic results, you just have to adapt your expectations to match what the tool is capable of instead of expecting your hammer to magically be a great screwdriver.
I do agree that the SOLVE EVERYTHING crowd are severely misguided, but so are the SOLVE NOTHING crowd. It's a tool, just use it properly and all will be well.
One issue with this analogy is that calculators really are precise when used correctly. LLMs are not.
I do think they can be used in research but not without careful checking. In my own work I’ve found them most useful as search aids and brainstorming sounding boards.
> I do think they can be used in research but not without careful checking.
Of course you are right. It is the same with all tools, calculators included, if you use them improperly you get poor results.
In this case they're stochastic, which isn't something people are used to happening with computers yet. You have to understand that and learn how to use them or you will get poor results.
> One issue with this analogy is that calculators really are precise when used correctly. LLMs are not.
I made this a separate comment, because it's wildly off topic, but... they actually aren't. Especially for very large numbers or for high precision. When's the last time you did a firmware update on yours?
It's fairly trivial to find lists of calculator flaws and then identify them in research papers. I recall reading a research paper about it in the 00's.
One issue with this analogy is that paper encyclopedias really are precise when used correctly. Wikipedia is not.
I do think it can be used in research but not without careful checking. In my own work I've found it most useful as a search aid and for brainstorming.
Paper encyclopedias were neither precise nor accurate. You could count on them to give you ballpark figures most of the time, but certainly not precise answers. And that's assuming the set was new, but in reality most encyclopedias ever encountered by people in reality were several years old at least. I remember the encyclopedia set I had access to in the 90s was written before the USSR fell..
> I do think it can be used in research but not without careful checking.
This is really just restating what I already said in this thread, but you're right. That's because wikipedia isn't a primary source and was never, ever meant to be. You are SUPPOSED to go read it then click through to the primary sources and cite those.
Lots of people use it incorrectly and get bad results because they still haven't realized this... all these years later.
Same thing with treating stochastic LLM's like sources of truth and knowledge. Those folks are just doing it wrong.
I don't necessarily disagree, but researchers are not required to be good communicators. An academic can lead their field and be a terrible lecturer. A specialist can let a generalist help explain concepts for them.
They should still review the final result though. There is no excuse for not doing that.
I disagree here. A good researcher has to be a good communicator. I am not saying that it is necessarily the case that you don't understand the topic if you cannot explain it well enough to someone new, but it is essential to communicate to have a good exchange of ideas with others, and consequently, become a better researcher. This is one of the skills you learn in a PhD program.
To me, this is a reminder of how much of a specific minority this forum is.
Nobody I know in real life, personally or at work, has expressed this belief.
I have literally only ever encountered this anti-AI extremism (extremism in the non-pejorative sense) in places like reddit and here.
Clearly, the authors in NeurIPS don't agree that using an LLM to help write is "plagiarism", and I would trust their opinions far more than some random redditor.
> Nobody I know in real life, personally or at work, has expressed this belief.
TBF, most people in real life don't even know how AI works to any degree, so using that as an argument that parent's opinion is extreme is kind of circular reasoning.
> I have literally only ever encountered this anti-AI extremism (extremism in the non-pejorative sense) in places like reddit and here.
I don't see parent's opinions as anti-AI. It's more an argument about what AI is currently, and what research is supposed to be. AI is existing ideas. Research is supposed to be new ideas. If much of your research paper can be written by AI, I call into question whether or not it represents actual research.
> Research is supposed to be new ideas. If much of your research paper can be written by AI, I call into question whether or not it represents actual research.
One would hope the authors are forming a hypothesis, performing an experiment, gathering and analysing results, and only then passing it to the AI to convert it into a paper.
If I have a theory that, IDK, laser welds in a sine wave pattern are stronger than laser welds in a zigzag pattern - I've still got to design the exact experimental details, obtain all the equipment and consumables, cut a few dozen test coupons, weld them, strength test them, and record all the measurements.
Obviously if I skipped the experimentation and just had an AI fabricate the results table, that's academic misconduct of the clearest form.
I am not an academic, so correct me if I am wrong, but in your example, the actual writing would probably only represent a small fraction of the time spent. Is it even worth using AI for anything other than spelling and grammar correction at that point? I think using an LLM to generate a paper from high level points wouldn't save much, if any, time if it was then reviewed the way that would require.
My brother in law is a professor, and he has a pretty bad opinion of colleagues that use LLMs to write papers, as his field (economics) doesn't involve much experimentation, and instead relies on data analysis, simulation, and reasoning. It seemed to me like the LLM assisted papers that he's seen have mostly been pretty low impact filler papers.
> I am not an academic, so correct me if I am wrong, but in your example, the actual writing would probably only represent a small fraction of the time spent. Is it even worth using AI for anything other than spelling and grammar correction at that point? I think using an LLM to generate a paper from high level points wouldn't save much, if any, time if it was then reviewed the way that would require.
Its understandable that you believe that, but its absolutely true that writing in academia is a huge time sink. Think about it, the first thing your reviewers are going to notice is not results but how well it is written.
If its written terribly you have lost, and it doesnt matter how good your results are at that point. Its common to spend days with your PI writing a paper to perfection, and then spend months back and forth with reviewers updating and improving the text. This is even more true the higher up you go in journal prestige.
Who knows? Do NeurIPS have a pedigree of original, well sourced research dating back to before the advent of LLMs? We're at the point where both of the terms "AI" and "Experts" are so blurred it's almost impossible to trust or distrust anything without spending more time on due diligence than most subjects deserve.
As the wise woman once said "Ain't nobody got time for that".
"If much of your research paper can be written by AI, I call into question whether or not it represents actual research" And what happens to this statement if next year or later this year the papers that can be autonomously written passes median human paper mark?
What does it mean to cross the median human paper mark? How os that measured?
It seems to me like most of the LLM benchmarks wind up being gamed. So, even if there were a good benchmark there, which I do not believe there is, the validity of the benchmark would likely diminish pretty quickly.
I find that hard to believe. Every creative professional that I know shares this sentiment. That’s several graphic designers at big tech companies, one person in print media, and one visual effects artist in the film industry. And once you include many of their professional colleagues that becomes a decent sample size.
> Plagiarism is using someone else's words, ideas, or work as your own without proper credit, a serious breach of ethics leading to academic failure, job loss, or legal issues, and can range from copying text (direct) to paraphrasing without citation (mosaic), often detected by software and best avoided by meticulous citation, quoting, and paraphrasing to show original thought and attribution.
Higher education is not free. People pay a shit ton of money to attend and also governments (taxpayers) invest a lot. Imagine offloading your research to an AI bot...
Where does this bizarre impulse to dogmatically defend LLM output come from? I don’t understand it.
If AI is a reliable and quality tool, that will become evident without the need to defend it - it’s got billions (trillions?) of dollars backstopping it. The skeptical pushback is WAY more important right now than the optimistic embrace.
The fact that there is absurd AI hype right now doesn't mean that we should let equally absurd bullshit pass on the other side of the spectrum. Having a reasonable and accurate discussion about the benefits, drawbacks, side effects, etc. is WAY more important right now than being flagrantly incorrect in either direction.
Meanwhile this entire comment thread is about what appears to be, as fumi2026 points out in their comment, a predatory marketing play by a startup hoping to capitalize on the exact sort of anti AI sentiment that you seem to think is important... just because there is pro AI sentiment?
Naming and shaming everyday researchers based on the idea that they have let hallucinations slip into their paper all because your own AI model has decided thatit was AI so you can signal boost your product seems pretty shitty and exploitative to me, and is only viable as a product and marketing strategy because of the visceral anti AI sentiment in some places.
No that’s a straw man, sorry. Skepticism is not the same thing as irrational rejection. It means that I don’t believe you until you’ve proven with evidence that what you’re saying is true.
The efficacy and reliability of LLMs requires proof. Ai companies are pouring extraordinary, unprecedented amounts of money into promoting the idea that their products are intelligent and trustworthy. That marketing push absolutely dwarfs the skeptical voices and that’s what makes those voices more important at the moment. If the researchers named have claims made against them that aren’t true, that should be a pretty easy thing for them to refute.
The cat is out of the bag tho. AI does have provably crazy value. Certainly not the agi hype marketing spews and who knows how economically viable it would be without vc.
However, i think any one who is still skeptical of the real efficacy is willfully ignorant. This is not a moral endorsement on how it was made or if it is moral to use but god damn it is a game changer across vast domains.
There was a front page post just a couple of days ago where the article claimed LLMs have not improved in any way in over a year - an obviously absurd statement. A year before Opus 4.5, I couldn't get models to spit out a one shot Tampermonkey script to add chapter turns to my arrow keys. Now I can one small personal projects in claude code.
If you are saying that people are not making irrational and intellectually dishonest arguments about AI, I can't believe that we're reading the same articles and same comments.
Isn’t that the whole point of publishing? This happened plenty before AI too, and the claims are easily verified by checking the claimed hallucinations.
Don’t publish things that aren’t verified and you won’t have a problem, same as before but perhaps now it’s easier to verify, which is a good thing.
We see this problem in many areas, last week it was a criminal case where a made up law was referenced, luckily the judge knew to call it out.
We can’t just blindly trust things in this era, and calling it out is the only way to bring it up to the surface.
No, obviously not. You're confusing a marketing post by people with a product to sell with an actual review of the work by the relevant community, or even review by interested laypeople.
This is a marketing post where they provide no evidence that any of these are hallucinations beyond their own AI tool telling them so - and how do we know it isn't hallucinating? Are there hallucinations in there? Almost certainly. Would the authors deserve being called out by people reviewing their work? Sure.
But what people don't deserve is an unrelated VC funded tech company jumping in and claiming all of their errors are LLM hallucinations when they have no actual proof, painting them all a certain way so they can sell their product.
> Don’t publish things that aren’t verified and you won’t have a problem
If we were holding this company to the same standard, this blog wouldn't be posted either. They have not and can not verify their claims - they can't even say that their claims are based on their own investigations.
Most research is funded by someone with a product to sell, not all but a frightening amount of it. VC to sell, VC to review.
The burden of proof is always on the one publishing and it can be a very frustrating experience, but that is how it is, the one making the claim needs to defend themselves, from people (who can be a very big hit or miss) or machines alike. The good thing is that if this product is crap then it will quickly disappear.
That's still different from a bunch of researchers being specifically put in a negative light purely to sell a product. They weren't criticized so that they could do better, be it in their own error checking if it was a human-induced issue, or not relying on LLMs to do the work they should have been. They were put on blast to sell a product.
That's quite a bit different than a study being funded by someone with a product to sell.
Yup, and no matter how flimsy an anti-ai article is, it will skyrocket to the top of HN because of it. It makes sense though, HN users are the most likely to feel threatened by LLMs, and therefore are more likely to be anxious about them.
> Clearly, the authors in NeurIPS don't agree that using an LLM to help write is "plagiarism",
Or they didn't consider that it arguably fell within academia's definition of plagiarism.
Or they thought they could get away with it.
Why is someone behaving questionably the authority on whether that's OK?
> Nobody I know in real life, personally or at work, has expressed this belief. I have literally only ever encountered this anti-AI extremism (extremism in the non-pejorative sense) in places like reddit and here.
It's not "anti-AI extremism".
If no one you know has said, "Hey, wait a minute, if I'm copy&pasting this text I didn't write, and putting my name on it, without credit or attribution, isn't that like... no... what am I missing?" then maybe they are focused on other angles.
That doesn't mean that people who consider different angles than your friends do are "extremist".
They're only "extremist" in the way that anyone critical at all of 'crypto' was "extremist", to the bros pumping it. Not coincidentally, there's some overlap in bros between the two.
How is that relevant? Companies care very little about plagiarism, at least in the ethical sense (they do care if they think it's a legal risk, but that has turned out to not be the case with AI, so far at least).
What do you mean how is that relevant? Its a vast majority opinion in society that using ai to help you write is fine. Calling it "plagiarism" is a tiny minority online opinion.
First of all, the very fact that companies need to encourage it shows that it is not already a majority opinion in society, it is a majority opinion among company management, which is often extremely unethical.
Secondly, even if it is true that it is a majority opinion in society doesn't mean it's right. Society at large often misunderstands how technology works and what risks it brings and what are its inevitable downstream effects. It was a majority opinion in society for decades or centuries that smoking is neutral to your health - that doesn't mean they were right.
> Secondly, even if it is true that it is a majority opinion in society doesn't mean it's right. Society at large often misunderstands how technology works and what risks it brings and what are its inevitable downstream effects. It was a majority opinion in society for decades or centuries that smoking is neutral to your health - that doesn't mean they were right.
That its a majority opinion instead of a tiny minority opinion is a strong signal that its more likely to be correct. For example its a majority opinion that murder is bad; this has held true for millennia.
Heres a simpler explanation: toaster frickers tend to seek out other toaster frickers online in niche communities. Occams razor.
If you have to take the call, and your main concern is desktop client malware...
At a startup a few years ago, since I was the engineering dept., I had to be on a lot of enterprise sales/partnership calls, and much of the time we had to use the other company's favorite videoconferencing software.
Rather than installing those dumpster fire desktop apps on my Linux laptop that had the keys to our kingdom, I expensed an iPad that would be dedicated to random videoconf apps.
We still get violated numerous ways, but at least compartmentalized from the engineering laptop.
(I also used the iPad for passive monitoring of production at night, like a little digital photo frame in my living room, after putting away the work laptop.)
You can still. There's a small dark pattern to discourage it, though. You go to the URL for the call, click the button to launch the app, and when that fails, you see a small link to do the call in the Web browser.
every once in a while, someone will ask me to screenshare on a shared monitor, then i will have to explain i cannot , because i am on zoom browser.
Its always great to see the reactions that gathers. Its a true rainbow: bemusement, curiosity, exasperation, outright suspicion...and everything in between!
I already have Zoom installed on the work computer but for some reason it has started doing this weird thing where every time I click a Zoom meeting link in Google Calendar, Google Chrome downloads a copy of the Zoom installer at the same time as it opens the already installed Zoom. I didn’t notice until I already had six recently downloaded copies of the installer in the Downloads folder.
No idea why this happens. But it’s probably part of the crappy pushiness of Zoom to get people to install their app that makes them trigger a download of the installer because either they are not detecting that Zoom is already installed at the right time, or they are so eager to download the installer that they don’t even care about whether or not you already have it installed.
I’ve disliked Zoom since the beginning for their antics, and the only reason I have it installed is because I have to for the meetings at work, and the work computer belongs to the company I work for anyway, not to me.
I had to do it once and is extremely difficult, I don’t remember the details but I think you have to do dozens of extra steps on your account configuration and it won’t work on your phone unless you request the desktop version of the website.
Click on the meeting, where you will land on a download landpage. Then click the big download blue button in the center of the screen. WHen you click it a link will appear in the 2nd row below the blue button, something like "continue from browser", click on that, and you are golden
reply