Thanks for your spectacular resources! I see that you began an Annotated Mamba repository -- any chance you could share when that blog page might go live?
Hi Jeremy & Eric, great to see your newest endeavor. I hope that Answer.AI builds on the success and impact that fast.ai has already enjoyed.
Given new developments in hardware (by companies not named NVIDIA), I'm wondering if you are keen on exploring the next generation of model architectures and optimization procedures that might exploit newer hardware. In other words, will research directions pivot based on the hardware lottery?[1] Are you in conversations with companies developing these alternative chips?
Yes I think it's a big opportunity too, although I'm mainly looking at leveraging the work Modular will be doing to harness diverse hardware, rather than handle that ourselves.
There is indeed lots of uncertainty, and all Milky Way mass estimates are model dependent. Also, the ~10^12 solar mass figure referred to above is an estimate of the total galaxy + dark matter halo mass. Stellar or baryonic mass estimates are closer to ~6 x 10^10 solar masses. I'm using "~" to mean uncertainty to within a factor of 2 or so.
> The idea that AI might transform scientific practice is therefore feasible. But the main barrier is sociological: it can happen only if human scientists are willing and able to use such tools.
As a tenure-track scientist who works in ML applications for astrophysics, I disagree with this sentiment. The main issue isn't that enough scientists are using tools to search through literature or form new hypotheses, the main issue is that scientists now have to validate and sift through AI-generated outputs in order to find useful signals, rather than validate and sift through experimentally derived or observed signals.
AI can be useful for hypothesis generation in my field [0], and I think that there are lots of great use cases where it can be used to summarize information. However, it always comes with the possibility that it might output complete nonsense [1], so scientists who adopt these tools will have to spend some of their time verifying their outputs.
As others have said, forcing a login is a huge barrier.
I wanted to see if any astronomy posts were on the front page, and when I didn't see any, I tried to check out the categories. But when prompted for a login/sign-up, I lost interest.
No worries, I've just removed this, now you can freely read any article with no account needed.
> I tried to check out the categories
Hm, okay!
> But when prompted for a login/sign-up, I lost interest.
This is a one I cannot remove that fast (due to the way backend processes these categories), but I'm just curious, why did you loose an interest? I mean, yes, it requires login, and so?
1. The posts I'd find most interesting in the site are ones related to astronomy and applied machine learning. If there wasn't anything on astronomy (or physics) then I probably wouldn't use the site in the long run.
2.HN is a news aggregator. Most links that reach the front page have something interesting to offer. If I can't access the part of the site I find interesting, then I'll like go back to HN and find another post. (This isn't something I'd love to admit, but oh well.)
I thought you said above that you had removed the login requirement to view categories? I am still being asked to login. Or did I misunderstand? Would be nice to access categories without login necessary. Not seeing a reason that should be required?
Nope, I'm sorry if I was unclear, I was saying I've removed requirement to click on articles in the feed. This is a bit more complicated task with categories (due to some backend things I need to change), but I do understand the problem
Assisted logins are any "Login with ____" options. Google, Facebook, Apple, et al offer this. Its a bad deal because they (to a degree) take control away from you and contribute to further unnecessary tracking. I have anecdotally also found it to be annoying in terms of disabling if I changed my mind and wanted to use my own username/address and generated password to more directly control my account associated with it.
Additionally, if your account with the assistee was ever terminated or restricted, you may have a bad time trying to (as I remarked about) get back in direct control of your account. I don't like having all these things daisy-chained. Its like to digital version of being under conservatorship or babysat. If you're not prone to phishing I would advise against it, although I would prefer resolving that cardinal sin in shorter order where possible and practicable.
Hi Jeremy, always a fan of your work! Just a technical note since it falls under my domain of expertise (astronomy) -- the example about MOND described here should actually have choice (E) as the correct answer!
As it happens I dug into this question in some detail a couple of weeks ago when analysing the dataset, including carefully reading the wikipedia page which the question comes from. AFAICT both D and E are kinda correct, but E isn't quite right because MOND doesn't entirely "eliminate the observed missing baryonic mass", but rather just reduces it from a factor of 10 to 2.
Is that not correct? (Of course I fully accept your expertise in this matter and this is just my curiosity, not trying to tell you you're wrong!)
Fascinating! I dug into the Wikipedia article, which cites a Scholarpedia article; the LLM answer seems to originate from a reference to this sentence [1]:
> So, MOND reduces the discrepancy in clusters at these radii to only a factor of ∼2−3 (Sanders, 1999; Ettori, et al., 2019)
So I think you're right, and today I learned something! I also checked if Stacy McGaugh had weighed in on this particular subject, and it seemed like there is still an issue for clusters [2], although interestingly the issue isn't mentioned in his latest blog post that summarizes the strengths/weaknesses with MOND [3]. Anyway, thanks for humoring me for a bit.
I believe neither MOND nor Condensed Dark Matter are theories exactly, so much as they are schemata for classes of theories. Both are struggling to produce a verified theory that accounts for all observations, and while the latter is much more widely regarded as likely being correct, MOND has not been conclusively falsified to everyone's satisfaction. I would guess that there are, at least in principle, MOND theories which work for galaxy clusters but have residual discrepancies when applied to galaxies.
If this is so, then a multi-choice question which conflates one particular MOND theory for MOND itself, and which depends on the specifics of that particular theory for selecting the 'correct' answer, is problematic: for one thing, it may make selecting the 'correct' answer more difficult for a student who has specific knowledge about the topic. This is just one of several problems with multi-choice questions, though, fortunately, it does not seem to have any bearing on the very interesting phenomenon you have discovered.
In terms of the actual article -- really nice finding. Or I guess, nice set of experiments to decipher what lots of LLM researchers have been finding!
I've noticed somewhat similar behavior while training graph neural networks to model physical systems, except that it takes way longer than a single epoch to get there. Or course, there's no pretending involved with my GNNs, but the models do have very constrained representations, so once they start to figure out how to represent the physics at hand, the loss plummets dramatically.
> Please do not propose this regulation. If consumers actually cared about their IoT devices receiving security updates, companies would be doing it. The fact that companies are not already doing this is evidence it's not important to consumers. People may express frustration, but their purchasing behavior speaks louder than their words.
Or, maybe, companies are exploiting consumer ignorance and we're not dealing with an efficient market.
