It was in the article last night, they deleted it for some reason after my comment.
One of the authors has a peer reviewed NLP publication [1], the other has several publications in computer vision. I don’t know how they got research credits from Google.
Well that is pretty dubious. That completely changes the metric of how much money and how difficult this costs. (And I definitely believe you because there's several other comments with that same number). Could be a typo? But I feel like that's something you'd say. Then again, these are masters students.
It's $50k per training run. Their main contribution has been finding the optimal hyperparameters, not described in the OpenAI paper. Obviously you need more than one training run to to that.
Interesting. I'm doing ML in a niche area so I'm wondering if I can get funding (for my PhD). Obviously there's politics involved, but honestly I just want to try out methods and experiment (typical grad student I guess). So if they have a funding route I'm stoked. Thanks for the info.
In most programs, if you get accepted into the Ph.D. program, you're funded. Obviously, you should check first. Different programs treat these things differently with TA requirements, but most Ph.D. students are funded by an "RA" (research assistant, aka, doing your actual eventual-thesis research...).
Oh I'm funded. Definitely was told not to go unless I'm fully funded. But $500k for computing resources is bigger than a lot of small term contracts (I'm mostly on research and usually don't have to TA unless there's a hiccup in funding schedules. Gov money...). I definitely see that amount for long term, but this seems like a pretty short project.
Got it. (1) Yes, 500k is a lot, but (2) the TFRC program is making fairly expensive TPUs available at shockingly good prices for researchers willing to do some tire-kicking and feedback-providing. So 500k of TPU time is easier to come by than, say, 500k of generic compute.
