I asked this question in a college-level class with clickers. For the initial question I told them, "This is a trick question, your first answer might not be right". Still less than 10% of students got the right answer.
This idea that students are "more generally intelligent" requires specific arguments to support. ChatGPT alone has a vast breadth of knowledge that is impossible for a human to keep up with as well as superior skills to a student in any number of fields. I can find evidence that it has an IQ of around 124 [0] which is going to stretch most students (although in fairness the same article also speculates an IQ of 0).
Students can keep ahead of it with training in specific fields and it has a few weaknesses in specific skills but I think someone could make a reasonable claim that ChatGPT has superior general intelligence.
> superior skills to a student in any number of fields
It's abhorrent any time we measure pure distilled intelligence.
When asked to come up with any non-basic novel algorithm and data structure, it creates nonsense.
Especially when you ask it to create vector instruction friendly memory layouts and it can't code in its preferred way. I had some fun trying to make it spit out a brute-force-ish solver for a problem involving basic orbital mechanics and some forces. Wouldn't even want try something more complicated. It can do generalized solvers somewhat, since it can copy that homework, but none that can express the kinds of terms you'd be working with (despite those also having code available in some research papers).
Speaking of which, it cannot even figure out some basic truths in orbital mechanics that can be somewhat easily derived from the formulas commonly given, nine times out of ten (you can get there if you're very patient and are able to filter its wrong answers).
But at the end of the day it was still a valuable tool to me as I was learning these things myself, since despite being often wrong, it nevertheless spat out useful things I could plug into Google to find more trustworthy sources that would teach me. Really neat if you're going in blind into a new subject.
Claiming that Chatgpt is more intelligent than a student is the same as saying an encyclopedia or a library is more intelligent than a student. Sure they retain more information. But Chatgpt is not AGI and it has no idea what it is even talking about.
> I can find evidence that it has an IQ of around 124
Despite me being someone who is generally impressed by the best LLMs, I think this says more about IQ tests than it does any AI.
Which isn't to shame those tests — we made those tests for humans, we were the only intelligence we knew of that used complex abstract language and tools until about InstructGPT — but it does mean we should reconsider what we mean by "intelligence".
My gut feeling is that a better measure is how fast we learn stuff. Computers have a speed advantage because transistors outpace synapses by the degree to which a pack of wolves outpaces continental drift (yes I did do the calculation), so what I mean here is how many examples rather than how many seconds.
But as I say, gut feeling — this isn't a detailed proposal for a new measure, it likely needs a lot of work to even turn this into something that can be a good measure.
One idea is to put more pressure on writers of letters of recommendation. There is obviously already some rather weak "reputational incentive" to not give letters of recommendation to fraudsters. But it could be made a more formal policy. Letters could be required to include a statement about the candidate adhering to various formalized "good behavior" policies. Then when fraud happens there could be actual consequences for the letter writer. This would be like the culture in software development where you don't blame people for writing bad code, you blame reviewers for letting in bad code. (Idea comes from Angela Collier, in relation to other forms of antisocial behavior).
1. There is already a lot of administrative bloat in the academia, and ideas like this are responsible for most of it. Every time you replace an informal process with a formal one, you add to the bloat. And the formal process rarely works as intended.
2. Letter-writers have no control over the work of the person they recommend. They cannot prevent the submission of a manuscript or a grant. They would be taking a lot of risk for no gain.
3. Letters of recommendation are a form of corruption and nepotism. If you would have a conflict of interest for reviewing a manuscript or a grant, you should not be allowed to write a letter of recommendation either.
If what you propose would be implemented, why would anyone ever write a letter of recommendation? I'm currently doing an unpaid favor to the institution which asks for letters of recommendation, as I spend my time so they get my opinion about the candidate. If instead of thanking me for it they would want to put more pressure on me and assign actual consequences, well, that's insulting and I'd pass, I owe them literally nothing.
I guess you can prepare the necessary packages and mount them in the WASI file system for the Python binary (WASI allows this). Then you can import these packages in your Python code.
I'm using the new generation of small models to do semantic search for music lyrics. The first step is giving the model the text and asking, "What is this text about?" Without fine-tuning and only minimal prompt engineering these models can understand most languages, pull out the most relevant phrases, and list major themes and speculative ideas of what the lyrics mean. I'm super impressed with the results. Reading the answers feels like grading ambitious undergraduate student essays (both the good and the bad).
This is a good one. It is all too easy to spend years casually "interested" in a topic without developing a true foundation of the important ideas in the field.
In my case I spent two decades casually reading about economics. But then one day I ordered an economics textbook to work through the problems, and found out that I basically knew nothing and couldn't do any of the problems. A few months of slowly working through the textbook gave me a much deeper level of understanding. I could do the problems now.
This is awesome! This reminds me of MinBLEP audio synthesis of discontinuous functions (https://www.cs.cmu.edu/~eli/papers/icmc01-hardsync.pdf). Instead of doing things at high sampling rate and explicitly filtering, generate the band-limited version directly.
In the article, talking about smoothstep approximation of sinc: "I'd argue the smoothstep version looks better" Why would this be? I would have thought the theoretically correct sinc version would look nicer.
sinc is perfect if you are looking only at frequency response. But in images, you also want to preserve locality, that is, processing of one part of the image should not affect the rest of the image. For example, sharpening an edge should only affect the edge in question, not its surroundings. It comes in contradiction with the idea of preserving frequency response. Frequencies are about sine waves, and sine waves are wide, infinitely wide in fact.
BTW, that's also the reason why in quantum mechanics you can't know both position and momentum (a frequency) precisely.
So we need to compromise, and like in scaling algorithms, you have 3 qualities: sharpness (the result of a good frequency response), locality and aliasing. You can't have all 3, so you need to pick the most pleasant combination.
The extreme cases are:
- Point sampling: excellent locality and sharpness, terrible aliasing
- Linear filtering: excellent locality and no aliasing, very blurry
- sinc filtering: excellent sharpness and no aliasing, terrible locality (ringing artefacts)
Using smoothstep is a good compromise, it has a bit of aliasing because it is a step function and it has a bit of smearing because it is smooth but none of the effects as so bad as to be unpleasant.
Side note: for audio, frequency response is more important than locality, that's why sinc window functions are so popular.
> "I'd argue the smoothstep version looks better" Why would this be? I would have thought the theoretically correct sinc version would look nicer.
For a fixed well-defined mathematical problem, you might be able to solve it optimally or approximately. One perspective is to treat the problem as given and immutable and then try to compute an exact or optimal solution.
But often the original problem statement is fairly arbitrary, based on a bunch of guesses or simplifications, and you might be able to get a better result by changing the problem definition (perhaps unfortunately making it much messier to solve exactly) and then solving the new problem statement approximately.
What's the actual problem we're trying to solve here? Generate something that looks visually pleasing. Why is an expression involving cosine the natural way to define that problem statement mathematically? There's likely a lot of freedom to here to vary our problem definition.
It might be interesting to start with the smoothstep multiplied result and take the derivative and look at how that differs from a normal cosine, and ponder why that might produce a more pleasing result than a cosine.
It seems like a theoretically correct box filter might not actually be the best filter to use? By approximating it you get a different filter, and whether it's a better filter is something you need to judge by looking at the result.
It looks like the sinc version is still adding a little bit of some higher frequencies (the dampened sine wave), and the approximation doesn't. Maybe those higher frequencies don't actually make things look better?
> In the article, talking about smoothstep approximation of sinc: "I'd argue the smoothstep version looks better" Why would this be? I would have thought the theoretically correct sinc version would look nicer.
In this case we are sort of mimicking the eye. The eye doesn't do sinc-bandlimiting (it does a sort of angular integration -- it sums the photons received in a region).
I say "sort of", because we're really doing two steps: first we are projecting a scene into a screen, and then the eye is viewing the screen. We want (in most cases) that what the eye sees in the screen corresponds to what it would see directly (if seeing the scene in reality).
The naive rendering approach simply samples an exact point for each pixel. When there's high pixel variation (higher spatial frequency than the pixel frequency), as you move the camera the samples will alternate rapidly which wouldn't correspond to the desired eye reconstruction. The eye would see approximately an averaged (integrated) color over a small smooth angular window.
Note we really never get the perfect eye reconstruction unless the resolution of your display is much larger than the resolution your eye can perceive[1]. But through anti-aliasing at least this sampling artifact disappears.
This window-integration is not an ideal sinc filtering! Actually it's not bandlimiting at all! since it is a finite-support convolution -- bandlimiting is just a convenient theoretical (approximate/incorrect) description.
In the frequency domain this convolution is not a box (ideal sinc filtering), it's smooth with ripples. In the spatial domain (that's really used here), it probably does look something like a smoothstep (a smooth window)[2]. The details don't matter if the resolution is large[3].
[1] Plus we would actually need to model other optical effects of the eye (like focus and aberration) that I won't go into detail :) But you can ask if interested.
[3] Because our own eye integrates the pixels anyway. Again this does ignore other optical effects of the eye (such as "focus" and aberration) that vary with distance to the focal plane, and more.
Just fixing a mistake: what I described is the window function, not the integrated (in this case, cosine) function that was used in the article. In this case there would still be ripples when applying the shown window function (in cosine integration). I do think ripple-free are probably better functions (or some faster decaying ripples) because of limited floating point precision generating artifacts (which can be seen in the center in the second demo).
Agree, automated testing is good as far as it goes but really shouldn't be the whole story with grading.
Another fun tool is CrowdGrader (https://www.crowdgrader.org/). It's a nice way to distribute reviewing/grading throughout the class to make it more scalable. When I was teaching I really liked it, it meant I could assign more interesting creative assignments at a faster pace. It's a different type of "automation".
Exactly, i deal with a few of data processing programs, wish my more experienced colleagues told me about these algorithms before, but doesn't this algorithm only compute an approximation and not the exact value ?
I would recommend all the works by Robert Nozick, listed at #14 on the list for "Anarchy, State and Utopia". While that might be his most influential book, I found his other books more fun to read.
I asked this question in a college-level class with clickers. For the initial question I told them, "This is a trick question, your first answer might not be right". Still less than 10% of students got the right answer.