- finding funders willing to risk billions for capped returns.
Neither happened. And I'm not sure the public would invest 100's of billions on the promise of AGI. I'm glad there are investors willing to take that chance. We all benefit either way if it is achieved.
I feel as though you're ignoring the most important part of that sentence. I assume you meant to write;
I believe that AGI will be a net benefit to whomever controls it.
I would argue that if a profit driven company rents something valuable out to others, you should expect it would benefit them just as much if not more, than those paying for that privilege. Rented things may be useful, but they certainly are not a net benefit to the system as a whole.
No, I believe AGI will have a net benefit for all of humanity. The telephone system was a net benefit for all Americans even though for a time AT& T (Ma Bell) controlled it.
AGI is fantasy at this point and your assumption that AGI would give OpenAI unprecented powers is the Musk/Yudkowsky/Hinton argument that AI will dominate and enslave us.
Drop those assumptions and my point stands that throughout history, monopolistically-controlled transformative technologies (telephones, electricity, vaccines, railroads) have still delivered net benefits to society, even if imperfectly distributed. This is just historical fact.
> AGI is fantasy at this point and your assumption that AGI would give OpenAI unprecented powers is the Musk/Yudkowsky/Hinton argument that AI will dominate and enslave us.
Yeah, like I said, room for improvement. I find the argument that AGI or sAGI should be feared, or is likely to turn "evil" absurd in the best case. So your arguing against a strawman I already find stupid.
Telephones, increased the speed of information transfer, it couldn't produce on it's own. Electricity allowed transmission of energy from one place to another, and doesn't produce inherent value in isolation, vaccines are in an entirely different class of advancement, (so I have to idea how you mean to apply it to the expected benefits of AGI? I assume you believe AGI will have something to do with reducing disability), railroads again, like energy or telephones, involved moving something of value from one place to another.
AGI is supposed to produce a potentially limitless amount of inherent value on its own, right? It will do more than just move around components of value, but more like a diamond mine, it will output something valuable as a commodity. Something that can easily be controlled... oh but it's also not concrete, you can never have your own, it's only available for rental, and you have to agree to the ToS. That sounds just like all previous inventions, right?
You're welcome to cite any historical facts you like, but when you're unwilling or unable to draw concrete parallels, or form convincing conclusions yourself, and hand wave, well most impressivive inventions in the past were good so I feel AGI will be cool too!
Also, the critical difference (ignoring the environmental differences between then and now) between the inventions you cited, and AGI, is the difficulty in replicating any technology. Other than "it happened before to most technologies" is there reason I should believe that AGI would be easy to replicate for any company that wants to compete against the people actively working to increase the size of their moat? copper wire, and train tracks are easy to install. Do you expect AGI will be easy for everyone to train?
oh, sorry dude... I wasn't expecting the indirect insult to be the only thing you read... my intent was less for you to take offense, and more to point out how you're arguing against something I never said and don't believe. I would have been interested in the reasoning behind the claim, and the parallels you saw, but was unwilling to tolerate the strawman.
Thanks. I'm sorry I jumped to the conclusion that you were making the doomer arguement. I see now your argument is much more subtle and raises some interesting points. If I understand it correctly, it's like what if one company owned the internet? But worse than that, what if one company owned access to intelligence? I'm old so I remember when AT&T owned the American phone system. We couldn't hook up anything to the phone jack without permission, so intuitivly I did understand your argument, but my opposition to doomer arguments (pause research! regulate!) got in the way.
A lot of funding was predicated on them making the transition. Also they would not have been able to IPO without the transition, so there was a funding problem when you look at it that way.
This mental model is also in direct contradiction to the whole purpose of the embedding, which is that the embedding describes the original text in a more interpretable form. If a piece of content in the original can be used for search, comparison etc., p much by definition it has to be stored in the embedding.
Similarly, this result can be rephrased as "Language Models process text." If the LLM wasn't invertible with regards to a piece of input text, it couldn't attend to this text either.
In principle, it should be possible to identify malign IPs at scale by using a central service and reporting IPs probabilistically. That is, if you report every thousandth page hit with a simple UDP packet, the central tracker gets very low load and still enough data to publish a bloom filter of abusive IPs, say a million bits gives you pretty low false-positive. (If it's only ~10k malign IPs, tbh you can just keep a lru counter and enumerate all of them.) A billion hits per hour across the tracked sites would still only correspond to ~50KB/s inflow on the tracker service. Any individual participating site doesn't necessarily get many hits per source IP, but aggregating across a few dozen should highlight the bad actors. Then the clients just pull the bloom filter once an hour (80KB download) and drop requests that match.
Any halfway modern LLM could probably code the backend for this in a day or two and it'd run on a RasPi. Some org just has to take charge and provide the infra and advertisement.
The hard part is the trust, not the technology. Everyone has to trust that everyone else is not putting bogus data into that database to hurt someone else.
It's mathematically similar to the "Shinigami Eyes" browser plug-in and database, which has been found to have unreliable data
Personally talk to every individual participating company. Provide an endpoint that hands out a per-client hash that rotates every hour, stick it in the UDP packet, whitelist query IPs. If somebody reports spam, no problem, just clear the hash and rebuild, it's not like historic data is important here. You can even (one more hour of vibecoding) track convergence by checking how many bits of reported IPs match the existing (decaying) hash; this lets you spot outlier reporters. If somebody always reports a ton of IPs that nobody else is, they're probably a bad actor. Hell, put a ten dollar monthly fee on it, that'll already exclude 90% of trolls.
I'm pretty pro AI, but these incompetent assholes ruin it for everybody.
As talked about elsewhere in this thread, residential devices being used as proxies behind CGNAT ruins this. Not getting rid of IPv4 years ago is finally coming to bite us in the ass in a big way.
If usability tickets are closed because the company doesn't want to bother, then maybe these gears deserve to have sand put in them.
I generally approve of subversive actions which are naturally damaging if and exactly if the accusation they are based on is true.
That is, the logic is something like "well, either it gets fixed, in which case it's a victory for good, or they're hypocrites who don't really care, in which case 1. it wastes their time and 2. they deserve to have their time wasted."
The tokenizer covers the entire dataset. It's basically just a fixed-size Huffman code, grouping together common fragments of letters- for instance, the 100 most common English words are probably all single tokens.
During learning, the model proceeds in roughly the same way a child would: it starts by grouping tokens together, learning the deep regularities of language such as "news[paper]" being more likely than "news[q77.bfe]". Then it incrementally assembles these fragments into larger and larger chains. Similarly , it first learns thematic groupings, such as "word" being more likely somewhere after "dictionary" rather than "stop what I was reading to get the dictionary out every time I encountered a banana assault hungry". Then it starts to pick up "patterns": "as a [baby|child|kid] I had no [idea|concept|clue]". At some point in this process it naturally abstracts concepts from languages: "as a child" starts being internally represented by the same neurons as "als ich ein Kind war".
Then some magic happens that we don't understand, and out pops a neural network that you can talk to and that can write programs and use tools. To be clear, this is the case before RL: probably these patterns are now widespread in the training data, so that the model already understands how to "complete the pattern" on its own. RL then does some magic on top of that to bring it from 20% benchmarks to 80% and presto, AI assistant.
>Claude sometimes thinks in a conceptual space that is shared between languages, suggesting it has a kind of universal “language of thought.” We show this by translating simple sentences into multiple languages and tracing the overlap in how Claude processes them.
I think concept here means it is assigned a point or an area in the many-dimensional embedding space. The "concept" has no obvious form, but similar words, synonyms or words from another languages meaning roughly the same are very close together in this space.
reply