You can actually get an ASIC manufactured for a few thousand dollars via CMP or Europractice. So not quite Google money. The difficulty is in paying for the software licenses you need to go from Verilog to DRC checked GDSII files (which is what you need to send to them).

In fact personally I think this is a much better route for open source hardware. Reverse engineering FPGA bitstreams impressive, but you're swimming against the tide. If we had good open source tooling for synthesis/place-and-route/DRC checking and good open source standard cell libraries (and these things exist, eg. qflow, they're just not amazing currently) then truly open source hardware could actually be a thing. Even on 10 year old fabs you'll get much better than you could do on an FPGA (you just have to get it right first time).

Alright, let me be the skeptic here and ask. Honest question, not intending to just dismiss anybody outright. But I have questions.

I've looked up Victor Taelin and his HighOrderCo company that supposedly just raised $4.5M. So here we have this Brazilian guy who graduated university in 2017, and I don't mean that in a bad way. It's just a fact, a data point. Fresh ideas are always welcome. From his Twitter feed seems to be deeply into the crypto space. He is over- or at least heavily upselling this HVM project as the future of computing. Lots of fuzz about interaction nets and linear logic and that this is the next thing after Turing Machines and Lambda calculus. Ok, strong claims. It makes me a tad suspicious though. Outperforming GHC? Running a new chain that's not a chain so it's not a crypto currency, but somehow it's a chain, but it's formally verified, hm, ok. And how can he strongly claim this to be the superior thing if he just announced how to model complex numbers in his HVM project, and that GPT-4 helped him get this insight? I'd have expected Clifford Algebras to be basic knowledge for everybody working in this space.

I'm only superficially familiar with ETH, with Haskell's GHC, with Rust, with HOTT. But, I'd like to hear from people who know more than I do, and who are not already obvious fanboys of HighOrderCo with submarine PR interests -- how much weight should I put into this? Is it worth reading about, getting my head busy? I've spent an hour now trying to check up on things, but apart from a lot of fluff and quite a lot of technical jargon that went above my head, I'm left confused.

Is there something or is this just supposed to confuse me to get into the next scam?

Edit: Just to add to this. https://news.ycombinator.com/item?id=35344514 discusses an article written about the underlying tech. It's a terrible article on the level of a newcomer to programming who doesn't know much about functional programming and even admits as much. Victor Taelin comments with "Very well written and beautiful article." This must be a scam. I'll stop wasting my time on this until somebody manages to convince me otherwise.

For those confused about how the hyperdimensional computing (HDC) approach (also known as "vector symbolic architectures"/VSA or "holographic reduced representations"/HRR) described in the article differs from the use of vector embeddings in more mainstream artificial neural networks:

1. HDC is largely organized around the observation that certain symbolic-like operations become vastly simplified in high-dimensional spaces and can be performed with simple algebraic manipulations.

For instance, if you have a dictionary of words, each represented by a random (but fixed) high-dimensional vector, you can store subsets of these words just by summing their vector representations together. This works because random high-dim vectors are nearly orthogonal with very high probability. This implies that the sum of several of such vectors will have essentially a zero dot product with words (ie their embeddings) not included in the subset and a much larger dot product (~1 if the vectors are all normalized) with words included in the subset (as long as the number of words in the subset is sufficiently smaller than the total dictionary size). Hence the sum encodes the subset since subset membership can be checked with a dot product.

Notably, this also works when the dictionary size is exponentially large relative to the vector dimension, since it is possible to sample an exponentially large number of near-orthogonal vectors in a high-dimensional vector space (unlike in a low-dimensional vector space). This is also very mathematically similar to how a [Bloom filter](https://en.wikipedia.org/wiki/Bloom_filter) works, which is a probablistic data structure commonly used for set membership queries when the set is extremely large (e.g. all possible URLs). (Note also, though, that this is at the expense of being able to decode/recover the set elements directly from the summed representation.)

HDC/VSAs capitalize on other nice properties of random high-dimensional vectors as well, e.g. via the ability to "bind" two words together using a circular convolution. See [Kanerva 2009](http://rctn.org/vs265/kanerva09-hyperdimensional.pdf) for a nice review of various examples like this. There are also various similar ways to represent sequences of words, tree structures, graphs, etc, all within a fixed- but high-dim vector space, so they provide a nice way to represent "syntactic" structure in data objects, if you will.

2. None of the above emerges as a property of network training. In fact, a network isn't even necessarily required, which could be a feature or a bug, depending on your perspective.

As a feature, this is useful in the sense that it provides an immediate no-training-required way of representing objects with fairly complex relational structure (e.g. a sequence of words or a tree-structure) as a fixed- but high-dimensional vector. In a traditional neural network (although this may be less true for very modern LLMs) trained to do something with, say, sentences of a max of 20 words, the network would likely fail to "figure out how to represent" sentences composed of 100 words. With the HDC/VSA approach, however, you get a representation of such a sentence right off the bat, and don't have to worry about it being inside/outside your training dataset. The "utility" of such a representation is not necessarily obvious, and will depend on exactly how it is created, but IMO it is nice to know that there is a systematic way of constructing one that does not interfere with others (with very high probability).

On the other hand, one of the main drawbacks of this approach (at least so far) is that it has generally not been obvious how to make these systems learn robustly, so that e.g. vector representations could change through learning to capture more semantic relationships between words and objects. Nonetheless, given the above one can imagine how the HDC/VSA approach may provide something akin to a useful inductive bias or initialization for representation learning in more trainable systems.

It will certainly be interesting to see if and how these might get incorporated into modern AI systems in the coming years.

Does anybody know if there is a similar playground, but for evaluating a single model by comparing different prompts with different temperatures?

I am building this by myself using Streamlit, but was wondering is there a ready solution.