But it isn’t. Knowing that information is encoded in the weights gives us a route to deduce what a given model is doing.
And we are. Research is being done there.
> "Information is encoded in a brain's neurons and this affects our actions". Literally nothing useful you can do with this.
Different entirely. We don’t even know how to conceptualize how data is stored in the brain at all.
With a machine, we know everything. The data is stored in a binary format which represents a decimal number.
We also know what information should be present.
We can and are using this knowledge to reverse engineer what a given model is doing.
That is not something we can do with a brain because we don’t know how a brain works. The best we can do is see that there’s more blood flow in one area during certain tasks.
With these statistical models, we can carve out entire chunks of their weights and see what happens (interestingly not much. Apparently most weights don’t contribute significantly towards any token and can be ignored with little performance loss)
We can do that with these transformers models because we do know how they work.
Just because we don’t understand every aspect of every single model doesn’t mean we don’t know how they work.
I think we’re starting to run in circles and maybe splitting hairs over what “know how something works” means.
I don’t think we’re going to get much more constructive than this.
I highly recommend looking into LoRas. We can make Loras because we know how these models work.
But it isn’t. Knowing that information is encoded in the weights gives us a route to deduce what a given model is doing.
And we are. Research is being done there.
> "Information is encoded in a brain's neurons and this affects our actions". Literally nothing useful you can do with this.
Different entirely. We don’t even know how to conceptualize how data is stored in the brain at all.
With a machine, we know everything. The data is stored in a binary format which represents a decimal number.
We also know what information should be present.
We can and are using this knowledge to reverse engineer what a given model is doing.
That is not something we can do with a brain because we don’t know how a brain works. The best we can do is see that there’s more blood flow in one area during certain tasks.
With these statistical models, we can carve out entire chunks of their weights and see what happens (interestingly not much. Apparently most weights don’t contribute significantly towards any token and can be ignored with little performance loss)
We can do that with these transformers models because we do know how they work.
Just because we don’t understand every aspect of every single model doesn’t mean we don’t know how they work.
I think we’re starting to run in circles and maybe splitting hairs over what “know how something works” means.
I don’t think we’re going to get much more constructive than this.
I highly recommend looking into LoRas. We can make Loras because we know how these models work.
We can’t do that for organic brains.