Hacker News
new
|
past
|
comments
|
ask
|
show
|
jobs
|
submit
login
Rhapso
on July 19, 2023
|
parent
|
context
|
favorite
| on:
Mini – The Minimal Language
FTR LLM embeddeds are a lot less efficient than ASCII for communication and are likely to stay that way.
riku_iki
on July 20, 2023
[–]
absolutely not. Transformer layers already communicate using embeddings, and ASCII would be absolutely less efficient there.
Rhapso
on July 20, 2023
|
parent
[–]
And how many bits are in an embedded vector?
riku_iki
on July 20, 2023
|
root
|
parent
[–]
12k for gpt3.
riku_iki
on July 21, 2023
|
root
|
parent
[–]
It is not bits, but weights
Rhapso
on July 23, 2023
|
root
|
parent
[–]
So somehow ascii is less information dense than 12k 32-bit floats per token?
Guidelines
|
FAQ
|
Lists
|
API
|
Security
|
Legal
|
Apply to YC
|
Contact
Search:
