Also, commercial software is consistently behind from open source.
I only use open source LLMs for writing (Qwen 32b from Groq) and open source editor of course, Emacs.
If some people can write better using commercial LLMs (and commercial editors), by all means, but they put themselves at a disadvantage.
Next step for me, is to use something open source for translation, I use Claude for the moment, and open source for programming, I use GPT curently. In less than a year I will find a satisfying solution to both of these problems. I haven't looked deep enough.
The point of DeepSeek-OCR is not an one way image recognition and textual description of the pixels, but text compression to a generated image, and text restoration from that generated image. This video explains it well [1].
> From the paper: Experiments show that when the number of text tokens is within 10 times that of vision tokens (i.e., a compression ratio < 10×), the model can achieve decoding (OCR) precision of 97%. Even at a compression ratio of 20×, the OCR accuracy still remains at about 60%. This shows considerable promise for research areas such as historical long-context compression and memory forgetting
mechanisms in LLMs.
It's main purpose is: a compression algorithm from text to image, throw away the text because it costs too many tokens, keep the image in the context window instead of the text, generate some more text, when text accumulates even more compress the new text to image and so on.
The argument is, pictures store a lot more information than words, "A picture is worth a thousand words" after all. Chinese characters are pictograms, it doesn't seem that strange to think that, but I don't buy it.
I am doing some experiments of removing text as an input for LLMs and replacing with it's summary, and I have reduced the context window by 7 times already. I am still figuring what it the best way to achieve that, but 10 times is not far off. My experiments involve novel writing not general stuff, but still it works very well just replacing text with it's summary.
If an image is worth so many words, why not use it for programming after all? There we go, visual programming again!
Combinators are math though. There is a section in the paper that covers the topic of graphs and charts, transforming them to text and then back again to image. They claim 97% precision.
> within a 10× compression ratio, the model’s decoding precision can reach approximately 97%, which is a very promising result. In the future, it may be possible to achieve nearly 10× lossless contexts compression through text-to-image approaches.
Graphs and charts should be represented as math, i.e. text, that's what they are anyway, even when they are represented as images, it is much more economical to be represented as math.
The function f(x)=x can be represented by an image of (10pixels x 10pixels) dimensions, (100pixels x 100pixels) or (infinite pixels x infinite pixels).
> It should be a "right to not have product forced on you."
Even better, a "right to modify everything you own, in any way you like". Don't you like the micro-controller installed by the manufacturer? Buy another one, with the correct firmware programmed from scratch, and swap it off.
We are already well into a new era of software, in which software can be programmed by itself, especially Rust. What is missing is money transactions for software companies and their employees located everywhere in the world.
"Devices with no surprises". Retail shops in conjuction with electronics engineers put new controllers in everything and re-sell it. Open source software, auditable by anyone and modified at will.
Programs for every car, every refrigerator etc cannot be programmed by a company located in one place, not even 10 places. It has to be a truly global company.
In other words, I want your device, I don't want your closed source software.
Are you willing to indemnify the manufacturer from any liability for anything that might go wrong on the car from then on? No factory warranty once you make changes. Potentially losing access to recall repairs because of the changes you made. In this age of software the entire car is increasingly designed holistically. The engineer might decide to use a particular grade of aluminum on a control arm knowing that the controller software is designed to never exceed certain limits.
> Are you willing to indemnify the manufacturer from any liability [..] No factory warranty once you make changes.
Car manufacturers have figured out how to make expensive cars with good materials and very safe as well. The problem is cheap cars, which can be much more defective and dangerous to drive.
There is a solution to that though. 10-50 people combining their buying power, getting an expensive car and time sharing their usage of it. A mix between public transportation, robo-taxi and personal ownership.
> The engineer might decide to use a particular grade of aluminum on a control arm [..]
That's a problem indeed, a 3d printer for example might be off by some millimeters in some dimension, the manufacturer accounts for that in software and it prints well afterwards. What kind of materials are used is important for sure, but the properties of metals used in the car can be made public, especially if the manufacturer is paid premium and just sold an expensive car instead of a cheap one.
The thing with software though, is that it can be infinitely extended and modified. I can have ten thousand programs more running in my computer tomorrow, with no change to anything physical. Physical stuff need to be manufactured, transported, warehoused, so there is always a limit.
Consumers want always more stuff, if 10 programs are available they want 10 programs. If 100 programs are available they want 100 programs. It never ends. Proprietary software is not ideal there.
The era of intelligent robots is the end of washing machines and many other specialized machines, even tractors.
Two kinds of machinery are needed.
One very basic and cheap robot to go around the neighborhood and gather clothes in some boxes, and transfer them in a designated room to wash them.
Two state of the art robotic hands mounted on the wall, and connected to AC (no batteries). The two arms are going to be controlled by computers even a whole rack of them, with many GPUs in them. The whole setup might use 10KW of energy, it will wash clothes by hand, it will be fast, dexterous and accurate. Expensive as well. In 3 minutes it will wash 100 t-shirts much better than the best human on the planet, or any other non-intelligent machine.
Then the small basic robot returns the clothes to the house.
Yeah, why have a washing machine when you can just let your droid hand wash the dishes? At high enough temperatures and proper scrub that's likely going to be better and take less time.
Not droid, just arms. If it needs to be fast, lift weights (just the arms or even more), have high quality cameras and be connected to a lot of compute, it needs AC from the grid.
AC wires, better not move around, especially when there is water. It has to be mounted on the wall.
> That's when A.I. starts advancing itself and needs humans in the loop no more.
You got to put the environment back in the loop though, it needs a source of discovery and validity feedback for ideas. For math and code is easy, for self driving cars doable but not easy, for business ideas - how would we test them without wasting money? It varies field by field, some allow automated testing, others are slow, expensive and rate limited to test.
Simulation is the answer. You just need a model that's decent at economics to independently judge the outcome, unless the model itself is smart enough. Then it becomes a self-reinforcing training environment.
Now, depending on how good your simulation is, it may or may not be useful, but still, that's how you do it. Something like https://en.wikipedia.org/wiki/MuZero
That requires a lot of human psychology and advanced hard economic theory (not the fluffy academic kind). With human controlled monetary supply and most high-level business requiring illegal and immoral exploitation of law and humans in general, it's not a path machines can realistically go down or even want machines treading down.
Think scams and pure resource extraction. They won't consider many impacts outside of bottom line.
Simulated environment suggests the possibility of alignment during training but real time, real world, data streams are better.
But the larger point stands: you don't need an environment to explore the abstraction landscape prescribed by systems thinking. You only need the environment at the human interface.
The question is where should AI advance itself? Which direction? There are an infinite number of theorems that can be derived from a set of axioms. Infinite. AI can't prove them all. Somebody needs to tell it what it needs to do, and that is us.
Sorting a finite number of elements in a sequence, is a very narrow application of AI, akin to playing chess. Usually very simple approaches like RL work totally fine for problems like these, but auto-regression/diffusion models have to take steps that are not well defined at all, and the next step towards solving the problem is not obvious.
As an example, imagine a robot trying to grab a tomato from a table. It's arm extends across 1 meter maximum, and the tomato is placed 0.98 meters away. Is it able to grab the tomato from the point it stands, or it needs to move closer, and only then try to grab the tomato?
That computation should better be calculated deterministically. Deterministic computation is faster, cheaper and more secure. It has to prove that: $tomato_distance + $tomato_size < $arm_length. If this constraint is not satisfied, then: move_closer(); Calculate again:$tomato_distance + $tomato_size < $arm_length.
From the paper:
> Our system employs a custom interpreter that parses "LLM-Thoughts" (represented as DSL code snippets) to generate First Order Logic programs, which are then verified by a Z3 theorem prover.
I don't understand your claim about 'Deterministic computation is faster, cheaper and more secure.' That's not true at all.
In fact, for many problems the fastest and simplest known solutions are non-deterministic. And in eg cryptography you _need_ non-determinism to get any security at all.
>[..] to first metabolic cycles, cells, multi-purpose genes, modular development genes, etc.
One example is when cells discovered energy production using mitochondria. Mitochondria add new capabilities to the cell, with (almost) no downside like: weight, temperature-sensitivity, pressure-sensitivity. It's almost 100% upside.
If someone tried to predict the future number of mitochondria-enabled cells from the first one, he could be off by 10^20 less cells.
I am writing a story the last 20 days, with that exact story plot, have to get my stuff together and finish it.
When math starts falling from the sky, generated by AI of course and proved with theorem provers, then everything will start falling from the sky. There will be a way to have more houses than anyone would ever need, for every person on the planet.
Funny thing is, most Coinbase or Kraken or Binance transactions of BTC and Ethereum are simple SQL transactions in a normal database.
An immutable ledger to put information is the best invention since sliced bread, and that can only be achieved by an underlying economic system to maintain such a ledger.
That being said, probably all crypto (except one) will go to zero very soon. Crypto is a bubble, but it doesn't mean that the underlying technology is useless.
Why not two people share a device, and when passed from one person to another, delete applications and install all apps and profiles from scratch using verified checksums saved on a blockchain. An OS which could do that is something like Nix. When passed to the previous person same thing, delete and install everything from scratch.
Using smartphones in a smart way, not a dumb way, like timesharing mainframes of the past. Same procedure could be applied to cars and other devices.
Android's Multiple Users feature does exactly this. Multiple users accounts with all user data completely sandboxed and restricted to each user. All user data is cryptographically protected on storage devices.
The actual SE filesystem available to a logged in user is pretty complicated. But the short story is that user-data is completely isolated. Presumably application binaries (which require digital signatures by default) are shared; although the "installed" state is not. Successive releases of Android have restricted access to any legacy "shared" data on the device (media folders particularly; pictures and video taken by the camera device have been strongly protected since Forever).
Verified checksums on a blockchain are only useful if they are verified by some provider who associates a blockchain ID with a real-world identity. Not sure what "blockchain" really adds. If anyone can create a blockchain ID, then "verification" doesn't really provide useful information.
> Multiple users accounts with all user data completely sandboxed and restricted to each user.
User data and user programs. Clean installation kind of user programs.
> Verified checksums on a blockchain are only useful if they are verified by some provider who associates a blockchain ID with a real-world identity.
Nix associates a unique id to each program version or package or config file. The verification happens on the Nix package manager.
The user uploads his exact config of OS somewhere, in his own home server, at a goverment server, at AWS, on a blockchain, somewhere. A blockchain seems like the best solution to me.
This assumes that these two persons will never need to use a smartphone at the same moment, which is a bit of a logistical puzzle.
Installing apps is the trivial part; isolating, or removing / reinstalling user data is much harder. Especially a few gigabytes of it. An SD card could work maybe.
This all goes against the grain of the smarthpone UX, the idea of a highly personal device that you can use for anything, and might need (or benefit from) at an arbitrary moment.
If the point is reducing e-waste, the solution would rather be opening up the hardware enough to provide long-term software support, LineageOS-style.
> This assumes that these two persons will never need to use a smartphone at the same moment, which is a bit of a logistical puzzle.
In general no one wants to share anything with anyone, but when two people cannot afford a device individually, but it is within reach when they buy it together, time-sharing becomes a totally acceptable solution.
> Installing apps is the trivial part; isolating, or removing / reinstalling user data is much harder. An SD card could work maybe.
Checksums might overlap by quite a bit. No need to remove programs installed by both users. If the total installation of each user is 10 GB, but the installation diverges 300MB only, not a big deal in most cases.
I only use open source LLMs for writing (Qwen 32b from Groq) and open source editor of course, Emacs.
If some people can write better using commercial LLMs (and commercial editors), by all means, but they put themselves at a disadvantage.
Next step for me, is to use something open source for translation, I use Claude for the moment, and open source for programming, I use GPT curently. In less than a year I will find a satisfying solution to both of these problems. I haven't looked deep enough.
reply