without the rational bloom filter, no compression can take place. The new idea is rational bloom filters and this is just an example of how a rational bloom filter can be used to compress video. Its a novel approach to video compression. I posted an update on the top of the thread.
I received some great feedback so I have decided to focus on raw video for now and more rigorous testing around noisy videos. I will continue to update the repo frequently. Its only very early days but the raw video testing has yielded some decent results(with some caveats).
- Compression ratio of 4.8% (95.2% size reduction)
- Compression speed: 8.29 frames per second
- Decompression speed: 9.16 frames per second
- Only 4% of frames needed as keyframes
- Perceptually lossless output (PSNR: 31.10 dB)
### Current Limitations and Future Work
While the compression results are promising, the system is not yet truly lossless in its handling of color channels. The current implementation faces challenges with the YUV to BGR color space conversion process:
1. *Color Space Conversion Precision*: Small rounding errors occur during conversion between YUV and BGR color spaces, leading to minor differences in pixel values (average difference of ~4.7).
2. *Channel Processing*: The current implementation processes color channels in BGR format after conversion, which introduces additional precision loss.
The plan moving forward is to:
- Implement direct YUV processing without conversion to BGR
- Develop a true bit-exact handling of color data
- Refine the Bloom filter parameters specifically for chroma subsampling patterns
- Create a dedicated verification system for each color channel independently
I want to be able to prove its mathematically lossless but I am quite a ways from this. I plan to pursue this lossless compression idea and I have a few other ideas in different domains around using the rational bloom filter.
Author here. The repo is a complete mess but I do have some some code in there to generate graphs and whatnot if you're willing to dig through the code. I will make this much more concrete with lots of proper testing. Its very much still a messy work in progress.
I applaud you for uploading even if it’s a bit disorganised still, and I do the same. It’s better to have something than nothing. Sometimes I see people talking about something but they don’t want to upload their code yet because they have to clean it up first. And then either they don’t get around to ever cleaning it up the way they wanted, or by they time they do it will have fallen off everyone’s radar and be forgotten. At least with a messy repo it’s possible to poke around, and not the least to star it and maybe check back later to see if they cleaned it up.
Author here. Totally agree that H.264 can be lossless. Generally its lossy. My idea(which I am still working out) is to compress the difference of frames using a rational bloom filter. I previously posted here about using conditional bloom filter that rely on rational k. The idea was to use different values of k based on whether the url was more likely to be malicious than not. This results in a lower fp rate for the same filter size when compared to integer k. I then saw this paper[https://arxiv.org/html/2502.02193v2] was posted recently which describes an almost identical approach(theirs is much nicer). I will do much more rigorous testing as my current setup is a bit sloppy but I hope it illustrates the idea.
So it sounds like the use of rational Bloom filters here is just to get a better compression ratio, but the basic technique could be used with classic Bloom filters—is that right? Do you know how much you gain in space savings from using rational Bloom filters? It’s not obvious to me how much of a gain it would be.
I think you might be able to compress I and P frames somewhat descent, this way. But you only seem to address spatial domain, except for deltas(?) Or do you have some way to apply bloom filters to motion estimation as well?
https://consistent.fun/ . I have always been an extremely inconsistent person when it comes to building habits. I am sure most of us have read the book atomic habits(if you haven't i'd highly recommend it). It's amazing how small habits done each day can snowball into huge life changes. I made this app to help me visualise my micro habits in a colourful yet low pressure way. I don't want to feel anxiety when I open up a productivity app. At the end of the day I think productivity and self improvement boils down to just being consistent with doing a small bit everyday. Feel free to give it a try if you think this is something you'd be interested in. It's free to track up to 3 habits and its €2 per month if you want to track unlimited habits. Open to any feedback or suggestions. It was mainly designed for a desktop browser but should look decent on mobile browsers. I would make this entirely free but I gotta pay monthly costs for a database & digital ocean
I have been using it since I developed it so about 2 weeks. I made it available for anyone to use today. I just wanted something simple that I could use but didn't give me anxiety every time I opened it like some other apps.
I have gotten that feedback a few times so I will have to fix this. Essentially the reason you don’t see any comments is that nobody has commented on that location yet. I should have probably pre-populated different areas but it’s quite awkward to pull in information based on long and lat coordinates without relying on paid apis like google places api or foursquare places api which can get quite expensive.Perhaps there is an easier way to scrape certain sites but then again it’s a bit complicated to match that scraped data to a long & lat coordinate. I am experimenting with adding the ability to have a dynamic link which updates based on long & lat coordinates which will then open a perplexity search query for stuff in that area(which is free) and possibly a notes section where you can add notes(trip planning etc) that are saved in local storage. I want to keep this entirely free to use and minimal cost for me to host. If you have any recommendations I would love to hear them. Thanks- R
