Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

> the trouble there is that the really fun case is to do this over the network with http (say to solve Python dependencies, to extract the metadata from wheel files) in which case this method doesnt work

If the web server can tell you the total size of the file by responding to a HEAD request, and it support range requests then it will be possible.

https://developer.mozilla.org/en-US/docs/Web/HTTP/Range_requ...

Or am I missing something?



You can't do this with mmap though, you can't instruct the OS to grab pages via HTTP range requests.


With userfaultfd(), you can. Not necessarily a good idea, though...


Write a fuse layer.


Or a signal handler (but yes, it is overkill).


You are correct, this works. There even is a file system built around this idea: https://github.com/fangfufu/httpdirfs




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: