I cant wrap my head around how good youtube is keeping, serving, and querying massive amounts of video data so well. I don't know how a site like youtube is even possible.

I wonder if anyone has done the math at google if they can keep up. Each video is between 100MB to tens of GB I imagine, especially high-def ones. As more people come into the internet age, and more people start creating video content, the number of publishers, times the average number of videos a publisher makes over a lifetime, etc..

Same with Vimeo - the numbers have got to be astronomically slanted in favor of a cataclysmic loss of data at some point, or simply a policy of deleting things 10 years old (deemed no longer relevant). The internet archive is going to have its hands full if it's trying to back this thing up.

I mean to some degree videos that don't show up in search results and aren't linked anywhere are basically none existant anyway. I don't know why a site like youtube still serves videos like that. Youtube could remove at least 20% of their videos and not affect almost any users, but I don't know if it will ever come to that. The math is just absurd considering the amount of storage that youtube will have to serve 10, 20 years from now.