I wanted to learn powershell, and I was interested in the history of worms on the internet, so I made a script to check https://isc.sans.edu/ and send me an email if the status was not green. Turns out, powershell has an all-in-one function to send an SMTP email, which I thought was hilarious. I used windows task scheduler to run it every hour.
Second, I was struggling with my Cable internet connection, and it was incredibly difficult to nail down what was causing my slow speeds. To help with the diagnostics, I wrote a little python script to download the files that Netflix uses for speedtests (check Fast.com) and logged the average of 10 runs to a file. It similarly used task scheduler to run every 10 minutes and as a result I now have a big old dataset of SpeedTests and unix timestamps which I graphed while playing around with Pillow
I wrote a python bot to Read project Gutenburg books to twitter. This was to participate in a contest that involved seeing which "team" could get a hashtag into more tweets. A twitter bot takes about five minutes to make, as I found a brilliant little Python-Twitter library. Unfortunately, my teammates got cold feet and convinced me to de-rate the bot's posting frequency, meaning we barely lost to a group of 60 or so sorority girls doing it the "correct" way. However, as we did it sort of below-radar, the loophole of automating it is still open and valid.
Currently I'm working on automating the process of creating photoreal scenery for Microsoft Flight Simulator. The FSX scene has built tools to turn open source land-use data into ground scenery like houses and forests, and Microsoft included the SDK and Documentation as part of the game[1], but since high resolution photo-real satellite photography is an incredibly expensive asset, nobody has automated that part of the process. I most likely will not be successful, as I have no source for the photo-real imagery.
Personally, I find that absurd, as my tax dollars as an American likely pay for incredibly high resolution (higher than 1 foot per pixel) satellite imagery of the entire world, and I'm not sure how the NSA providing that to citizens would hurt national security. The Stallman in me feels that I've already paid for that data, and it should be public domain. Hell, even a year old would work fine. The biggest problem would be Google and Microsoft lobbying to prevent the devaluation of their expensive assets.
I managed to confirm that my perceived changes in connection quality were ACTUAL changes in connection quality. I never took the time to do anything better with it though, but it does show that My provider seems to have done SOMETHING, as the speed floor has moved up, and significant events (sometimes as slow as dialup) became significantly rarer. It's also fun because it allows you to get a rough idea of how over provisioned the line is, as you can see day/night cycles.
My cable modem provides a status page with info about signal to noise ratio and other stats. I planned on adding more details to the script by scrapping from that page, but it requires a login first and I've never taken the time to learn how to manage that kind of state, as I do most scripty/webby things with raw requests from python's urllib.request module, like a plebian.
I used to do the same with my DSL connection. I noticed that the connection would drop or degrade when it rained sometimes. I started scraping the modem's status page with curl (the status page didn't require logging in), and graphing the SNR and data rates with gnuplot. Then I added temperature and precipitation data scraped from Weather Underground with a Python script to a separate CSV file.
I did confirm that the SNR varied with temperature, with both daily and long-term cycles, assuming from thermal expansion/contraction in the lines and connections. I also observed dropouts when it rained.
The strangest thing was that there was also a hard 12-hour cycle: every morning and evening, at the same times, the SNR would suddenly drop (in the morning) and go up (in the evening) by several dB. I never figured out what it was, but I always guessed it was some equipment turning on or off that caused interference on the line.
One of the coolest parts actually was simply using GNU `date` to parse dates in natural language, so I could run something like `plotmodem 2 months ago to 1 month ago` to graph the data from the previous month, or `plotmodem 2 weeks` to see the last 2 weeks of data. I used `date` to convert natural-language dates to a standard format which I then parsed out of the CSV files with grep.
Thanks for the links! The harddrive in my server died recently so I haven't had a chance to get it up and running again yet, but I will definitely update the script once I do!
That seems to be the prevailing theory, and in fact there's already a lot of work in the area, but I'm worried of falling afoul of terms of service. Scraping hundreds and hundreds of square miles of 1-foot per pixel satellite data from google might catch their attention
I wanted to learn powershell, and I was interested in the history of worms on the internet, so I made a script to check https://isc.sans.edu/ and send me an email if the status was not green. Turns out, powershell has an all-in-one function to send an SMTP email, which I thought was hilarious. I used windows task scheduler to run it every hour.
Second, I was struggling with my Cable internet connection, and it was incredibly difficult to nail down what was causing my slow speeds. To help with the diagnostics, I wrote a little python script to download the files that Netflix uses for speedtests (check Fast.com) and logged the average of 10 runs to a file. It similarly used task scheduler to run every 10 minutes and as a result I now have a big old dataset of SpeedTests and unix timestamps which I graphed while playing around with Pillow
I wrote a python bot to Read project Gutenburg books to twitter. This was to participate in a contest that involved seeing which "team" could get a hashtag into more tweets. A twitter bot takes about five minutes to make, as I found a brilliant little Python-Twitter library. Unfortunately, my teammates got cold feet and convinced me to de-rate the bot's posting frequency, meaning we barely lost to a group of 60 or so sorority girls doing it the "correct" way. However, as we did it sort of below-radar, the loophole of automating it is still open and valid.
Currently I'm working on automating the process of creating photoreal scenery for Microsoft Flight Simulator. The FSX scene has built tools to turn open source land-use data into ground scenery like houses and forests, and Microsoft included the SDK and Documentation as part of the game[1], but since high resolution photo-real satellite photography is an incredibly expensive asset, nobody has automated that part of the process. I most likely will not be successful, as I have no source for the photo-real imagery.
Personally, I find that absurd, as my tax dollars as an American likely pay for incredibly high resolution (higher than 1 foot per pixel) satellite imagery of the entire world, and I'm not sure how the NSA providing that to citizens would hurt national security. The Stallman in me feels that I've already paid for that data, and it should be public domain. Hell, even a year old would work fine. The biggest problem would be Google and Microsoft lobbying to prevent the devaluation of their expensive assets.
[1] https://msdn.microsoft.com/en-us/library/ff798293.aspx