It's incredible how Shannon published "A Mathematical Theory of Communication", while actually working on the "Communication Theory of Secrecy Systems":
What an absolute genius who disrupted two fields in two years.
I was exposed to Shannon's channel capacity theorem of a binary erasure channel in a grad comms theory course, and totally fell in love with the beauty and elegance of his mathematical formulations. Since that day, teaching Information Theory has been my most favorite hobby--which also used to a career. I love to see students' eyes pop when you eventually show how elegantly simple it is to express disorder (entropy) and to find fundamental limits on source and channel codes using this expression.
Claude Shannon also had early ideas about machine learning. AT&T has a 1950s video of the maestro himself demonstrating a maze solver with physical output. A true genius. In those days, geeks were dressed in suit.
In EE we of course learned about the Shannon capacity of a channel, but when I read his proof using higher dimensional sphere packing I was amazed. Inspired me to attempt to look further into the math we see in engineering. I highly recommend anyone interested read his paper.
Yeah, I just re-encountered Shannon while reading about the board game Hex. The Shannon Switching game is the base case of Hex. Guy was firing on all cylinders ;)
Since we are on the topic, does anyone know what Shannon was up to after disrupting 3 fields?
...
He got into ball juggling and built the mathematical theory that led to creation of an automated juggling machine: https://cs.stanford.edu/people/eroberts/courses/soco/project....
He actually showed that ball juggling is one of the most complex cognitive tasks that a human can undertake:
"The physical constraints that affect mastery and limit the number of objects juggled arise from gravity - more specifically, Newtonian mechanics (h=1/2gt2). Each ball must be thrown sufficiently high to allow the juggler time to deal with the other balls. The need for either speed or height increases rapidly with the number of objects juggled."
What else? Well, one thing was making a lot of money as an investor. I don't remember quite what Fortune's Formula said about his overall rate of return over many decades, but iirc he made less than Warren Buffett mainly because he started with much less. (That book is about information theory and betting.)
I read Gleick's The Information[0] a few years ago and was riveted. Do you have any suggestions for further reading if I wanted to take a hobbyist interest?
I would read "Lifespan" by David Sinclair, who used Shannon's information theory to model epigenetic changes that cause us to age. Fascinating read and goes to show how far reaching impacts a foundational mathematical theory can have.
Heh, I think I'm going to start bringing up this paper whenever somebody tells me they majored in communications. This stuff is fascinating! Shannon really was a genius.
https://pages.cs.wisc.edu/~rist/642-spring-2014/shannon-secr...
https://pure.mpg.de/rest/items/item_2383164/component/file_2...
What an absolute genius who disrupted two fields in two years.
I was exposed to Shannon's channel capacity theorem of a binary erasure channel in a grad comms theory course, and totally fell in love with the beauty and elegance of his mathematical formulations. Since that day, teaching Information Theory has been my most favorite hobby--which also used to a career. I love to see students' eyes pop when you eventually show how elegantly simple it is to express disorder (entropy) and to find fundamental limits on source and channel codes using this expression.