Nearly all questions answered in the "about map" link:
=========================================
The data behind this map is based on every geocoded tweet in the United States from June 2012 - April 2013 containing one of the 'hate words'. This equated to over 150,000 tweets and was drawn from the DOLLY project based at the University of Kentucky. Because algorithmic sentiment analysis would automatically classify any tweet containing 'hate words' as "negative," this project relied upon the HSU students to read the entirety of tweet and classify it as positive, neutral or negative based on a predefined rubric. Only those tweets that were identified by human readers as negative were used in this analysis.
To produce the map all tweets containing each 'hate word' were aggregated to the county level and normalized by the total twitter traffic in each county. Counties were reduced to their centroids and assigned a weight derived from this normalization process. This was used to generate a heat map that demonstrates the variability in the frequency of hateful tweets relative to all tweets over space. Where there is a larger proportion of negative tweets referencing a particular 'hate word' the region appears red on the map, where the proportion is moderate, the word was used less (although still more than the national average) and appears a pale blue on the map. Areas without shading indicate places that have a lower proportion of negative tweets relative to the national average.
The numbers that appear in the map during a mouse hover indicate the total number of hateful tweets and number of unique users sending them in each county.
==========================================
EDIT: The mouse overs don't appear to work very well in Chrome or Firefox, but from the one or two times I was able to see some numbers it appears that each red circle may be a dozen or less tweets. Also, the hot zones dissipate significantly the further you zoom in, so without any statistics or numbers it's difficult to draw conclusions.
A very interesting experiment, but given that the data is only normalized by Twitter traffic (non-response bias) this is in no way indicative of the actual distribution of racism.
> Because algorithmic sentiment analysis would automatically classify any tweet containing 'hate words' as "negative," this project relied upon the HSU students to read the entirety of tweet and classify it as positive, neutral or negative based on a predefined rubric. Only those tweets that were identified by human readers as negative were used in this analysis.
I wonder how well a Bayesian classifier would work if the this was used as a training set. If it worked relatively well, there's no reason why you couldn't create a live version of the map.
Not very well. Twitter sentiment is a difficult problem.
Consider using millions of training examples (vs. thousands). This was done as part of the "distant supervision" Twitter sentiment technique. What this means is that tweets with positive emoticons were labeled as positive sentiment, and negative emoticons were labeled as having negative sentiment. Emoticons were stripped before training. This system got 80% accuracy.
I want to see that predefined rubric. I am unwilling to believe Iowans are more racist than Mississippians. Being racist in Iowa means hating like 3 people in the next county over.
As someone originally from MS, let me clarify that the state is not what most people imagine it is based on various movies or their U.S. history class. Mississippians take "the hospitality state" seriously.
To be clear, I'm solely focusing on the opportunity for racism. E.g., it's not an accident that the Germans, in the aggregate, that anti-Catholic sentiment in the 1850s was significantly stronger in the North than in the South.
It seems as though this comments section is filled with self-proclaimed "night owls." I'm curious, have any of you read the linked articles about variables that influence people to remain awake later than they should and have these types of erratic sleep patterns / modified circadian rhythms (heavy computer use / blue light exposure)? And, have any of you tested your own reaction to curtailing these late-night activities while introducing natural sunlight stimulant in the morning?
I'm not asking in a derogatory way; I'm genuinely curious since I experimented with this myself and found positive results. My guess is that most haven't, and that it's easier just to accept a "night owl" classification in lieu of hundreds of thousands of years of human dependence on the sun and fire for light. Based on my limited research it seems that those without significant mental illness sleep well with habit and controlled stimuli (including natural light). Has anyone, through any amount of reasonable testing, found this NOT to be true?
I have always been a night owl. With or without computers. Even when I spend time camping out doors. I like to stay up, I don't like waking up.
Once as an experiment I began waking up an hour later every day. I went around the clock, felt great, couldn't stop my body from doing it again, had to work hard to avoid it happening a third time.
Not exhaustive testing, but as a child (before ever having a computer, and with very very limited television access) I would stay awake for hours in the dark, and be dragged out of bed in the morning well after the sun had come into my room.
The past month I've been up at 6am, rather than ~9am. As an experiment rather than for any good reason.
What you say about light and stimulation sounds convincing, but for me it was about habit. I was in the habit of ignoring my alarm. I was in the habit of getting up when I needed to, and no earlier.
To start off with, I used an alarm app that wouldn't deactivate until my phone touched my credit card, left in the shower. 'Sleep as a Droid', Strong recommend.
I'm still not convinced this is a viable solution for waste management (not that there's a simple alternative), given the environmental implications. Yes, air pollution is pretty regulated and has been reduced significantly since the 80's[1], but the residues resulting from the filtering process are still heavily toxic and it's not clean by any means[2].
Unfortunately, the vast majority of Groupon customers do not return for additional purchases and the people who capitalize on these types of deals are 100% price-conscious vultures[1] (also, I used to be one of these people). At the end of the day, this was a very poor business decision and just a PR gamble.
People are primarily driven by price when they buy goods, so after 6 months or so people will forget this extremely generous act. Razer fans will remain razer fans, but unless Razer has a product leaps and bounds ahead of the competition (some are, but most are only marginally superior) very few will become loyal customers due to this act of kindness.
The thing that's a bit different about this is that most of these customers are NOT price-conscious vultures, they are enthusiasts. Otherwise, they would have said "Razer? Who the hell is that?" These products are not well known outside hardcore PC gamers. PC gamers are willing to drop a TON of cash on their hobby or else they wouldn't be hardcore PC gamers.
The longtime Razer fans to some degree seem to be canceling their orders. At least with the ones you'll be shipping still I'd think the majority of those are strong PC gaming enthusiasts who have friends who are as well. Their stuff is quite shiny and attractive in general.
I might buy a mouse because of the deal, and if the quality is as good as you say, I'll be back in the future for more. But as of now, mice are commodity to me--the coupon helps push me into trying out a "luxury mouse". It's risky but perhaps not ineffective customer acquisition.
Slightly off topic, but is anyone else confused by the fact that several runners in this video[1] reacted to the explosion but did not turn and proceeded to keep running? These people weren't exactly leading the pack (at a 4+ hours finish time), so my guess is that they are just severely deprived of nutrients and aren't processing what's happening, but it's still a bit baffling to me why some didn't turn to see what was going on.
*EDIT - thanks to those who confirmed. My confusion is now totally irrelevant after seeing the finish-line view of the same people[2]. It just took them a few seconds to register and they were just in a running flow.
I've never run a marathon, but I've attended several supporting runners I knew. I spent my time at the finish line watching the runners come across.
Pretty much everyone who finishes is some level of fucked up. Their body chemistry is way-the-hell out of whack, with problems from dehydration, hyponatremia, glycogen depletion, and boosted or dampened immune system reactions. Usually there are people at the finish looking for trouble cases and shunting them directly to the medical tent. (Because the runner might not even recognize that how bad a state they are in.)
I've had hyponatremia once after an ill-advised hike, and I literally couldn't form sentences out loud.
It doesn't surprise me that people weren't reacting in a normal way, because they weren't in a normal state of being.
Most people simply don't know how to react in a situation like this. Is normal to do silly things when normal people is inmersed in an emergency. Some people take photos standing in the way to the exit, others flee away in the wrong direction creating a risk of stampede, most not trained people (and even some of the trained people) do exactly the type of things that you should not to do... is pretty habitual.
If you've ever run a marathon, you'd know. 4 hours is still a pretty decent time for 26 miles and, for most runners, the last leg up to finish line is usually a pretty exhausting/euphoric time. Aside from your own heartbeat and breathing, you really don't notice much else.
To me, it looks like they respond physically to the shockwave but are not mentally registering anything other than the finish line.
I raced bicycles in college and I can say with confidence that after a 4-6 hour training ride I was wiped out physically and mentally. Had an explosion gone off when I was in that state and only a few blocks from home I would have made sure I was OK and kept pedaling for gatorade and a shower. I don't see their behavior as terribly puzzling given my fairly extensive experience with an exhausted state of mind.
I had done serious mountain biking and my worst injuries are not from doing extremely dangerous things like hard dropping from a 3 meter slippery stone, witch is something you prepare very well, but for really really stupid errors when being physically, and specially mentally exhausted.
I was 2centimeters away from losing an eye, with itching and hurting wounds in my face, some fractured bones.
On retrospect it was like part of my brain disconnected for most of the time when exhausted.
I could understand the man whose legs stop supporting him with the blast, hopefully he recovers.
I spoke with someone in Grande's (competing ISP) network planning division and he explained that getting into new buildings downtown was very expensive / difficult given Austin's telco requirements (have to dig up streets, etc.). Is this also the case for Google's fiber layout, or is there any speculation that buildings (condo and apartment) downtown will see this service?
Am I the only one dissatisfied with the way media delivery has played out the past couple of years? We started with a rapidly developing delivery system (the internet) and content makers who couldn't see the value in capitalizing on this sales channel by making movies, TV shows, and songs available to consumers at reasonable prices (read: less than physical media costs), which led to piracy, global dissatisfaction, and decreased industry revenue.
Fast forward to 2013, where we have...
Ecosystems: iTunes, Amazon Video, Netflix, Hulu, Vudu, Blockbuster Ondemand, Android, Cinema Now, and Vdio
Which can be accessed on TVs, web browsers, smartphones, tablets, dvd players, bluray players, gaming consoles, media streaming devices (Roku, Apple TV, etc.), and HTPCs.
... all with different user experiences (most are poor), varying media coverage / depth, and a pricing system that only satisfies bulk consumers that want to watch a bunch of legacy content.
Larger companies that were first to build relationships with content makers have been the most successful, but the entire process still wreaks of bureaucracy, middle-men, and band aids. Netflix seems to be the only one that realizes this, and has started producing their own content (e.g. House of Cards and Arrested Development), and I think this is the only true catalyst that will change the industry for the better.
I'm not saying it's a perfect model, but the alternative is having to pay $100/month for a cable subscription to watch garbage you don't want to see, or $10+ to see it in a dirty movie theater, or $15 to "buy" a movie that was $15 in the 90's for a physical DVD. Even if there are 5 Netflixes competing against each other I would rather pay 5 x $7.99 a month to watch content I actually find relevant than pay for junk I don't use.
If anything, merging content with distribution sparks competition to produce content at reasonable prices. Netflix has also resorted to releasing content in a more enjoyable manner (all at once vs. weekly with commercials).
The original point was that a "netflix" provides better content than cable to start with.
The argument against it was that the company providing the content would be exclusively providing that content.
The rebuttal is that it would be better to pay 5 companies that all produce exclusive content than to pay for cable, which "provides" mostly stuff you don't want to see (and frequently don't have the option to not watch, if you're viewing any of their content, ie: commercials).
And my rebuttal is that paying 5 (or 10, 20, 50) companies that all produce exclusive content is almost exactly what cable is, the only differences are you pay a middleman instead of the companies directly, and usually can't choose individual providers, they come in preset packages.
I'd rather see independent producers distributing their content over various netflix-like providers, otherwise we're just transplanting our current content production model to the web. That will suck for both consumers and producers.
"Netflix seems to be the only one that realizes this, and has started producing their own content (e.g. House of Cards and Arrested Development), and I think this is the only true catalyst that will change the industry for the better."
There's a big difference between a company like Netflix producing their own content, making it available for watching uninterrupted, whenever you like -vs- any of the content providers that charge cable companies for their content streams that include commercials and are generally not available "on demand", packaged together through a single company that won't give you just the content you want, when you want it, but charges you for some combination of provider's streams in a package.
Hence the argument of "the alternative is having to pay $100/month for a cable subscription to watch garbage you don't want to see". 5x7.99 is not the cost of cable. The only cable packages you can get for that price are extremely limited. Also, they will require another $4.99 a month for on-demand, on top of your regular cable fee, unless you're paying more than $100 a month, in which case, it's included. ¬.¬
In that scenario most of the content you are paying for multiple times. Or you can look at it as the redundant content is free and you're paying $7.99/month for House of Cards.
Wouldn't it be better if prices for the content you want were more reasonable? Like, if you could buy a season of House of Cards for $7. And buy an old movie for $0.25?
Like, if you could buy a season of House of Cards for $7
I'd like to buy a new porsche for $10k but that's not happening either.
Most of Netflix's shows cost over $4M an episode[1]. Taking that as a base you've got a cost of $48M a season (these are production costs not total cost, at $7 a season you need 7M subscribers just to break even ). Considering most shows have less then 4M viewers a night your pricing model pretty much is just throwing money away.
If it were to be used, how would advertising factor into this? Not saying $7/season is viable but could a low price be buffered out by advertiser sponsorship?
It's at least a step up from how studios like HBO operate. If you want to watch HBO's content, there's no legal way to do so without being part of an expensive cable package. If you wanted to watch House of Cards and you weren't already a Netflix member, you could just pay the $8 (or get a free first-month trial) to essentially rent the entire Netflix catalog for an entire month. That's what you might have paid not too long ago to borrow a single scratched-up DVD from Blockbuster for a few days.
There is a hefty artificial delay during which you cannot get access to newer content that way. I don't see last weekend's episode of Game of Thrones available anywhere other than HBO Go and torrent sites, currently.
Check out Vhx.tv too. A really great new service. I think this is the first I have seen a place go in the right direction. Or at least in the direction I would like to see streaming/downloading video services to go.
I'm skeptical she'd take this kind of action, but maybe Adria is more naive than I thought. First she is shocked at the reaction of her public ostracization, then she thinks suing an ex employer is within her best long-term interests? Even if she wins (not likely, contrary to the legal opinion submitted here last week) and gets a few hundred thousand dollars she'll be even more blackballed by the rest of the industry than she already is.
Her ONLY safe play, long-term, was to apologize for how she handled the issue (which was not exclusive to pointing out the inappropriate commentary!) and make a lateral career shift of some sort.
I'm still amazed there are journalists reporting that the developers made sexist jokes, and furthermore embarrassed at America's inability to be comfortable with sexuality. The dongle joke was 5th grade humor and the forking comment had 0 implied innuendo. Somehow these privately-told jokes, somewhat sexual in nature, were inferred as sexist, and people lost their jobs because of it!
This whole situation sickens me--the threats against Adria especially--but the only thing truly surprising at this point is that Adria did not envision her attempt to summon pitchforks would backfire, with pitchforks being turned on herself. This is the kind of naivety that you would expect from someone unfamiliar with the internet; not someone who works with programmers for a living.
Cool idea and I'm a big fan of the simple design. I have a friend here (Austin resident myself) who has been painfully vetting options this month through CL.
Looks like some of the filtering tools don't work -- I selected a min and max age of 23 and 30 respectively and the results shows people beyond that range. Screenshot: http://puu.sh/2lZ6m
I also think it would be a good idea to take a few pages out of Airbnb's book of how they handle reputation and let people link their Facebook account, as well as offer a background-checking service for people to help boost their marketability. A references function (like Linkedin) would also help.
Thanks for pointing out the bug with the filtering. I appreciate it and will fix it right away! Airbnb is amazing and I totally agree with you that we need a verification tool. That is something I have been looking into but in the mean time I thought it would be good just to get the site out there. I don't have tons of time so I have to pick and choose which features to add carefully so it all makes sense. If you have any other suggestions I'd love to hear them.
=========================================
The data behind this map is based on every geocoded tweet in the United States from June 2012 - April 2013 containing one of the 'hate words'. This equated to over 150,000 tweets and was drawn from the DOLLY project based at the University of Kentucky. Because algorithmic sentiment analysis would automatically classify any tweet containing 'hate words' as "negative," this project relied upon the HSU students to read the entirety of tweet and classify it as positive, neutral or negative based on a predefined rubric. Only those tweets that were identified by human readers as negative were used in this analysis.
To produce the map all tweets containing each 'hate word' were aggregated to the county level and normalized by the total twitter traffic in each county. Counties were reduced to their centroids and assigned a weight derived from this normalization process. This was used to generate a heat map that demonstrates the variability in the frequency of hateful tweets relative to all tweets over space. Where there is a larger proportion of negative tweets referencing a particular 'hate word' the region appears red on the map, where the proportion is moderate, the word was used less (although still more than the national average) and appears a pale blue on the map. Areas without shading indicate places that have a lower proportion of negative tweets relative to the national average.
The numbers that appear in the map during a mouse hover indicate the total number of hateful tweets and number of unique users sending them in each county.
==========================================
EDIT: The mouse overs don't appear to work very well in Chrome or Firefox, but from the one or two times I was able to see some numbers it appears that each red circle may be a dozen or less tweets. Also, the hot zones dissipate significantly the further you zoom in, so without any statistics or numbers it's difficult to draw conclusions.
A very interesting experiment, but given that the data is only normalized by Twitter traffic (non-response bias) this is in no way indicative of the actual distribution of racism.