I think it's sad that he had to resort to publicly releasing this exploit because he couldn't find a way to contact Google about it.
In the past, when I've had problems, I couldn't contact them either. They've done a great job at making sure there's no human contacts available. You have to post something in a public forum and hope they'll contact you. (They won't.)
Sending an email to security@google.com will result in a quick response. As part of their bug bounty program Google would have paid $1,000 for this bug if not more.
The biggest problem when the 911 (and equivalent emergency services) was getting the word out there that it existed. It required a huge, years-long marketing campaign. Perhaps they should have inhouse ads targeted at keywords like "report vulnerability".
I emailed security@google.com when I found a security issue with Google +1 (http://news.ycombinator.com/item?id=2630355) but didn't get any response. I guess it's not really an issue for them.
No, it won't. Trying to contact Google when I was working for a major corporation and TRYING TO GIVE GOOGLE MONEY resulted in the exact same issue as the OP, and this was in 2006, not 2011. The only way to get their attention was create a blog post about how I couldn't contact them.
Sometimes (not necessarily in this case) #1 on HN. Being noticed in the right place at a right time may proove to be much better for one's life than a quick money boost.
One of the main reasons they have a bounty program is to prevent people releasing information about bugs before they have been fixed. I don't see why they should give him a bounty.
I'm not sure how he was unable to find their security@google.com email address. Searches like "Google security" and "Google report vulnerability" have http://www.google.com/about/corporate/company/security.html (which has a prominent section on reporting security issues) as their first result.
Because now you know the email is spelled "security@google.com", it would be easy to come up with search query that returns that email address after the fact. If you going by the instinct, search for "google bug report", ...I'm on page 20 and still couldn't find that email. (Personalize search turned off.)
Because the security email address is not a place for bug reports: it shouldn't be ranking for that query. If it did, the team would just be swamped with "bug reports" ;)
Try searching for "google report security" or "security vulnerability google" and you'll find the right information.
Which is sad, because in this case we have a security vulnerability and a bug at the same time, so how one expresses that in a web search depends only on one's current mindframe and set of associations. Eg. when I hear "Google security" I imagine guys with guns, not a place for bug reports.
Any company worth its weight that stands to lose big time from a security vulnerability will have a page with info on how to report it to them, and searching "[company name] security" will usually get you right to it. Disclosure is what you do after you've contacted them and not heard back.
I don't think it's reasonable to expect most people not immersed in the proper culture to see this as a "security" issue. They'd agree with "vulnerability" if you suggested it to them, but "bug" is really the first word that comes to mind.
It's obnoxious how hard it is to report bugs to Google. And posting in their forum is a joke anyway. Google's new two-factor authentication? Really neat right? Yeah, well, it's buggy and there is no way to report bugs for it. I posted in the forum and was received by crickets.
I don't mind it most of the time, but when I have a real issue or something that is obviously broken and unnoticed, it sure is frustrating.
edit: There's also no category for "generic login problems" or "Other". So I'm stuck posting it in Gmail.
My favorite has always been the google apps problems that tell you to contact support to get a resolution but you can only contact support if you are a paying customer.
I am a google apps paying customer and it took them 4 weeks to get my domain issue corrected ! In that time i wasnt able to get mail so i had to change my mail to use godaddy until they got back to me. When they finally did they said it was because my dns records were pointing to godaddy ! After 1 week of not getting mails i would say an alternative is required. I actually had a feeling that this would happen :)
I really love google as a company, their products, api's, their talks and all the events they hold are so great, i have learnt alot, BUT man, when i couldn't access my mail as a paying customer and got the runaround for support for 4 weeks even though they said it was high on the priority list ... GRRR thinking about it just makes me mad.
Anyway on a good note, after this issue everything is working fine and i guess aslong as you dont need support then everything is good :)
Sadly, from what I have read over the years, your experience is not a solitary one. I hope Google, at some point, will address their customer service issues.
Setting up your DNS is not really something Google should do for you. You should have had an IT pro who's done this sort of transition before, manage the move. You could have had this resolved in hours not weeks. I agree though Google support sucks, the key is to not depend on it.
What problems have you run into? I recently switched over to 2-factor auth and while it's working fabulously I'd be interested to learn about any problems I could run into.
This bug could have been exploited for millions of dollars. Imagine giving a mafia boss control over the heartbeat of every rival. One blackhat SEO could have dominated any number of lucrative keywords.
If this bug has existed for a long time it's quite possible some guy is sailing around on a yacht that this bug paid for.
It's such a blindingly obvious bug that I really do wonder whether this might have been a backdoor/inside job by an employee. Google should very closely inspect the code change history.
Hopefully they also maintain a history of all page removal requests to see who might have been exploiting this.
It's not like savvy website owners have portals into the elusive world of WTF Is Happening To My SERP Rank? Nor do savvy website owners have magical avenues to rectify such events.
In other words, I'm not confident in the speed you can get Google to remedy an event where your website disappears from the results.
It is your responsibility as someone doing business to check up on it, and proactively take action if/when rankings go down. To not do so makes no business sense. Of course, if you're just a casual blogger who doesn't care about SEO, it doesn't matter. But if you do care, not checking is foolish
It's not a back door, it's an abuse of an existing approach.
Google could weight the process in one of two ways:
1. in favour of the complaint-maker.
2. in favour of the website-owner.
If they favour the complainant, then website deletion is presumed to go ahead. If the webmaster, then it is presumed to be held up.
Google chose a compromise: the complaint is acted on, after a delay. The webmaster gets notified through webmaster tools; after some period of time the removal goes ahead.
If Google flip the compromise around, they will make it nigh impossible to remove any websites from the index.
I think you're misunderstanding the article. Google webmaster tools allows the website owner to request links to their own sites be removed. The poster has discovered that this form can be used to request any url be removed, and Google will think it's being submitted by the owner of that URL.
This has nothing to do with users complaining about a URL.
Despite it being "fixed" not long after the blog post went live, I wonder how long/how many people knew about this bug. Seems like it would be a great trick for SEO (build page to certain PR/remove opponents ranking above you)
He included screenshots and a description. Neither are impossible to fake, but either it's a genuine mistake (in which case I would imagine someone would have pointed it out) or he's faking it.
You seem to be accusing him of faking, without any evidence or even a motive.
Also, what is the link you supplied supposed to show? To me, it's like someone who, hearing the accuracy of Genesis' creation story challenged due to lack of supporting evidence, replies, "Yes, but Exodus says the same thing." It's the same source making the same claim, with no corroboration.
You imagine that someone would have pointed out an honest mistake, but that could take time. Is absence of an immediate counter-demonstration proof (or even evidence) of truth? I don't think it works that way. See Fleischmann–Pons.
I understand your point, but this is different to Fleischmann–Pons because of Google's capacity to change things.
In any case, http://searchengineland.com/google-disables-url-removals-aft... appears to indicate that Google has disabled the tool and are investigating. That indicates to me that it isn't totally fake at least. Maybe it only happens in some circumstances, or maybe there is some other explanation, but I doubt Google would disable the tool completely if what he claimed didn't have some basis.
FWIW, I just submitted a removal request via GWT for a page on my own site, and the request was accepted without complaint and is pending. Maybe Google turned the tool back on already.
I tried a request for a page on my wife's site, via the prescribed URL mangling, and it was silently ignored.
re: Google's capacity to change things, that's exactly what I thought should trigger more skepticism: Not only has no one repeated the experiment -- no one ever can. Now it's history, and that can get murky.
But thank you for the link -- another voice is good evidence in my book.
Was it the "I'm not saying it's fake" part that made it seem to you that I seem to be accusing him of faking?
My point is that the evidence is wholly insufficient. It was the first post on a new blog (i.e. no reputation), no way to reproduce the reported issue, no reports of it having been reproduced by anyone else, no acknowledgment from Google (ok, maybe it's a little early for that), etc. I mean, from a journalistic, much less scientific, standpoint, it's pretty poor.
How can I provide evidence of a negative? Would you please provide evidence disproving my assertion that flying saucers visited my house last night? And motive? Wasn't that covered earlier by the word "linkbait"?
Maybe my calibration is way off today, but I'm surprised by the level of credulity I've been seeing. My original post (which I deleted and then reposted, sorry about that) got downvoted to subzero with no explanation. I agree with other posters that bugs happen, but I would have thought that such a major claim against a generally competent player like Google would require at least one independent verification.
I'll be curious to see if Google mentions this. Otherwise, we'll really never know. Well, you may, but I guess I'm a little more skeptical.
I think there were multiple reasons you were being downvoted:
1) The grandparent to your post accused the post of being linkbait, without explaining it in anyway. You appeared to be supporting this without any evidence that it was linkbait.
2) You took the discussion about it being linkbait, and introduced the accusation that the post was faked (not withstanding your "I'm not saying it is fake thing" - if you aren't saying that, then what are you saying?), which was totally unrelated to the thread of conversation.
3) You didn't provide any justification that it was fake - merely circumstantial evidence that doesn't preclude that it could be fake. It is pretty normal for someone to expose an exploit without independent verification - especially since it appears the author hadn't considered it a security hole (note the article doesn't mention security, and they didn't find the security@google.com email address which is pretty obvious if you are using the term "security" when you search).
Given these factors, I think the downvotes were justified.
I don't think there is anything wrong with being suspicious, but I think your post would have been better received if it had presented your suspicions differently.
I'll admit that the quotes around "screenshots" exposed a bias (though I privately considered "honest mistake" a real contender), so I'll take my lumps.
But I stand by my main point: I don't see any REASON to BELIEVE.* The only thing separating malice, incompetence and good work here are the unknown motives of some guy with an internet connection -- possibly an NYU student of Panos Ipeirotis, if you caught that story. I think it was fair to ask why people felt so confident with their "OMG! Google is incompetent!" posts, so quickly. I guess I picked the wrong grandparent.
* You've kindly provided a corroborating source, so "good work" is looking more likely.
How does the article deliver? I don't see, in the comments on his blog or here, any evidence that this has been independently reproduced. He's got a description, and some "screenshots" and, oh, Google seems to already have fixed it. How... convenient?
I'm not saying it's fake, but I don't see any reason to believe it. Am I missing something?
Does this _actually_ work though? You get the message "URL pending for removal" but does that mean it's really going to be removed? Perhaps this is just a default response.
I don't know how is possible that a so obvious bug passed their quality department, and I wonder if someone didnt discovered it before and was doing this to take out competitors indexes..
Bugs happen. Even big ones like this. Any engineer worth his money knows that no amount of Q&A will discover 100% of the bugs. But, as Joel Spolsky said somewhere, bugs are just bugs, you fix them and then they're fixed.
I know, I am an engineer and I obviously let bugs pass too. But this is a little too obvious to me, to check if the user is allowed to remove this url. Maybe I am neurotic? :)
I can see having this pass by a reviewer or two. They look and see all of this:
- There are permission checks
- The user has to be logged in to GWT
- The user has to have access to this page
- The user has to be the owner of the siteUrl
After all those permission checks, it might appear that everything was covered. It's just one little omission, verifying that the urlt parameter corresponds to a page within the siteUrl website, that was missed.
This is a pretty amateurish mistake, actually, and I'm shocked that it was in production at Google. Proper authorization checks are web programming 101.
If you make a checklist of security practices the QA testers should look for, they'd see and check off "proper authorization checks", as they were done on other fields of the same page. If you can't imagine a professional making this mistake, your mental image of an engineer is not realistic. Humans are not that perfect, and this mistake does not make everyone that reviewed this code an amateur.
Well, people are down voting me, but everyone ripped the developers of Diaspora apart for basically the same exact flaw in an early alpha release of their system. The Google fanboyism seems to be running strong here.
eh? I don't see that as one little omission. It's kind of straightforward that you would need to be the owner of the actual URL to be deleted, so checking the siteUrl is not sufficient. I can understand if they only intended to allow relative paths from the siteUrl, and if there was some kind of bug in special characters or parameter parsing. But no, this is as bad as that bug in Citibank's site where you could just change the account number in the address bar [1]. Color me disillusioned with Google's security practices.
I think the point is that it's a basic dev and QA fail not to check for this, especially with people of the caliber that Google is supposed to recruit.
Process always falls down at some point, it's why we have bugs in the first place. The point of view that "obvious bugs should never happen" is pretty obviously broken, you just try to make them as rare as humanly possible. Besides, simple looking things from the outside can be maddeningly complex from the inside - for all we know this could be related to an obscure bug in their test framework marking it as passed when it isn't.
Most of the time when people say "what a stupid mistake" they mean "that's a mistake I haven't made yet"
...but Intel released the Pentium and it couldn't divide by 10.
Sometimes testers use the shotgun approach, and things get missed. It can help to write exhaustive tests (you have a computer, right?) and try everything. But the problem space has to be orderly, orthogonal, something that can be spanned. This bug is in a pretty small problem space - an API with many dimensions (arguments), external dependencies. I'm not at all surprised something got thru.
Quoting Spolky regarding bugs and bug fixes? He has a horse in the game, I wouldn't quote him for anything bug-related.
Regarding bugs: a fast thinker always thinks about the consequences of checking in any code. Checking in code, not to mention releasing it, changes the word (in a very minor way but it still does). Not exploring the possible consequences and the alternative options is just negligence and/or lack of experience. You don't have to be a perfectionist to see this, you just have to be fast: finding a wise, balanced solution quickly every time you change the world.
My guess would be that someone took an internal tool with few security requirements and rolled it straight to production. If that is the case, this is probably an excellent candidate for a security audit, there are almost certainly more issues if something this basic was not in place.
The huge printing presses of a major Chicago newspaper began malfunctioning on the Saturday before Christmas, putting all the revenue for advertising that was to appear in the Sunday paper in jeopardy. None of the technicians could track down the problem. Finally, a frantic call was made to the retired printer who had worked with these presses for over 40 years. “We’ll pay anything; just come in and fix them,” he was told.
When he arrived, he walked around for a few minutes, surveying the presses; then he approached one of the control panels and opened it. He removed a dime from his pocket, turned a screw 1/4 of a turn, and said, “The presses will now work correctly.” After being profusely thanked, he was told to submit a bill for his work.
The bill arrived a few days later, for $10,000.00! Not wanting to pay such a huge amount for so little work, the printer was told to please itemize his charges, with the hope that he would reduce the amount once he had to identify his services. The revised bill arrived: $1.00 for turning the screw; $9,999.00 for knowing which screw to turn.
- There are often-neglected big support advantages of buying from large suppliers with good support.
(ok, big corporations do take that into account, but for small startups, making sure that the buyer feels at ease not having such a case blow up in their faces is important, and same for buying mission-critical software, though having a disruptive/competitive-advantage generating app might be worth the risk)
Somewhat related: I wish GWT had a "pattern" removal.
With one of my sites, by the time I noticed that certain pages were missing the "noindex" tag Google happily indexed over 4000 pages. Considering the rate Google is crawling those pages it may take years to be removed from the index. Obviously, submitting each link one by one is rather tedious.
Hopefully the author is going to release that extension after Google fixes this bug. I may actually bother clicking 4K times just to see that site "fixed"...
Grrr, google and it's acronym's, I thought GWT stood for Google Web Toolkit and I was really confused for a second. Instead this GWT stands for Google Webmaster Tools...
"but I've never heard of a law protecting one's right to be listed in a search engine."
I believe that in most countries where computer crime law exists, removing or modifying data that is not yours(1) is covered by the law.
(1) What's "yours" and what's not is of course a very tricky question when it comes to immaterial things. Whether a listing for your webpage generated by a third party is yours, I wouldn't bet (on either side.)
In germany, you might have a stab at sueing for lost income, for example if you are a shop with a large number coming from google search, or if you get large amounts of ad revenue from visitors from google.
I am always amazed how experienced programmers can make such obvious errors when processing user input. Why would I ask for a URL of the WMT account in the query string?
I just hope that there is no "for the lulz" guy running a batch script to see how many million URLs he'll be able to remove before this gets fixed.
Personally, I would be more concerned if someone with malicious intent and the ability to keep silent about what they have done had found & exploited this. [0]
Advertise: remove your competitor from Google's search results for a day! If I didn't think it was illegal, I'd probably pay for that, were I in such a situation.
[0] If, of course, it even existed in the first place. It seems plausible enough to me, even if I think it unlikely.
4 months ago one of my sites totally disappeared from Google. I wonder if this is because of this??? It's not a shady site, and there's no reason Google would remove ALL the pages.. if anything they'd penalize it.
In the past, when I've had problems, I couldn't contact them either. They've done a great job at making sure there's no human contacts available. You have to post something in a public forum and hope they'll contact you. (They won't.)