This is relatively easy to block, given that these companies publish their IP ranges. Similar to the Tor block, blocking "datacenter IP's" simply becomes the norm. This is why you have companies offering "resi's" or residential proxies to bypass such blocks for some years now.
There have been a number of Chrome extensions which monetize by selling your bandwidth to provide this kind of service. Not ethical but probably not a compromise.
This is not alledgedly but very much true. I was in the proxy business for a few years and in talks with the Luminati people to white-label their product for a very specific type of proxy.
It’s such a weird field to be in. It’s not illegal by definition of law, but you’re definitely in shady territory, with most of the customers being of the “get rich quick” persuasion. Or at the very least trying to cut corners. One way or another, they were not playing by the rules ;-)
A significant portion are grey market "ISPs" that purport to sell residential services but actually never do.
They sell these "residential" IPs to Amazon, other ecommerce retailers and shady people for an extreme price.
In the e-commerce world, scraping is necessary to stay in business. Amazon has armies of scrapers constantly monitoring their competitors, and in some cases automatically undercutting price updates.
Same on the consumer side. I like setting up alerts on camelcamelcamel and seeing price history (to make sure I'm not getting screwed) and to buy buy buy when prices hit a certain theshold.
I've heard of services that offer free proxy bandwidth in exchange for the user acting as a residential proxy node for other users and/or paying clients. It's generally marketed as a way for the user to avoid geoblocks and such. If this is clearly stated to the user upfront rather than buried behind half a dozen dark patterns and fine print, it seems like this business model could be conducted ethically. When it comes to whether the companies in this space are currently acting ethically though, I have serious doubts.
You have autobuying bots (sneakers or GPUs or other limited edition stuff) that need to do tricks like this. I wouldn't necessarily call them unethical.
In my opinion it's unethical if the website explicitly states in their terms of service that bot purchasing is not allowed. As far as I know, that is pretty common for websites that sell frequently scalped products, and near-ubiquitous among those that implement technical countermeasures.
I'd expect cybercriminals and fraudsters also find a pool of disposable residential IPs to be very useful.
A Terms of Service is not a proxy for morals though. Companies will put whatever is convenient for them in there, I'm sure you can come up with several examples that you wouldn't consider ethical.
I worked on a fairly large web-scraping project (around 2 million pages per day) and we used luminati. Amongst other things, they offer genuine residential proxies with user consent.
Reading that page, the “user consent” is dependent on third parties who are monetizing their app through this service to inform their users. I … um … doubt the third party app developers give a crap to accurately describe the traffic that will subsequently emanate from their users’ devices.
Just like everything else in this industry, the retort will be “but it’s the users fault! They didn’t scroll through the 2,375 page privacy policy, user agreement, hold harmless indemnification agreement, and terms of use when they agreed to an ad free experience for their mahjong app! What a stupid user! Ha!”
To that I say, enjoy the coming oppressive regulations. It’s already started. As a kid, I always wondered why we required stupid laws to regulate common sense. Now I know why.
We don't need oppressive regulations; we simply need courts to adopt a sane definition of "agree".
If you're foisting an adhesion contract on more than 1,000 people, they are not deemed to have "agreed" unless a majority of a random sampling (say, ten) of them actually read and understood the entire document. Otherwise it's void. "Read and understood" is decided by a jury as part of any litigation involving the contract. "Random sampling" is made by court evidentiary procedures.
If the contract is negotiated or it was presented to less than 1,000 people the rules stay the way they currently are, since those are the kinds of contracts that English common law was developed for.
luminati runs on holavpn which installs backdoor proxies on people's machines. that "user consent" is not actually known to the users but is buried in some terms & conditions when they're naively installing the "free VPN" software.
