I did not realize at first sight that this is an old article. It was standardized in the 90s and supported by most interfaces since the early 2000s but does not seem to be working properly as of today in terms of adoption. A big problem today regarding Ipv6 is the still missing auto configuration on most devices and router sided problems like don't sending out router advertisements with the prefix. Then there are other problems like no default gateway set. Technical skilled people are able to get Ipv6 running on their pc, but not the average guy.
Problems get much bigger on mobile which represents most of the worlds population digital presence. If everything properly configured Ipv6 works over ones own network normally as on a pc. But good luck with mobile data. Changing APN settings on android to Ipv6 does often not work / not supported by the ISP.
Almost half the world is using IPv6, according to Google's traffic statistics. In some countries it's 60%, 70%+ and I'm sure includes plenty of average guys.
This is mobile phones dominating that number. Almost every 4G / 5G phone right now uses a dual stack IPv4/6 connection. This is not necessarily problematic but home broadband is still lagging by a lot. I sometimes never get a prefix assigned. I have to redial PPPoE as IPv6 link uses IPv4 session. It's not plug and play for home broadband unless you use their modem router combo which has other issues.
Good point but there is still half of the worlds population missing which is due to missing auto configuration / proper support by the ISP. And I would guess that these numbers represent connections from home networks and not from people on the go using mobile data.
In my experience with consumer internet service and cellular service, ipv6 (when available) just works out of the box and you don't even have to notice.
>A big problem today regarding Ipv6 is the still missing auto configuration on most devices and router sided problems like don't sending out router advertisements with the prefix.
Huh, I haven't seen that issue with RAs. So far, my SLAAC deployment is playing nicely with my endpoints. About 40k devices, all BYOD. SLAAC seems like the only way to make this go. Because Android still has no intention of implementing DHCPv6. :(
> Please tell me what and where his added value is. And please no fear mongering about the rich getting poorer.
Daraprim is a very old drug used by about 2k people in the US! So the narratives by the media that gives one the feeling he kills millions is wrong.
97% in the US have health insurance + you get depending on the state your living in an extra charge by the IRS if you don't have one. Martin Shkreli gave the drug away for free if a person could proof that they could not afford it.
People are still dying because of daraprim every few years. But as I said it is a very old drug and has not improved since its inception in the 50s.
The price hike had not the goal that he and his investors could get rich quick (which would be there right as free human beings). The research of a drug and especially the clinical trails can cost several hundred millions of dollars, thus a price hike was necessary to discover a new and better form of daraprim.
There is a reason that rare disease drug sometimes cost a few 100k when only a handful of people need it.
Pharma is a long term game. He can sell daraprim for much more in the future than the $750 of the current version. In the long run he and his investors profit enormously but also the patients that need it since it is more safe.
When you develop a live saving drug, that only 2k people use, you have the right to charge whatever price you desire. People need to see that pharma is a business like any else and is not excluded from free-market and capitalistic principles.
Ìn addition, once you had your daraprim course you are fully healed - you "loose" the customer. Therefore a high price is needed even more to offset the cost and to profit.
Pharma companies put private resources into effort (capital, infrastructure, expertise etc.) for developing a rare disease drugs. Nobody has the right to demand anything from them nor condemn them.
When an entity buys a business (majority ownership) it can do whatever it pleases to do. Outcries and discussions about fees they charge and other things they allegedly do is completely illogical. This article is unbelievable hopeless.
People have to understand that a 'PE' fund and also other alternatives like a 'hedge fund' is simply a private partnership / specific legal vehicle. The term 'Private Equity' and 'Hedge fund' does not even exist in legal terms, it is market jargon. Not only will there never be any regulation but there can't be any. You can't just regulate a certain field of activity that operates solely in the legal world. The private partnership has an agreement to acquire companies, but it could also be about anything else! Giving the fund enough freedom to buy companies, buy land, buy art etc.
A PE fund and a hedge fund are NOT REGULATED by the SEC, they are REGISTERED with the SEC. Their filings are also never checked for correctness, thus only accredited investors (wealthy - e.g. income above 200K) are allowed under law to invest in such private entities if the fund markets itself!
You can not regulate a private investment vehicle in a country takes certain economic and free-market principles and incentives halfway serious. And if the public goes on the nerve of these management companies they just rebrand 'Private equity' to something else.
And don't forget, they OWN the portfolio company. There is no serious argument for regulation that prohibits a legit owner of a company to do a certain activity.
The need to adapt it for the average user is mentioned. The average person uses a pc (if ever - good luck with this on mobile) mostly for work (ms office + some ERP) and in some cases for private uses (e.g. news, e-banking, mails, important administrative work). If you go a bit deeper maybe reddit and video games. An average user would never want to link around stuff on the web with a hundred arrows and multiple colors. He simply does not care.
The author and the old guy in the video he linked to behave almost cult-like, especially the old guy: He literally claims that this IS the best method for working with documents ever, the www is a fork of his idea based on a "dumbed-down in the 70s at brown university", he does not understand why it has not already taken off and he thinks its the most important feature for the human race. Really?
If people really see potential in this and work in spaces like journalism and academic research there would be already big programs out there.
Yes it is good to have passion about something and yes it is good if someone has a real need for this and his delivered with a solution, but this will never go mainstream. And in my opinion not even in the segment of technical skilled people like engineers.
This is the typical invention that fits the "I know it is the best thing, I love it and almost pressure people to use it, but it has not taken off the lightest for decades" case.
The old guy in the video is Ted Nelson, the man who coined the term hypertext, made significant contributions to computer science, inspired two generations of researchers and continues to inspire as his works are being rediscovered.
There have been "big programs" but when the web came, fundamental hypertext research and development on other systems came to a grinding halt. Ted Nelson, and many other researchers, predicted many of the problems that we now face with the Web, notably broken links, copyright and payment as well as usability/user interface issues.
I don't know what an average user is, but what a user typically does or wants to do with a computer is somewhat (pre)determined by its design. Computer systems have, for better or worse, strong influence on what we consider as practical, what we think we need and even what we consider as possible. (Programming languages have a similar effect).
One of the key points of Ted Nelson's research is that much of the writing process is re-arranging, or recombining, individual pieces (text, images, ...) into a bigger whole. In some sense, hypertext provides support for fine-grained modularized writing. It provides mechanisms and structures for combination and recombination. But this requires a "common" hypertext structure that can be easily and conveniently viewed, manipulated and "shared" between applications. Because this form of editing is so fundamental, it should be part of an operating system and an easily accessible "affordance".
The Web is not designed for fine-grained editing and rearranging/recombining content and has started as a compromise to get work done at CERN. For example, following a link is very easy and almost instantaneous, but creating a link is a whole different story, let alone making a collection of related web pages tied to specific inquiries, or, even making a shorter version of a page with some details left out or augmented. Hypertext goes far deeper than this.
Although a bit dated, I recommend reading Ted Nelson's seminal ACM publication in which he touches many issues concerning writing, how we can manage different versions and combinations of a text body (or a series of documents), what the problems are and how they can be technically addressed.
> One of the key points of Ted Nelson's research is that much of the writing process is re-arranging, or recombining, individual pieces (text, images, ...) into a bigger whole. In some sense, hypertext provides support for fine-grained modularized writing. It provides mechanisms and structures for combination and recombination. But this requires a "common" hypertext structure that can be easily and conveniently viewed, manipulated and "shared" between applications. Because this form of editing is so fundamental, it should be part of an operating system and an easily accessible "affordance".
Here's where I'm stuck:
Hypertext - whether on the web or just on a local machine - can't solve the UX problem of this on its own, though. People can re-arrange contents in a hypertext doc, recombine pieces of it... but mostly through the same cut-and-paste way they'd do it in Microsoft Word 95.
The web adds an abstraction of "cut and paste just the link or tag that points to an external resource to embed instead making a fresh copy of the whole thing" but all that does is add in those new problems of stale links, etc.
So compared to a single-player Word doc, or even a "always copy by value" shared-Google-doc world that reduces the problems of dead external embeds, what does hypertext give me as a way of making rearranging things easier? Collapsible tags? But in a GUI editor the ability to select and move individual nodes can be implemented regardless of the backend file format anyway.
TLDR: I haven't seen an compelling-to-me-in-2023 demo of how this system should work, doing things that Google docs today can't that avoids link-rot problems and such, to think that the issue is on the document format instead of user tools interface side.
I have to catch some sleep, but I will address your questions as good as I can later. In the meanwhile, you might want to take a look at how Xanadu addresses the problems of stale links, and maybe some of your other questions will be answered.
Also, I highly recommend reading Nelson's 1965 ACM paper I mentioned to better understand the problems hypertext tries to solve and the limitations of classical word processing (which also expands to Google Docs).
Thanks, though I think even in these docs there are some early on concepts I just don't find convincing. Such as in the Xanadu doc, the Software Philosophy short version being a recomposed copy of live text from the long version. If I'm following their goals, they want live cross-editing through their "transpointing windows" - I really really don't, personally. I picture three docs, A, B, and C which is a summary composite of things pulled from A & B - C will still need its own manual curation and updating if A and B are changed to remain legible, flowing, and meaningful, and I'd rather have a stale doc than a garbled one.
The intro of the Nelson paper/Memex discussion is similarly alien to me. I don't think it's human-shaped, at least not for me. The upkeep to use it properly seems like more work than I would get back in value out. It's a little too artifact-focused and not process/behavior focused, IMO?
>I picture three docs, A, B, and C which is a summary composite of things pulled from A & B - C will still need its own manual curation and updating if A and B are changed to remain legible, flowing, and meaningful, and I'd rather have a stale doc than a garbled one.
I think I see what you mean. Garbling, as you mention it, is actually what Xanadu is supposed to prevent. The problem is that it is not explicitely mentioned that a document/version (a collection of pointers to immutable content) should also be an addressable entity (an part of the "grand address space") and must not change, once it has been published to the database.
In particular, if a link, e.g., a comment, is made to a text appearing in a document/version, the link must, in addition to the content addresses, also contain the address of that document/version (In Fig. 13. [1] that is clearly not the case and I think that's a serious flaw).
This way, everything that document C refers to - and is presented - is at the time it was when the composition was made. How revisions are managed is an orthogonal (and important) problem, but with the scheme above we lose no information about the provenance of a composition and can use that information for more diligent curation.
I understand. I think your last objection is very valid and perhaps needs far more consideration.
Anyway, I have limited computer access at the moment, but maybe you find the following response I wrote in the meanwhile useful. Ill get back to you.
---
Some remarks that hopefully answer your question:
The Memex was specifically designed as a supplement to memory. As Bush explains
in lengthy detail, it is designed to support associative thinking. I think it's
best to compare the Memex not to a writing device, but more to a loom. A user would use a
Memex to weave connections between records, recall findings by interactively following
trails and present them to others. Others understand our conclusions by retracing the
our thought patterns. In some sense, the Memex is a bit like a memory pantograph.
Mathematically, what Bush did is to construct a homomorphism to our brain.
I think it is important to realize that, when we construct machines like the Memex
or try to understand many research efforts behind hypertext. Somewhere in Linda Barnett's
historical review, she mentions that hypertext is an attempt to capture the structure of thought.
What differentiates most word processing from a hypertext processor are the underlying
structures and operations and ultimately, how they are reflected by the user-interface.
The user interfaces and experiences may of course vary greatly in detail, but by
large the augmented (and missing!) core capabilities will be the same. For example,
one can use a word processor to emulate hypertext activities via cut and paste, but
the supporting structure for "loom-like" workflows are missing (meaning bad user-experience),
and there will be a loss of information, because there are no connections, no
explicitely recorded structure, that a user can interactively and instantly retrace (speed and effort matter greatly here!),
since everything has been collapsed to a single body of text. The same goes for annotations,
or side trails that have no structure to be hanged onto and have to be put into margins,
footnotes or various other implicit structural encodings.
Hypertext links, at least how Ted Nelson conceptualizes them, are applicative. They are not
embedded markup elements like in HTML. In Xanadu, a document (also called version) is
a collection of pointers and nothing more. The actual content (text, images) and the documents,
i.e., the pointer collections, are stored in a database, called the docuverse. Each content
is atomic and has a globally unique address. The solution to broken links is a bit radical: Nothing may
be deleted from the global database.
In other Hypertext systems, such as Hyper-G (later called Hyperwave), content may be deleted and link integrity is ensured by a centralized link server.
(If I am not mistaken, the centralized nature of Hyper-G and the resulting scalability problem was its downfall when the Web came).
Today, we have plenty of reliable database technologies and the tools for building reliable
distributed systems are much, much better, so I think that a distributed, scalable version of
Ted Nelson's docuverse scheme can be done, if there is enough interest.
How a document is composed and presented, is entirely up to the viewer. A document is only
a collection of pointers and does not contain any instructions for layouting or presenting,
though one can address this problem by linking to a layouting document. However,
the important point is that processing of documents should be uniform. File formats such as PDF (and HTML!) are the exact opposite of this approach.
I don't think that different formats for multimedia content can be entirely avoided, but when
it comes to the composition of content there should be only one "format" (or at least
very very simple ones).
It really depends on the type of financial institution that has given the loan. A bank for this type of transaction is very unlikely nowadays, so I guess it is a private credit fund. Given that they have probably applied some sort of traditional Loan-to-value ratio, it's very likely that the $2.3 billion is only a portion of the total value. Meaning the cloud provider has received e.g. 50 or 80% of the total value of the H100's as a loan. It surely is not very much since electronics have a high depreciation rate from an accounting perspective (3-5 years max).
You have just highlighted the main fact about the www that so many technical skilled people are missing to see. Most consumers have little to no knowledge about anything technical, which is not bad at all since there is no reason why they should care. Now imagine telling those people that everyone should have their own website instead of a social media account. What a horrible and illogical thought in my opinion. Digital consumers only change behavior if something is 10x better. So from a consumer perspective, who cares about things like decentralized networks? Or Duckduckgo.com for example is probably for 99% of the www users some chinese russian inbreed virus. People started using ChatGPT besides google because it is simply 10x better for many cases, and not because it promoted with more privacy feature and less ads than google.
It's unnecessary indeed to not use Ipv6 addresses, 2^128 addresses and the many many features it offers like unicast etc. Ipv6 makes a server as a middlemen for some applications (Ipv4 only) completely obsolete.
But a big problem is that there is still no Ipv6 auto configuration at all on a lot of devices (e.g. no default gateway or no global address configured). Especially android devices and from experience also on Windows. Linux depends on the distro. Changing routing settings on android devices from Ipv4 to Ipv6 does often not work or is not offered by the ISP strangely.
And there are other problems like routers having enabled incoming and outgoing Ipv6 connections by default, which is good, but having router advertisements blocked by default, which is bad. Since there is no way for the OS to get the prefix to construct global addresses automatically. Most users today have little to no knowledge about networking and computers in general. So auto configuration is a must.
That leads to Ipv6 only servers being not reachable and thus the buying of Ipv4 addresses makes a lot of sense at this point.
Building IPv6 autoconf into the protocol was a mistake. DHCPv6 is better.
The problem is that when you autoconf on a local network you usually want more than just a route and basic DNS. Trying to do it in the IP protocol is a bad idea since the IP protocol is intended to almost never change. It belongs in a protocol that's less tightly bound to the IP stack that can be more easily extended like DHCP.
DHCP can also integrate with things like local DNS, while this is much harder to do with IPv6 RA and SLAAC.
SLACC is something that sounded good on paper but doesn't adequately capture the entire problem domain.
IPv6 in general needs to just deprecate all the parts of the protocol that are anything but just making IP addresses larger. Everything else is "second system effect" cruft that tends to impede adoption by adding complexity or adding features in the wrong place in the stack.
For clients, IMHO, SLAAC is fine and means I don't have to maintain and run a DHCP service anymore. One less thing that can fail, while SLAAC only fails me if the routers IPv6 link inside the given network goes down.
Servers on the other hand, I will provision with a static IP subnet on deploy, as part of the PXE install or configuration management process, depending on the environment. They will have an ephemeral address during the install, but then query for and persist their allotted address before rebooting into the installed environment as part of their post-install.
I guess we agree that we need a single source of truth, what physical device has what IP (range) in their possession at any time. DNS is a classic way to do that, but there are other solutions, from ITIL-style CMDBs to simple config management git repos. And of course the latter doesn't mean that we don't also update DNS based on IP-assignement, DHCP is not the only tool that can be made to interface with a DNS service.
> SLACC is something that sounded good on paper but doesn't adequately capture the entire problem domain.
Good news! Nothing in SLACC (sic) prevents you from using DHCPv6.
But now, since we have SLAAC as well, you get auto-magic working with simple link-layer connectivity without having to bother with extra infrastructure. If you need extra functionality, you have the option (not necessity).
Standard SLAAC only uses 64-bit prefixes. This is a problem if you want subnets and have an uncooperative ISP handing out 64-bit prefixes. Or maybe you have a /48 or /56 but want a nice readable hierarchy of subnets.
You don't even need 64 bits. IIRC with SLAAC you send NDP messages to the entire broadcast domain. So you need broadcast-domain-sized (i.e., small) device counts on every data-link-layer subnet. So why assign /64? But because some devices won't give up if SLAAC routers aren't there and and some mysteriously default to SLAAC when they shouldn't and in fact some devices only use it, now every bottom level subnet has to be /64. Blech.
This is just one of many IPv6 annoyances that don't even need to exist.
/64 is about a lot more than SLAAC, that's just one easily seen place it shows up. Even without SLAAC the /64iveness of IPv6 would still be there. End client subnets being /64 (with the exception of host routes and p2p networks) allow hardware route scaling to be roughly twice as high, allow network operators to never worry about what the client subnet size is (it's /64 and that's big enough for anything, if it's bigger it's an aggregate route, the address is always split in the half for routed part and client part), and allow various tricks like SLAAC assignment methods to work reliably. A /56 gives 256 subnets (and a /48 65,536), if a house needs more than that for hierarchy then something besides the base /64 is wonky.
Uncooperative ISPs will always exist. If the minimum size was /96 they'd hand that out instead of a /64. If there was no minimum size then they'd hand out a /124 or something equally as stupid. For all of the corporatize ISPs and well planned uses, starting at a /64 makes the rest of life easier.
It's the same logic, though often more accepted in this case, as the minimum advertiseable prefix on the internet being a /48. That also has nothing to do with how many theoretical end clients could fit in that and everything to do with what that actually means to equipment, scaling, and implementation planning.
The practical issues are unrelated to whether or not the two are mutually exclusive. In practice, the lack of RDNSS support from the get-go was quite the nuisance. Especially when combined with clients that refused to support the optional DHCPv6 method (e.g. Android/ChromeOS). Even with RDNSS, the feature set is quite limited compared to DHCPv6 but you may have to end up implementing both since there weren't enough incentives for all clients to go the extra mile and support it even if some of your other clients do.
Overall these days, beyond Google products, I think the ecosystem is in a healthy spot now and I think IPv6 chose the right path. It can still be a bit more hassle to manage in the enterprise space though and that's the one that needs to most convincing to migrate at this point.
As long as ISPs are unwilling to actually work on the problem on letting their customers use ipv6, applications/services will continue to be uninterested in exposing ipv6 for usage.
The worst foot-draggers are major sites like Github and cloud infrastructure. Google only got IPv6 in GKE this year in most regions.
The other big foot-draggers are corporate networks. Even if the ISP supports V6 many corporate networks do not because two generations of IT professionals learned how to do networking entirely through the lens of NAT as a requirement and don't understand how to do things without it. I've seen many IT peoples' brains just melt at the idea of things just having one address. In reality it simplifies things dramatically but sometimes getting people to grasp a simpler solution is actually harder than getting them to grasp a complex one.
I live in the USA and have had IPv6 at home for over a decade (and have used three different ISPs in that time). Many mobile networks are IPv6-first.
That being said, you can do NAT on IPv6 if you really want to, and maybe it will be needed to help soothe those with those emotional attachment to certain numbers. [fc00::192:168:1:0]/120 or [fc00::10:44:0:0]/96, for example.
Even with NAT IPv6 is better because the NAT can do 1:1 mapping for every IP inside and does not need to remap ports. There is no port exhaustion and P2P always works.
V6 NAT is unnecessary and dumb but it works better than V4 NAT.
Nowadays, you shouldn't be allowed to advertise "internet access" if ipv6 isn't supported.
Ipv6 is the current protocol. And some sites don't have ipv4.
(Amazon charging an extra for ipv4 is another sign that ipv4 should be a protocol for particular use cases, not for "the internet")
And it should be the same for software and connected hardware.
No ipv6 ? That's not a product that works over the internet.
On a personal side, what I host is only working on ipv6, as my ISP has stable ipv6 but not ipv4, and for the convenience of configuration.
And even cheapo internet plans on mobile and landline support ipv6 by default nowadays. (The government pushed for it)
> On a personal side, what I host is only working on ipv6, as my ISP has stable ipv6 but not ipv4, and for the convenience of configuration.
For me it's the other way around - I disable IPv6 on all my servers and only host anything on IPv4. I know it's frowned upon in networking circles, but IPv4 "just works" for me, and I want to reduce attack scope and maintenance burden (I had some problems with IPv6 messing things up, or my ipv6 firewall misconfigurations).
Exacly that! I do the same. Accessing of internal resources hosted on LAN?
No problem, just make overlay VPN network.
Need to host something via HTTP? mod_proxy to the rescue.
IPv6 is junk protocol, overengineered. I hope IPv6 will be used for all those internet consumers and IPv4 will stay where its place to be, interesting R&D projects :)
All major ISPs have had native ipv6 for customers in the US for at least 5 years. Not some funky bastardized implementation but native full ipv6.
Ipv6 is overly complicated and has been riddled with bugs for 30 years now. As long as ipv4 is an option many are going to choose to completely disable it. Some of the security concerns cannot be effectively filtered at all. There are numerous examples of these vulnerabilities from even just the last few years.
It’s hard for teams of engineers to secure properly much less a home user.
I completely disable ipv6 even with a deep understanding of it.
> All major ISPs have had native ipv6 for customers in the US for at least 5 years. Not some funky bastardized implementation but native full ipv6.
CenturyLink (or Lumen or whatever they want to call themselves today) only has 6rd, at least in my neck of the PNW. And it's best if you don't use it, as their CPE tends to do bad things if you do; my initial CPE would reboot if a fragmented 6rd packet came in over the WAN interface. The current CPE doesn't reboot, but v6 packets sometimes take about 1 second to transit the CPE, so I gave in and run the CPE as a bridge and do PPPoE on my own equipment.
Here in Boise I have two choices for internet, Sparklight or CenturyLink. CenturyLink will only deliver 80Mbps to my house and Sparklight does not support IPv6.
Absolutely, a very good point indeed. But I meant more specific applications like end-to-end messaging. Surely 'obsolete' is a bit of an overstatement. At the end it depends how one looks at and want things to be done.
How can a bot create fake impressions? When a bot (or just a simple program) makes a http request he fetches the raw html code only. AFAIK if you don't actually render the html code in a browser or requesting all the contents afterwards again with http requests (like GET ad.jpg, GET logo.png etc.), no google ad server should be hit. Now you could argue that bots could inflate the popularity of a website and therefore the cost to run ads on it. But I guess websites that show ads have most likely google analytics running, one of the only ways Google can actually calculate the popularity (besides Google Search and maybe Chrome history). So it should be no problem for Google to exclude bots from the popularity calculation by analyzing traffic. Maybe I am just missing something, I am also no ad expert at all.
It's not about bots creating fake ad impressions by accident. It's people writing bots whose purpose is to fake ad impressions and clicks. They'll then run it on their own website that's running ads, with the goal of being paid by the ad network for this fake traffic.
But isn't this a win situation for Google to a certain extent? Since it uses up the budget of the advertiser much faster. And the accuracy of filtering new revenue coming from ads as a company is already fairly limited in general. But maybe there are multiple reasons that Google really only wants to serve real humans to the ads of its clients.
It's not a win. The fake clicks will not convert to sales, and the advertisers are seeing a lower ROI on their ads and will go and spend their budget elsewhere in the future. All ad networks will try to filter out as many fraudulent clicks as possible, because they are not optimizing for the maximum revenue today but for the revenue in the long run.
But yes, of course this is not just about filtering out fake clicks. The draft proposal lists a bunch of use cases, most of which have nothing to do with ads.
Interesting explanation, I totally agree on the click-per-pay part. But how would you track the benefit of ads with paying-per-impression? I know its less expensive, but according to the article paying per view seems to be a quite big part of the ad business.
The article is just straight out wrong about "Google’s ad network charges per impression". The author clearly doesn't know anything about the area, made up some shit on how things could work, and just wrote it into their article with no fact checking.
You're right that attribution and measuring ROI is way harder and less precise for ads sold by impression than by click. That's why they're not the common form of advertising, especially on these kinds of ad networks. But for cases where the ads are per impression, the concerns about fraud would be exactly the same. It's not about a crawler accidentally generating impressions, it's about bots deliberately doing so.
Don't nail me down on this but I think since nowadays' websites are often dynamic, you most likely have to employ headless browsers in order to do whatever it is you want to do. This should then result in fake impressions.
That is the reason why most big quant hedge funds look for ghost patterns. Patterns that are not obviously at the first sight and which can be (hopefully) solely mined until the pattern disappears.
To catch onto obvious patterns you have to be not really fast but nano-microseconds fast, otherwise in a blink of an eye the price is already too high to jump in.
I imagine that if one has enough capital to fully arbitrage an opportunity, it is possible to
A) capture the entire available upside
B) estimate how large that opportunity might be, and not attempt to over-stoke it.
C) by capturing the entire opportunity, prevent the signal from being visible to the broader market. The only sign that a need is being met could be trade-volume rather than price movement.
That's why most quant hedge funds only have limited amount of aum. Most their strategies consist of a large amount of low volume trades. Low volume to prevent the ghost pattern being visible to others. This is especially true for non hft firms that hold positions in a much longer interval (minutes - hours - days). But this is untrue of course for lets say a big macro hedge fund that throws a hundred million dollars into a currency trade.
I totally agree on all three points and I even think that a certain amount of capital, opportunity related (not fees etc.), is not even needed to mine a pattern. Because according to the ones statistics and research the actually existing pattern should appear anyway.
Problems get much bigger on mobile which represents most of the worlds population digital presence. If everything properly configured Ipv6 works over ones own network normally as on a pc. But good luck with mobile data. Changing APN settings on android to Ipv6 does often not work / not supported by the ISP.