It can be rather difficult, mostly due to the occlusion calculations having to be conservative (must count visible things as visible, allowed to count invisible as visible, or things pop) and latency (must account for every possible position within max move speed * max latency, or things pop)
The naive raycast from player camera to other player would be fine for perf but may count partially visible as invisible, so its unacceptable. You'd have to raycast every pixel of the potentially visible player model to stay conservative. With movement + latency this expands to every pixel the player model could potentially occupy during your max latency period, and you need to consider the viewer moving too!
In practice this expands to a visibility test between two spheres with radius max_latency*max_movespeed + player_model_radius. Now, you could theoretically do a bunch of random raycasts between the spheres and get an answer that is right some of the time, but it would be a serious violation of our conservativeness criteria and the performance would get worse with more rays/better results. Also keep in mind that we need to do this for every single player/player pair a few dozen times per second, so it needs to be fast!
To do this, you need a dedicated data structure that maps volumes to other volumes visible from said volume. There are a few, and they are all non-trivial and/or slow to build well. (google for eg. potentially visible set, cell-portal graph + occlusion). You also trade performance for precision, and in practice you walls might become 'transparent' a bit too early. With all this being done, we can actually "do occlusion calculations server-side".
There's just one problem with this that I still don't know a solution for, namely precision. With fast players and imprecise conservative visibility, things you care about are going to count as visible pretty often, including stuff like enemies peeking from behind a corner (because they could have moved at full sprint for 100ms and the end of the wall is rounded away in your acceleration structure anyway) so all this complexity might not get you that much, particularly if your game is fast paced. You'd prevent some wallhacks but not the ones that really matter.
TLDR yes, it's actually hard and might not be good enough anyway
Just write a 10 TB hard disk full of plaintext mantras and let that bad boy spin at its usual ~5k rpm for a cost effective 50 PB of mantras per minute. Or go MAAS and write a few them into S3.
I'd say `x * a + (x * x) * b + (x * x * x) * c` is likely faster (subject to the compiler being reasonable) than `((((x * c) * x + b) * x) + a) * x` because it has a shorter longest instruction dependency chain. Add/Mul have higher throughput than latency, so the latency chain dominates performance and a few extra instructions will just get hidden away by instruction level parallelism.
Also x/6 vs x(1/6) is not as bad as it used to be, fdiv keeps getting faster. On my zen2 its 10 cycles latency and 0.33/cycle throughput for (vector) div, and 3 latency and 2/cycle throughput for (vector) add. So about 1/3 speed, worse if you have a lot of divs and the pipeline fills up. Going back to Pentium the difference is ~10x and you don't get to hide it with instruction parallelism.
* The first expression has a chain of 4 instructions that cannot be started before the last one finished `(((x * x) * x) * c) + the rest` vs the entire expression being a such a chain in the second version. Using fma instructions changes this a bit, making all the adds in both expressions 'free' but this changes precision and needs -ffast-math or such, which I agree is dangerous and generally ill advised.
My intuition is that with very small microbatch sizes you're very likely to end up in one of the two modes: either the vast majority of the samples are aligned and thus pruned away, or they are not aligned. Thus effectively you're dropping a fraction of the samples but without the advantage of removing the variance between samples that belong in different microbatches.
Yes. It’s more of a class spanning thing. I wanted batch composition across the two microbatches to be the same. So if you have class 1,2,3 in batch one and class 4,5,6 in class two I would fully expect the cosine distance to be orthogonal or worse, and it could be a good update. But if you have class 1,2,3 in batch one and class 1,2,3 in class two I would fully expect the cosine distance to be positively correlated and if not you should skip. So you could bring this to MB of size 5 for example but just make sure you have the same batch composition. This poses a big challenge in LLM training honestly bc technically classes is vocab size. So I need one “a”, one “b”, etc which is silly. This is why micro gradients in LLMs hit cosine distance of 2. So when you are sampling you kind of need to ensure the microbatches are of the same task at least.
Baseline time to grok something looks to be around 1000x normal training time so make that $20k per attempt. Probably takes a while too. Their headline number (50x faster than baseline, $400) looks pretty doable if you can make grokking happen reliably at that speed.
For a bit of reference, a full frame of Crysis (benchmark scene) was around 300k vertices or triangles (memory is fuzzy), so 3-10 log piles depending on which way my memory is off and how bad the vertex/triangle ratio is in each.
Author here: I never bothered counting the total vertices used per frame because I couldn't figure out an easy way to do it in Renderdoc. However someone on Reddit measured the total vertex count with ReShade and it can apparently reach hundreds of millions and up to 1 billion vertices in closeups in large cities.
Edit: Checked the vert & poly counts with Renderdoc. The example scene in the article processes 121 million vertices and over 40 million triangles.
“If you’ve used more triangles than there are pixels on the screen, you’ve messed up.”
I call this kind of thing the “kilobyte rule” because a similar common problem is shoving tens of megabytes of JSON down the wire to display about a kilobyte of text in a browser.
It’s actually very nearly optimal: using as many triangles as needed to display as much detail as possible while staying under the one triangle per pixel limit.
That’s the opposite of using 10K pixels to render two pixels on the screen.
> Mesh simplification for Unity. The project is deeply based on the Fast Quadric Mesh Simplification algorithm, but rewritten entirely in C# and released under the MIT license.
> "UE4 has had automatic LOD generation since it first released - I was honestly baffled when I realized that Unity was missing what I had assumed to be a basic feature.*
> Note that Nanite (which I assume you're referring to) is not a LOD system, despite being similar in the basic goal of not rendering as many polygons for distant objects.
Sounds right. I remember seeing "1M Triangles" in the performance HUD and thinking, that's crazy, a million triangles. Probably very few shared vertices once you account for edge splits, billboards, etc.
The framing here is a little unfair, its DLSS that does not support AMD hardware, not the other way around. This is clearly a deliberate decision by Nvidia. Meanwhile AMD/FSR supports both AMD and Nvidia hardware.
DLSS directly leverages the tensor cores of particular NVidia cards.
Whether or not this is just a marketing gimmick is up to you and your own conclusions on the matter, but AMD doesn't seem to have an equivalent hardware tech for this yet. It seems to be taking steps in the right direction with the WMMA instruction for accelerating matrix multiplication in a similar fashion, but performance so far head-to-head seems lacking, and I haven't seen anything definitive out of AMD suggesting this for gaming purposes.
The framing is fair unless you expect NVIDIA to ditch their ML based solution for a class of inferior implementations which in turn can run just about anywhere.
DLSS isn't "NVIDIA flavored FSR", it's a more advanced pipeline that requires significantly more architectural alignment. FSR 3 was going to be the closest thing to an equivalent and won't run everywhere FSR 2 does.
Sounds like using the wrong metric, the upscaled image should be compared against the original full resolution one, not the downscaled one. Obviously you can't know what the full resolution one looks like when actually upscaling (vs testing), but you can make an educated guess.
You can remove the guesswork. You can start with a high resolution (or even raster export of a vector image for an extreme example), downsample it (various methods and downsampling factors for completeness), then attempt to upscale it.
The thing is, most styles don't have names, only artists using them. If you remove artist names from the training data, you lose the ability to describe those styles, including as combinations (so you can't even do "influenced by rutkowski" styles). This obviously sucks for the user, and I don't think artists should be able to deny control over style just because it makes imitation easy.
In this context, putting an artist back in the model is a pretty sensible form of disagreement/protest. I admit the "I'll take this down if they ask me to" is strange, but mostly because its too weak; the LoRA should just stay up, not demand another no from someone who already made their opinion clear.
> If you remove artist names from the training data, you lose the ability to describe those styles
It's almost as if artists develop their own styles that are just as uniquely theirs as any one work produced by them in that style, if not more so! When people put "by Greg Rutkowski" in their AI art prompts, they are not asking for results in a style that exists independently of Greg Rutkowski despite his name somehow being the only way to identify it. They are, in fact, asking for results in the style of Greg Rutkowski in the sense that works by Greg Rutkowski were labeled as such and fed into the training process, programmatically creating a digest of works by (at least) Greg Rutkowski that we call a "model", and that model is as a result capable of mimicking the style of Greg Rutkowski specifically. They are asking for results that look like they were painted by Greg Rutkowski, and they care enough to ask for Greg Rutkowski specifically because Greg Rutkowski is a highly skilled artist who developed his own distinctive style which is the central pillar of his intellectual property as an artist, and his competitive advantage.
Intellectual property law as it stands today obviously couldn't have anticipated this technology, and didn't. However, if this kind of thing is still legal in ten years, that will represent a major systemic failure of copyright and trade secrets law to protect the core intellectual property of artists in the same spirit as it currently protects intellectual property in other domains.
That list is only really useful for major styles that are well known/tagged, early SD versions knew a list of artists _much_ longer and more relevant than that. (some of the styles on that list are not related to 2d images). If you consider the list a reasonable substitute for artist names, I'd ask you for a prompt that replicates the common Rutkowski + Mucha style using just the styles listed.
>> It's almost as if artists develop their own styles that are just as uniquely theirs
Hard disagree on this. Styles do not and should not have ownership, legally or otherwise. Not in painting, music, film, games, or any other kind of art. Anything else kneecaps derivative styles and erects a massive barrier to the advancement of culture. Straight copies are mostly avoided because they add nothing over the original.
You're also misunderstanding the typical use of Rutkowski and other names. Most prompts I've seen and used use multiple artists for a unique blend of styles (including the much maligned and overused Rutkowski + Mucha), not just straight copies. While asking for Rutkowski indeed asks for a very specific style (as well as any themes etc common to his work), its usually in combination with other equally specific and somewhat contradictory styles, each guiding the result in some direction and adding elements but none of them being exactly the desired end result. Just asking for straight up Rutkowski was a short lived cargo cult and is now more of a meme than anything else.
> If you consider the list a reasonable substitute for artist names, I’d ask you for a prompt that replicates the common Rutkowski + Mucha style using just the styles listed.
Not looking at the list, but replacing “by Greg Rutkowski” with “detailed realistic muted fantasy” and “by Alphonse Mucha” with “Art Nouveau” seems to work pretty well for reproducing both “by Greg Rutkowski” alone and for the Rutkowski + Mucha combination. (Oddly, replacing either of those, but not the other, does not work as well for reproducing the combination effect as replacing both.)
> Just asking for straight up Rutkowski was a short lived cargo cult and is now more of a meme than anything else.
Rutkowski + Mucha is as much of a cargo cult as Rutkowski alone, even if it is a longer-lived one.
> If you consider the list a reasonable substitute for artist names
I didn't say or suggest any such thing.
> I'd ask you for a prompt that replicates the common Rutkowski + Mucha style using just the styles listed.
You're completely missing the point, which is that if you want the model to (e.g.) combine the individual styles of two specific artists in its output then you are not just looking for a style, you are looking for mimicries of those specific artists combined and normalized by the model. The model is able to do this because works by these artists were annotated as such and fed verbatim into the training process. Of course it can't mimic a specific artist unless you mention them by name!
> Hard disagree on this. Styles do not and should not have ownership, legally or otherwise. Not in painting, music, film, games, or any other kind of art. Anything else kneecaps derivative styles and erects a massive barrier to the advancement of culture. Straight copies are mostly avoided because they add nothing over the original.
You (the general you, an amalgam of the AI art defenders I've seen) claim the right to create these unauthorized derivative works via an automated process with zero reliance on novel human authorship beyond a short text string, then work backwards from there to rationalize to yourself why what you're doing is okay. Then, when artists who've invested decades of their lives into the development of their technique and process and relationship with art tell you that your rationalization is a facile one and that you are stealing from them, you are so sure you know better than them that you sweep aside their concerns and accuse them of jealousy.
In context, it's an absurd and frankly disgusting level of self-absorption. It's like the way crackpots relate to real working scientists, angrily casting the lack of attention paid to them and their "theories" as jealous conspiracy and gatekeeping rather than considering that maybe they simply have no idea what they're talking about, because why would they? But in this case the big money is unfortunately on the side of the crackpots.
I don't care about the "meta", and I don't care about "memes" or any other specifics of the subculture of AI art tool users. What I care about is that these tools obviously launder the intellectual property of artists into ostensibly non-infringing derivative works. I think the comparison with how human artists develop and apply technique and style is intellectually fascinating and worthy of discussion^[1], but from a legal and moral perspective comparing the two is like comparing a detective listening to a wiretap and taking notes by hand with the NSA's PRISM program or similar. They are superficially the same from a certain narrow perspective (and I believe there has been no end of facile rationalizations deployed to this effect), but broadly they are substantively different, morally and practically and especially in their implications for society. Everybody knows that justifying the latter on the basis of the former is foot-in-the-door bullshit that flies in the face of the spirit of the law.
Where will the law go? Who knows! All I know for sure is which side has the money.
^[1] I think if you go back in my commenting history you'll see me talking about that at length, and even taking what may appear to be the other side of this debate.
> You (the general you, an amalgam of the AI art defenders I’ve seen) claim the right to create these unauthorized derivative works via an automated process with zero reliance on novel human authorship
If they are created “via an automated process with zero reliance on novel human authorship” they, ipso facto, are not “derivative works”, which are original creations of human authorship subject to separate copyright (which, nonetheless, require permission from the copyright holder of the original work to legally create unless an exception to copyright protection, like fair use, applies.)
What you probably want to argue is that they are unauthorized mechanical copies, not unauthorized derivative works.
Of course, if they are unauthorized mechanical copies, it is probably usually a partial copy of the training set, which if it has a copyright it is a compilation copyright owned by whoever assembled the training set, not of any particular work within the training set. Now, whether the training set is an unauthorized derivative work when compiled is another question.
I don't really want to argue either because I'm not a lawyer and from a lay perspective AI works don't seem to fit neatly into either category. It's obvious to me that infringement occurs in spirit, which is effectively what i'm arguing; beyond that I really couldn't speculate on the details of how a successful case to that effect would be built, or how new law would be written.
> They are asking for results that look like they were painted by Greg Rutkowski, and they care enough to ask for Greg Rutkowski specifically because Greg Rutkowski is a highly skilled artist who developed his own distinctive style
Sometimes, but not always.
Lots of people prompt by cargo cult -- you find lots and lots of images generated by peppering the list with stuff like "masterpiece, best quality, 8k, award-winning".
Are all of those found in the training set? Do they actually have the right result?
It's statistics, not magic. The machine doesn't know what you mean, it only knows what's in the dataset. Some tags are not well represented enough to have the result you'd expect from the name.
Googling around suggests "masterpiece" mostly translates to higher contrast and saturation.
> they are not asking for results in a style that exists independently of Greg Rutkowski despite his name somehow being the only way to identify it.
Some might be. There's not always a tag for every kind of aesthetic, so one possible solution is to find an artist that uses it.
> which is the central pillar of his intellectual property as an artist, and his competitive advantage.
And which copyright law says he doesn't have. Style isn't copyrightable.
Sure—they often have such a poor understanding of art and the artist that they may not know that's what they're looking for, but that doesn't change the fact that it is, indeed, what they are looking for.
> And which copyright law says he doesn't have. Style isn't copyrightable.
The part of my comment you quoted says nothing about copyright law, so I'm not sure what you're trying to contradict here. An artist's personal style is a central pillar of their intellectual property in the sense that it takes massive personal investment to develop and everything they create flows from it, and it is a central pillar of their competitive advantage in the sense that it's aesthetically unique, positively stimulating to the viewer, and links their identity to their entire body of work in a way that can be exploited in the marketplace.
The fact is that copyright law currently has no idea what to make of these AI tools. I think the only concrete opinion so far adopted by the US copyright office (which may in the future be superseded by legal precedent or law) is that products of AI alone are not products of human authorship and thus cannot themselves be copyrighted. It will take time for arguments to be made, for money to change hands, and finally for more broadly useful law to be settled, which is why I previously alluded to a period of ten years to let things shake out.
If copyright law does step up to protect artists from this novel attack on their intellectual property, I doubt that protection will come in the form of allowing styles to be broadly copyrightable. From the contemporary commercial perspective, most of the value proposition of these tools is the ability to automatically and at scale launder the intellectual property of artists into derivative works that ostensibly do not infringe on the original works; given that, it is very much in the spirit of existing intellectual property law to write new law to protect artists.
Only time will tell whether the inevitable evolution of intellectual property law to account for AI tools favors individual artists or large tech corporations.
> Only time will tell whether the inevitable evolution of intellectual property law to account for AI tools favors individual artists or large tech corporations.
Definitely tech corporations. I'd be careful placing my hopes on copyright.
You know there's a lot of public domain out there, as well as permissively licensed images, right?
So here's my future prediction:
1. If forced, AI companies will prune their datasets of anything not permissively licensed.
2. They'll start friendly community initiatives to fill in the holes. Just like Wikipedia has requests for pictures, so can AI.
3. There's probably going to be more interest in doing more with less, and in training AI on AI output.
Given that AI output is deemed non-copyrightable, the end result is a vast amount of public domain pictures, which can be freely used without paying anyone.
Think it's unlikely? I think it'd very likely based on what happened with the software industry. The industry heavily encouraged Open Source as opposed to Free Software, and as a result today you can easily run a business with zero licensing concerns or payment to anyone.
And once you're there, there's no taking this back. There's not once in my life that I've thought "You know, it's nice that GCC and Clang are free, but I really feel like paying big bucks for a compiler".
> Definitely tech corporations. I'd be careful placing my hopes on copyright.
Yes, I expect you're right.
> [...] Think it's unlikely?
The part of your scenario I'm most skeptical of is the part where artists freely contribute their work to AI training sets en masse. Even putting aside the current (admittedly somewhat reactionary) trend of artists being loudly contemptuous of AI tools and (to a large extent) the people who use them, I don't think the incentives for artists to post their work online are quite the same incentives that drive programmers to open-source their work, and I really don't think having your work subsumed into a vast training set where it completely loses its identity is nearly as satisfying (or professionally enabling, for that matter) as being able to show e.g. that your open-source library gets 100k downloads every day and is a critical component of X, Y, and Z popular software packages. Art—especially art created for non-commercial reasons, which is what we're talking about here—is often very personal, and as a result artists crave direct engagement from the people looking at it.
If the hypothetical "community initiatives" you're proposing can make such contributions more rewarding, either by drastically improving attribution on the output side of the model (seems unlikely) or by offering some other incredible value proposition distinct from the "opportunity" to have one's works blended into the AI slurry, maybe. But if the AI industry and community ever do actually start listening to artists and respecting their intellectual property to the point that they are no longer simply taking what they want just because they can, I think it will be an uphill battle to convince artists to freely give the same without compensation.
It seems more likely to me that large AI companies would need to pay artists to contribute to training sets, making themselves and their peers partially or wholly redundant. I don't think that's great either, but at least it isn't bald-faced theft of intellectual property.
> The part of your scenario I'm most skeptical of is the part where artists freely contribute their work to AI training sets en masse.
You're not quite getting it.
First, there's the public domain. Centuries of art hanging in museums and the like is there for the taking. There's also some modern works there like I think works made for the US government.
Second, there's permissive licensing. For instance I uploaded a bunch of stuff to Flickr that I don't mind at all being used for AI. That's also free for the taking.
Third, if AI output is not copyrightable, AI output can be fed to itself. So lacks in SD can be filled by generating, filtering and reinforcing what you want.
Fourth, if any jurisdiction declares that training is fair game, that can also be added.
Fifth, it occurs to me that if AI output is not copyrightable, then it possibly allows proprietary AIs to be gradually blended in. Eg, say Adobe runs its own. It might own the training set, but if Adobe legally licensed everything, but the output is not copyrighted, then you can feed it to another system still and get some benefit from Adobe's work that way.
This constitutes already a starting set that can be used.
> But if the AI industry and community ever do actually start listening to artists and respecting their intellectual property to the point that they are no longer simply taking what they want just because they can, I think it will be an uphill battle to convince artists to freely give the same without compensation.
So you don't. You don't appeal to traditional artists, you appeal to AI users, who are seeing their cool tools attacked by the artists. You convince them to find more material in a public library, or to make some (since AI output is not copyrighted).
> I don't think that's great either, but at least it isn't bald-faced theft of intellectual property.
I honestly fail to see the long term benefit. I see little satisfaction in "Well, the machine took my job, but at least I fought a legal battle to make sure modern AI is trained from squeaky clean 18th century art".
Once you have a clean AI that's it, that's the end game. You can even sell that as a perk. Why deal with copyright, royalties, and all that nonsense? Here's a machine that will do whatever you want for cheap and won't ever ask for anything.
I guess I must be failing to respond to the point you're trying to make in the way you expect. I admit that I am not quite sure what that point is.
There are several overlapping concerns here. One concern is the possibility of AI systems violating the intellectual property rights of living artists whose work is not in the public domain and who have not given permission for it to be used in this way. Another (but not the only other) concern is working artists being outcompeted by AI in the marketplace. You seem to be responding to the second concern; I am responding to the first, since it's most relevant to the context of Greg Rutkowski's name and work being included in AI art training sets without his consent and (now) explicitly against his wishes.
If you think I'm arguing that removing copyrighted works from the training sets of AI systems whose operators haven't secured the rights to said works will ultimately prevent such systems from partially or totally outcompeting human artist(s) in some market(s), you're wrong. I have not argued that and I will not, since I don't believe it. What I believe is that the cat is out of the bag with these systems and that we will be stuck with them for the foreseeable future, regardless of how their creators manage to finagle training sets, regardless of their impacts on the prospects of working artists and more broadly on culture. I suspect attempts to curtail their use or ban them outright would do more harm than good. But I also believe that all of the above being true doesn't give us the right to run roughshod over the existing intellectual property rights of artists as we grease ourselves up for the long slide into cultural oblivion.
> Only time will tell whether the inevitable evolution of intellectual property law to account for AI tools favors individual artists or large tech corporations.
In the end, its going to be “large tech corporations” vs. “large media corporations”, with both sides citing individual artists (in one case, their users, and in the other, their suppliers) to present the clash of titans as a David vs. Goliath story with themselves cast as David.
> If you remove artist names from the training data, you lose the ability to describe those styles
Do you?
With base SD1.5, “by greg rutkowski” produces extremely similar results to “detailed realistic muted fantasy”. Except for the case where what you are actually trying to do is directly mimic the work of an artist with a strong unique style that is very heavily represented in the source data (Rutkowski, IIRC, has extremely little representation in the SD training data, leaving aside question of whether the other parts of that description apply), I don’t think you are losing the ability to refer to styles without artist names.
