That's a biased definition, by it's own definition. ;)

Leave out the part about being wrong, and you will have the gist of what I'm saying. Also leave out the identifiable part: bias exists regardless of whether or not it is recognized.

Bias is how we work with subjectivity. When I answer a question, my answer will be specific to my bias. Without that bias, I could not formulate an answer, unless my answer was the one and only objectively correct way to express an answer to that question.

Computer programs are missing the bias feature. Everything written in a computer program is completely and unambiguously defined, all the way down to the language's foundational grammar.

LLMs are designed to introduce the bias feature. The limitation of this approach is that an LLM replaces the entire stack. None of the features of computation we are used to are compatible with an LLM. You can compute logic or bias, not both.

When you say that the definition I gave of bias is biased (in the sense I defined), what direction does it have a tendency to be wrong in? I assume by “wrong” you mean “not matching how people use the word”?

To clarify, when I said “identifiable”, I didn’t mean “identified”. I meant “in principle possible to identify”. Like, if you have a classifier between inputs where another thing (the thing being judged for bias) gets right answers and inputs where it gets wrong answers, and this classifier is both substantially simpler than the other thing, and gets a significantly better than chance success rate, and like, there is a human comprehensible thing about the inputs that this classifier is basing things on, then that’s a bias of the thing that is being judged for bias.

_____

Now for your definition:

Ah, I see, so your definition of “bias” is something like “a perspective” (except without anthropomorphizing) . It is something that picks among multiple options in a way that isn’t unambiguously specified by precise rules. (Kind of reminds me of filters/ultrafilters. Probably not actually particularly analogous, but still came to mind. I guess a closer analogy would be the concept of a choice function.)

The issue I have with this definition is that it doesn’t capture the (quite common) usage of “bias” that a “bias” is something which is bad and is to be avoided.

When people say that a process, e.g. a ML program, is “biased against brunettes” (for example) they generally mean this as a criticism of that process. And I think this being a criticism is a major part of what is meant by the word “bias” (in this type of usage of the word, not in the sense of a constant term in an affine map).

I do get that often people say that “everyone has their own biases” and “it is impossible to be unbiased (about [topic])”, and they will sometimes describe their general perspective as a way of warning people about their own biases, and this somewhat fits with the “a bias is a perspective/choice-function “ type definition, but, I think it fails to capture the reason that people mention biases : because they think they can lead to being wrong (either leading to inaccurate conclusions or to unjust/immoral/unfair choices). I don’t think it is just a warning of “I sometimes have to make a choice among several options where there is no canonical right choice, and you might make different such choices”. It is instead a warning to others that one, like everyone else, is fallible, and moreover, that there may be patterns in those failings that one does not perceive (on account of those same failings), but that others, who have different patterns in their failings, might perceive, and, at the same time, things that others might perceive as failings but are not, due to their own failings.

Hm.

But, I do note a shortcoming in my definition that yours doesn’t seem to have: if multiple people who believe that there is no such thing as objective aesthetic quality are talking about the aesthetic qualities of various works, they might sometimes describe their patterns in their aesthetic judgements as “biases”, especially when these patterns are differences in how they judge things aesthetically vs how others (would) judge those things aesthetically. This seems more in line with the definition you gave than in the definition I gave, because such people don’t believe that there is a truth of the matter as to the aesthetic quality of the works, and therefore would not consider the ways they differ to be patterns in being wrong, only in being different (or just in being). Though, I think it seems to have some aspects of both. The definition you gave doesn’t seem to really include the pattern aspect.

____

Still, I think when people complain that a machine learning model is biased, what they mean is usually more like the definition I gave?

____

I noticed another shortcoming in my definition. Sometimes the “bias” that people complain that something has is not really any individual answer/output being wrong, but rather something about there being something wrong/undesirable in the distribution of the outputs. For a simple example, if dice aren’t fair, we call them biased. This could conceivably be more along the lines of the “the constant term in a affine map” sense, but I think people would say the same thing about something that e.g. selects applicants, even if it never picks an applicant that is objectively less preferable over one that is more preferable, if it among equally qualified candidates has a tendency that would be unfair, this is still called a bias even if any individual such choice would be fine. Fixing this would be a small change in phrasing, or perhaps a footnote with clarification that the thing that is “wrong” doesn’t have to be in any individual output.

> When you say that the definition I gave of bias is biased (in the sense I defined), what direction does it have a tendency to be wrong in? I assume by “wrong” you mean “not matching how people use the word”?

I mean wrong, as in it conflicts with the subjective context I established by using the word my particular way. That was just a tongue-and-cheek way to illustrate the semantics of we are exploring here.

> To clarify, when I said “identifiable”, I didn’t mean “identified”. I meant “in principle possible to identify”

Sure, and I still think that can't work. Bias is a soupy structure: it's useless to split it into coherent chunks and itemize them. There are patterns that flow between the chunks that are just as significant as the chunks themselves. This is why an LLM is essentially a black box: you can't meaningfully structure or navigate a model, because you would split the many-dimensional interconnections that make it what it is.

> Ah, I see, so your definition of “bias” is something like “a perspective” (except without anthropomorphizing).

I actually am anthropomorphizing here. Maybe I'm actually doing the inverse as well. My perspective is that human bias and statistical models are similar enough that we can learn more about both by exploring the implications of each.

> The issue I have with this definition is that it doesn’t capture the (quite common) usage of “bias” that a “bias” is something which is bad and is to be avoided.

This is where anthropomorphization of LLMs usually goes off the rails. I see it as a mistake in narrative, whether you are talking about human bias or statistical models alike. We talk about biases that are counterproductive for the same reason we complain about the things we like: it's more interesting to talk about what you think should change than what you think should stay the same. Bias is a feature of the system. Instances of bias we don't like can be called anti-features: the same thing with a negative connotation.

The point I'm making here is that bias is fallible, and bias is useful. Which one is entirely dependent on the circumstances it is subjected to.

I think this is a really useful distinction, because,

> Still, I think when people complain that a machine learning model is biased, what they mean is usually more like the definition I gave?

this is the box I would like to think outside of. We shouldn't constrain ourselves to consider the implications of bias exclusively when it's bad. We should also explore the implications of bias when it's neutral or good! That way we can get a more objective understanding of the system. This can help us improve our understanding of LLMs, and help us understand the domain of the problem we want them to solve.

> For a simple example, if dice aren’t fair, we call them biased.

This is a good example. I'm extending the word bias, so that we can say, "If dice are fair, then they are biased toward true randomness." It's a bit like introducing infinity mathematics. This has the result of making our narrative simpler: dice are always biased. A player who wants fairness will desire random bias, and a player who wants to cheat will desire deterministic bias.

----

The reason I've been thinking about this subject so much is actually not from an interest in LLMs. I've been pondering a new approach where traditional computation can leverage subjectivity as a first-class feature, and accommodate ambiguity into a computable system. This way, we could factor out software incompatibility completely. I would love to hear what you think about it. In case this thread reaches max depth, feel free to email my username at gmail.