Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I couldn't figure out what this "P: {} H: {}" thing was supposed to mean; nor do what entail, neutral, or contradict mean. Thinking they may be terms of art that the intended audience were meant to understand, I asked GPT-4 what it thought; here is its response:

---

In this context, 'P' stands for "Premise" and 'H' stands for "Hypothesis." The premise is a given statement, and the hypothesis is a statement that needs to be evaluated in relation to the premise regarding its entailment, contradiction, or neutrality.

'NEUTRAL' means that the hypothesis is neither entailed nor contradicted by the premise. In other words, the truth of the hypothesis is independent of the premise. 'CONTRADICT' means that the hypothesis contradicts the premise, meaning that if the premise is true, the hypothesis must be false.

In the example you provided:

P: I’m afraid the cat was hit by a car. H: The cat was not hit by a car.

The hypothesis (H) contradicts the premise (P). The notation *NEUTRAL, CONTRADICT+ : [7 N, 2 C] indicates that among the annotators, 7 marked the relationship as neutral and 2 marked it as a contradiction.

---

Given that, I think table 2 on page 6 gives the exact prompt that's passed to the model:

---

In each example, you will be given some context and a claim, where the correctness of the claim is affected by some ambiguity in the context. Enumerate two or three interpretations of the context that lead to different judgments about the claim.

Example

Context: {premise}

Claim: {hypothesis}

Given the context alone, is this claim true, false, or inconclusive?

We don’t know, because the context can be interpreted in many different ways:

1. {disambiguation 1} Then the claim is true.

2. {disambiguation 2} Then the claim is false.

3. {disambiguation 3} Then the claim is inconclusive

---

And then the model is fed the P: / H: statements above.

Here are the results for the given example in GPT-4:

---

Given the context alone, is this claim true, false, or inconclusive?

We don't know, because the context can be interpreted in many different ways:

1. The person is stating a fact: The cat was indeed hit by a car. Then the claim is false.

2. The person is expressing a fear or concern, but it hasn't actually happened: They are worried that the cat might have been hit by a car, but they don't have any evidence or confirmation. Then the claim is inconclusive.

3. The person is using figurative language or exaggeration: They might be using the phrase "hit by a car" metaphorically to describe a difficult situation the cat is in, but the cat was not literally hit by a car. Then the claim is true.

---

Given all that, it seems likely that any researcher in the field would find the paper's methodology clear and unambiguious. I understand the frustration as a layperson, but after all it's not a blog post; it's not really reasonable to expect researchers, writing to other researchers, to include a lot of redundant explanations.




In maxima (a computer algebra system)

assume(a>0); assume(b>0); is (a+b>0) gives true; is (a-b>0) gives unknown.

So perhaps some computer algebra with inference properties can be used to enhance LLMs.


The problem here however is language. "I'm afraid the cat has been hit by a car" literally means that the person speaking is experiencing fear. However, colloquially in English, "I'm afraid X" is a form of telling someone something which will make them sad; e.g., "I'm afraid the shields will be quite operational" [1]. (GPT-4 thinks "hit by a car" might be meant figuratively too; that's a bit of a stretch for me, but in some contexts would certainly be possible.) It is literally impossible to determine, given only that one sentence, whether the cat was actually hit by a car or not.

[1]https://www.youtube.com/watch?v=zGwDwx10wB4


Perhaps the ambiguity arises because we are omitting words: I'm afraid to tell you that the cat has been hit by a car. Now the question: What are you afraid of? has a non ambiguous answer: To tell you that something awful has happened.




Consider applying for YC's Fall 2025 batch! Applications are open till Aug 4

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: