Very similar to your second example, I've been using ChatGPT to calculate the total calories in a recipe. Before, I would consult a database like the USDA food database for nutrition info and do measurement conversion manually. I've tried asking ChatGPT for the same recipe and the total amount was within 1% of what I calculated. It runs into a couple snags I've noticed though.
First, it always gives a calorie count for cooked meat, but it should assume the meat is uncooked since I said it was for a recipe.
Second, it seems to struggle with the concept of uncooked rice. If you ask it to work with 1 "rice cooker cup" of rice, it refuses because that isn't a standard measurement. If you swap in the converted standard measurement (3/4 cup), it still is way off. It told me 3/4 cup uncooked rice is about 150 calories when cooked. That's a third of what the USDA database gives. When you point out that 3/4 cup uncooked rice is a large serving after being cooked, it changes its answer to 375 calories, still about half of what the USDA database gives. But this is fine for me because rice is not typically part of my recipes since it doesn't usually require special preparation.
Overall it reduces a 10 minute task to 10 seconds, but you need to know enough about the ingredients to spot obvious problems in its result. In my case I could see the calories given for meat was way too high, and way too low for rice. It gave a better answer after telling it to fix the former and ignore the latter.
I tried a second recipe and the total it gave was 2% under my calculation, but I did not see any obvious error in its result so I could not correct it further.
It is unfortunate that you kind of have to trust the numbers are correct, but this is no different than the nutrition details on sites like MyFitnessPal which are often wrong when you closely examine it.
Wait. 150 is one third of the USDA answer, and 375 is half of the USDA answer?
What is the USDA answer?
This equation is beyond my paygrade!
Edit:
I asked our GPT3.5 bot to solve this, and it hallucinated "pulling up the USDA database" ; complete with a "(a few moments later...)" message before giving me 160 calories as the USDA answer.
I asked the same bot (identical prompts) with GPT-4-Turbo enabled and it went through "step by step" to say the correct answer is 461 calories because 1/3 cup uncooked is 1 cup cooked, so 1 rice cooker cup (160g) = 3/4 cup uncooked, so 2.25 cooked * 205 = 461 cal.
Is that the right answer? If so, 375 seems far from "half"
The database gives 716 calories for 1 cup uncooked rice. So, 537 calories for 3/4 cup. It was a mistake to say 375 was half of 537, my intention was to point out that it's not close.
I don't have access to GPT-4 but your results are interesting. I don't understand why it's trying to approximate the yield of the uncooked rice after cooking before giving a calorie estimate. From my testing it doesn't do this for other ingredients like raw chicken.
It seems to be more accurate when giving calorie estimates for a list of ingredients instead of a single ingredient. For example, in a recipe that includes "boneless skinless chicken thighs (1 lb, raw)" ChatGPT gives "about 880 calories" as the answer. But in a new prompt if you ask it only about the chicken and nothing else, it gives "between 1100 and 1300 calories." The USDA database shows 144 calories per 100 grams, so 653 calories per pound. The first answer by ChatGPT is much more accurate.
I have no idea why this is the case, and I bet this would be easy for a researcher to demonstrate if true. Alas, I don't have API access.
You should probably consider most calorie calculations to be +-50% anyway due to huge differences between human digestion and the way calories are estimated in a lab.
First, it always gives a calorie count for cooked meat, but it should assume the meat is uncooked since I said it was for a recipe.
Second, it seems to struggle with the concept of uncooked rice. If you ask it to work with 1 "rice cooker cup" of rice, it refuses because that isn't a standard measurement. If you swap in the converted standard measurement (3/4 cup), it still is way off. It told me 3/4 cup uncooked rice is about 150 calories when cooked. That's a third of what the USDA database gives. When you point out that 3/4 cup uncooked rice is a large serving after being cooked, it changes its answer to 375 calories, still about half of what the USDA database gives. But this is fine for me because rice is not typically part of my recipes since it doesn't usually require special preparation.
Overall it reduces a 10 minute task to 10 seconds, but you need to know enough about the ingredients to spot obvious problems in its result. In my case I could see the calories given for meat was way too high, and way too low for rice. It gave a better answer after telling it to fix the former and ignore the latter.
I tried a second recipe and the total it gave was 2% under my calculation, but I did not see any obvious error in its result so I could not correct it further.
It is unfortunate that you kind of have to trust the numbers are correct, but this is no different than the nutrition details on sites like MyFitnessPal which are often wrong when you closely examine it.