Everything they published thus far in the generative AI space has been abysmal in quality compared to the competition. I'd be hella surprised if this reaches GPT-4 levels of quality...

I'm a GPT4 subscriber and a Google GSuite work subscriber. I've been using the latest Bard this morning to write and refine python code, and it's just as good if not slightly better than GPT4. I asked it to refine some obtuse code with lots of chaining, and it did an admirable job writing accurate comments and explaining the chained logic. It's ridiculously anecdotal of course, but I used Bard for all of 5 minutes last time they announced. This time seems different.

Manifold has this at 69%, so here's an opportunity to take some people's internet points: https://manifold.markets/brubsby/will-googles-gemini-beat-gp...

Looks like it's 75%? It briefly dropped to 68% in October/November, but has been fairly consistently around 75% for a while.

It's very active today: 50+ trades in the last hour. When I checked it was 69%, but it's gone up and down since then. Click on the "trades" tab to see.

Ah fair enough, this seems to be an awkward UI. Over time though 75% seems to be the representative recent position. It is up to 85% now though!

I’m curious which instances of overpromising you’re referring to.

Like how much they hyped up Bard, which when released turned out to be barely competitive with GPT3.5. E.g. https://www.reuters.com/technology/google-ai-chatbot-bard-of...

I definitely think GPT is better than Bard, but Bard definitely did live up to the hype in a few ways. The two that blew my mind (and still do to some extent) are the blazing speed and the ability to pull information real time (no more pesky knowledge cutoff date). Bard also felt pretty comparable to 3.5 to me, better in some things and worse in others. Coding was definitely a bust with Bard.

ChatGPT via plugins and tools can access real time data, the bot I built for slack at my work has the ability to load Web pages, search DDG etc.

I do not recall Bard being said to be better than any particular other model, but then having worse performance by some metric when released.

Your link isn’t really an indication of an overpromise.

Bard isn't a model, it's a product. Saying comparisons against "Bard" without specifying a particular point in time are like analyses of "ChatGPT" without specifying a model. There have been a number of releases adding more features, tool use, making it smarter, and crucially adding more languages. ChatGPT is not fine-tuned in different languages – it manages them but lacks cultural context. That's one place Bard is quite far ahead from what I've seen.

all that shows is that google screwed up their positioning, and openai got it right

people don't see a difference between model and product, they think "gpt3 is ok", "gpt4 is great", "bard is like gpt3"

it's not the consumer's fault when the business has a positioning mistake, the business has to try and win the consumer back

Most people don't use LLMs. Of those that do most people just know they're using "ChatGPT". A slim minority care about the model.

In my opinion, not focusing on the model, focusing on the product, and focusing on positioning for normal users (free, fast, fine tuned in many languages, "easy"), is a better product positioning.

> In my opinion, not focusing on the model, focusing on the product, and focusing on positioning for normal users (free, fast, fine tuned in many languages, "easy"), is a better product positioning.

Does google agree? doesn't the fact that they're so deliberately creating user-focused branding for different models (ultra, pro, nano) show they also see the value in the differentiation?

I can't speak for Google, and must emphasise that these are personal opinions. However I'd say that this entire marketing push is mostly for the super-engaged early adopters, not targeted at the general public. Looking at the YouTube videos, the more they seem to be targeted towards a general audience the less they mention these specifics. So, I suspect that the Ultra/Pro/Nano branding will mostly be used on the advanced Bard product that they speak about in the launch blog post, and on the APIs available to developers.

In terms of AI? Last year? A better question is what wasn't an overpromise?