> Are there leaderboards that you follow or trust?
Not for OCR.
Regardless of how much some people complain about them, I really do appreciate the effort Artificial Analysis puts into consistently running standardized benchmarks for LLMs, rather than just aggregating unverified claims from the AI labs.
I don't think LMArena is that amazing at this point in time, but at least they provide error bars on the ELO and give models the same rank number when they're overlapping.
> Also, do you have preferred OCR models in your experience?
It's a subject I'm interested in, but I don't have enough experience to really put out strong opinions on specific models.
Having ripcorded out after realizing the author was trying to prove that water was wet, I'll assume that it's "normalized entropy", in a range of 0-1, indicative of the distribution across the space.
That seemed to me like a radically high estimate of people who use ad blockers. But I see that the first page of results on MyFavoriteWebSearchEngine support that claim.
> This was a tough decision, having used Gmail since 2007/2008. However, I had to draw the line and stop giving Google my data for free.
>
> The problem with email is that everything is transmitted in plain text.
Interestingly, one of my biggest problems with Gmail is that they don't allow actual plaintext. I used to routinely collaborate with developers who were vision-impaired, and the official Gmail phone app wouldn't let me send them plaintext email. Instead, it was some sort of HTML thing. Unfortunately, we sometimes sent code snippets to each other over email, and though admittedly it looked more or less fine, Gmail changed the underlying representation enough that my collaborators' screen readers would mess up on the parsing.
This led to me leaving Gmail on my phone, which led ultimately to me leaving Gmail entirely.
I think you use the term "plain text" differently from the author of the post. I think they refer to the fact that there is no end to end encryption. Google has access to the clear text of all messages and can index/analyze them.
The article does call out plain text email without formatting or attachments. Plain text typically refers to visual formatting, while clear text refers to lack of encryption.
From the title, I had thought that this would be a new tool for searching science, such as searching the arxiv. But this is actually a survey.
I quote the conclusion of the survey:
---
In conclusion, rapid advancements in artificial intelligence, particularly large language models like OpenAI-o1 and DeepSeek-R1, have demonstrated substantial potential in areas such as logical reasoning and experimental coding. These developments have sparked increasing interest in applying AI to scientific research. However, despite the growing potential of AI in this domain, there is a lack of comprehensive surveys that consolidate current knowledge, hindering further progress. This paper addresses this gap by providing a detailed survey and unified framework for AI4Research. Our contributions include a systematic taxonomy for classifying AI4Research tasks, identification of key research gaps and future directions, and a compilation of open-source resources to support the community. We believe this work will enhance our understanding of AI’s role in research and serve as a catalyst for future advancements in the field.
---
I jumped at this because I'm a mathematician who has been complaining about the lack of effective mathematical search for several years.
How do you view o3? I personally find it superior to google search almost always. Do you find that it often misses key references? (also mathematician)
Google is completely inadequate at mathematical search. But here is a concrete problem that no search seems to handle: given some complicated integral (say, some contour integral involving a K-Bessel function), find where it appears in the literature.
Most search will totally fail, because this is made of math symbols. Embedding-based search will give various related things involving, say, integrals and Bessel functions. But then I end up opening Gradshteyn and Ryzhik and trying to find where in this book the relevant terrible integrals appear.
This is a common experience for analytic number theorists. And it's a lousy experience.
It depends of what you need, but for example for calculus is a nice program. There is also sympy and Wolfram Mathematica. For symbolic computation I think that Mathematica is the strongest then maxima and then sympy, but sympy is based on python and I think it will get stronger. If you need numerical computation then there is octave or matlab or julia.
Also, do you have preferred OCR models in your experience? I've had some success with dots.OCR, but I'm only beginning to need to work with OCR.
reply