I’m one of the founders of Shaped. We’ve spent the last few years building relevance infrastructure.
Over the time we've noticed a convergence happening: The stack for Search (usually Elastic/OpenSearch) and the stack for Ranking (Vectors + Feature Stores) are merging. Maintaining two stacks for the same math is inefficient.
We just launched Shaped 2.0 to collapse this stack. It treats Relevance like a database problem:
1. Ingest: Connect to Snowflake/Postgres/Kafka.
2. Index: We handle the embeddings (ModernBERT, etc.) and statistical feature engineering.
3. Query: We built a SQL-like language (ShapedQL) to retrieve, filter, and rerank candidates in <50ms.
Instead of writing Python glue code to merge BM25 and Vector results, you can do it in a single query:
SELECT title, description
FROM
semantic_search("$param.query"),
keyword_search("$param.query")
ORDER BY
-- Combine semantic relevance (ColBERT) with a personalized model
Yeah exactly. I was really worried about reducing the serendipity that HN provides (as it's arguably why I've used it for so long as well) but the configurability allows it so that everyone can tweak their level of personalization to get their perfect goldilocks level.
Yeah I definitely agree. I think the nuance is that personalization doesn't have to completely overrun the current ranking algorithm, it more just slightly boosts known similar content you might like. Should be the best of both worlds really.
I think you're missing what I'm describing: which is that HN is a rare thing for what it is
And for what it isn't
Putting a little bit of what something specifically isn't (algorithmic personalization on HN) can change something and your own perception of what "is trending" or what is hot right now
There are very few corners of the web without this kind of algorithmic feed & maybe not everything needs to be "best of both" when including a little of both completely changes it
The performance for us was best when we evaluated a couple of options, both in terms of scale and latency. I also like the arrow/dataframe interface. We use arrow everywhere else at Shaped so it was a natural integration.
Yes when integrating Shaped you connect up the data sources needed to ingest: interactions, items and users. The Shaped interface then allows you to select which exact fields should be used for creating a Shaped model. We provide a full SQL interface to do this, which gives a lot of flexibility.
Our dashboard provides monitoring to help understand what data is ingested and view data quality over time. We expect customers to monitor this but also have alerts on our side and jump in to help customers if we see anything unexpected.
The dashboard also shows training metrics over time (how well does the model predict the test set after each retrain?) and online attribution metrics (how well does the model optimize the chosen objective?).
Customers can disable retraining if they want (which is essentially pinning the model version to current), we can do model version rollbacks on our side if we see an issue or if requested but it's not a self-serve feature yet. Because we've made it easy to create or fork a Shaped model, we've seen customers often create several models as fall-backs that rely on more static data sources or are checkpoints of a good state.
Australia represent! Although we're based in NYC we still are a mostly Aus/international team over here, it's great!
The biggest change is some of the less sexy stuff, like scale and security. E.g. we're now able to scale to 100M+ MAU companies with 100M+ items, and we have a completely tenant isolated architecture, with security as a top priority.
We've also made the platform more configurable and lower levels and we've found that people like choosing their own models and experimenting rather than just relying on our system.
Finally, we launched search only a couple of months ago and are currently heavily focused on building a best-in-class experience there.
Compared to Vespa, we're much easier to get setup on. A big part of this is that we have real-time and batch connectors to all leading CDPs and data warehouses. E.g. if you're on Amplitude it takes < 10mins to stream data directly to Shaped and start seeing initial results.
Being quicker to setup, also means it's quicker to build and experiment with new use-cases. So you can start with a feed ranking use-case the first week and then move to an email recommendation use-case the next week.
In terms of actual performance and results, we've never gone head-to-head in an A/B test so i'm not sure the specifics there honestly!
The short answer is: we're better at recommendations and personalization and lean towards more technical teams (e.g. even with data/ML experience). They're better at traditional search and, these days, lean towards less technical teams.
Yes play.shaped.ai! We just opened that up in a gateless way for this post. Let me know what you think. I should also mention that these demo models are on our cold-tier so that it doesn't break things, in production there's a big speed up.
I’m one of the founders of Shaped. We’ve spent the last few years building relevance infrastructure.
Over the time we've noticed a convergence happening: The stack for Search (usually Elastic/OpenSearch) and the stack for Ranking (Vectors + Feature Stores) are merging. Maintaining two stacks for the same math is inefficient.
We just launched Shaped 2.0 to collapse this stack. It treats Relevance like a database problem:
1. Ingest: Connect to Snowflake/Postgres/Kafka. 2. Index: We handle the embeddings (ModernBERT, etc.) and statistical feature engineering. 3. Query: We built a SQL-like language (ShapedQL) to retrieve, filter, and rerank candidates in <50ms.
Instead of writing Python glue code to merge BM25 and Vector results, you can do it in a single query:
SELECT title, description FROM semantic_search("$param.query"), keyword_search("$param.query")
ORDER BY -- Combine semantic relevance (ColBERT) with a personalized model
We have a free tier ($300 credits/no-cc) if you want to try the SQL abstraction yourself.Try it here: https://console.shaped.ai/register