I really, really like your idea with Evidence. I long for a mode in Metabase that’s like a “notebook mode”, where the main focus is narrative and it’s ornamented with viz, rather than the other way around.
I want to be able to publish this notebook when I’m done and then be able to hand that around, the same concept that you’ve built Evidence around. I think that’s a very good idea, so thanks.
The main thing keeping me from switching is that Metabase’s query builder and visualizations are too good for 95% of my work. It’s hard to picture going back to writing _all_ the SQL.
I hear that. We're making a lot of progress on reducing the amount of sql you need to write, keeping it DRY etc. Making the dev experience really buttery and high leverage is definitely a priority.
Here are a few of the things we're working on in that regard:
1. Making our components issue their own queries so that you don't need to write full sql expressions, just fragments when you're defining the chart you want.
2. Improving intellisense -- right now you get "slash commands" and snippets in our vscode extension to invoke components (which are really sweet), but we're aiming to get to a full intellisense type of experience.
3. Supporting external metrics layers where it makes sense. We've got some users using Cube, we're interested in Malloy, and the dbt semantic layer, those types of things.
One of our team members is very keen on building something he calls "Evidence studio", sort of a wsywig you could invoke during development for generating basic queries, setting up charts etc. that syncs the components into their text representation. That'd be further off though :)
Platforms like this are always pretty funny. It's basically a gateway drug to their cloud platform (which isn't free) that they hope you use, but then they keep it open source so that they don't have to pay salaries to w2 employees. Smart thinkin'!
For anyone who is interested in our cloud service, it's an easy way to put your project online, keep it up to date with your data, and place it behind access control.
For many organizations, hosting Evidence in their own infrastructure is easy enough, but if you don't want to manage that, we are happy to manage it for you.
It is not free (that's how we pay the salaries), but pricing is available here:
Right now I'm heavily into Grafana but when it comes to BI it kinda falls flat, I regularly have to fall back to using the Plotly plugin to create the charts (but it's getting better, at least you can do a normal scatter or bar chart out of the box since version 8. Labeling the axes is still a problem though). Navigation is also problematic, like jumping to a source table takes a lot of effort to make (basically you have to create a new dashboard and do some hyperlinking instead of there being a ready to go "view source data" button). I feel like there's a lot of friction to get Grafana to do BI, but I've also invested so much time in it I'm afraid to jump ship...
Superset looked good, but operating superset quickly runs into the same Python issues all Python software suffers from.
Sometimes it would just break for no apparent reason. Configuring it was a nightmare of magic Python code and unclear settings. Trying to use plugins was equally painful: due to the poor boundary separating the applications dependencies from the plugins dependencies, adding a db connector could just bork the whole application.
Very sound advice. I started the setup process for Superset, and it's even worse on Windows. Contrast that to Superset which is a single .jar file and worked instantly.
What do people use for an "analytics.js" for reporting events with common items like campaign data, user device and user profile measurements, and related from browsers and devices?
Is there any open source BI tool that can be embedded in some other product so that users (not product developers but final users) can create their own dashboards?
IMHO Metabase is too complex for end users: too much "code" (queries) to write and very limited in the presentation layer (widgets users can use to show the results).
I was thinking of something closer to Power BI, e. g. something like AppSmith but more BI-oriented (AppSmith is a generic tool to build your own applications, kind of like Visual Basic or Delphi).
Looks pretty neat. Having not fully investigated it yet, I will say the one thing I usually run into with this and other BI tools is a lack of flexibility in the UI for forming queries. It's sometimes limited to one table or view at a time. I wonder why more of them don't use more flexible querying techniques, perhaps just due to the risk of a bad query being formed?
>Where can I find persistent application data storage?
>We use the .us-data folder to store PostgreSQL data permanently. You can delete this folder if you want, it will be recreated with the demo data after restarting the datalens-us container
I really like the concept of Domo. They have ETL, modeling, a warehouse and BI in one app ("data-stack-in-a-box"). I've interviewed 20 of their customers and the general sentiment was pretty bad. There's a long sales process, a longer process to get it set it up, and they've built all the modeling and connectors themselves (vendor lock in, none are best-in-class).
Definite (https://www.definite.app/) is a data-stack-in-a-box. We have a built-in modeling layer for core metrics and an AI assistant to answer any one-off questions.
A few ways we're different:
Built on open source - We run the data stack for you and give you a single app to manage and analyze your data, but it's all built on open source standards. So if you decide at any point you want to run it all yourself, the code is yours to lift and shift to your own infrastructure.
Battle tested connectors - We're using Meltano / Singer (open source library from Stitch) for our connectors, so they've been used heavily in production for years.
Self-serve that actually works - A lot of tools promise self-serve, but AI is making this real. We've invested heavily in making it possible to ask questions and get accurate answers. The AI queries a modeled view of your data that can answer questions that depend on well defined metrics (e.g. ARR, DAU, etc.).
1. When you bring your own data warehouse, we parse the query history, convert it so an AST and learn JOIN's from there
2. When you're using our managed data warehouse and ETL, we already know most of the JOIN's (e.g. we know how to join the Hubspot data we ingress to Stripe)
3. For anything not covered by #1 or #2, we have a modeling layer where you can specify how to join tables.
We have Domo at work and it just seems overly complicated and insane. I'm wanting to learn it instead of just polling data sources and adding them to a local Opensearch instance, but... too verbose for me.
I would have no idea. Why are you asking me and sharing a link to something which tells you. That’s weird, and I’m not interested in clicking the link anyway.
Cool. Rolling your own BI seems fraught with peril at most orgs where I imagine the buy vs build decision is always buy. How many PowerBI or Tableau seats do you need before rolling your own internal BI platform starts to make sense?
I don't think it ever makes sense because the large players will always be able to make new innovations (mobile apps, natural language querying, SSO integrations, etc) that, short of large corps hiring BI teams to invest in the open source ecosystem like superset, your open source solution won't have.
I must like to live dangerously. In all seriousness though there are low cost alternatives to those mega BI tools that suit many use cases. If I wasn't rolling my own I'd probably start with Metabase or Superset. What I use: https://github.com/totalhack/zillion
Not only you. The company that I work for delivers software to government agencies. Dependencies like libraries or stack parts that are Russian or Chinese are simply banned. Even Harbor was rejected (and probably should be, as they provide a bunch of their own built Docker images, so you never know what your stack runs). I know "but it's open source" etc., and somebody may feel offended, but that's reality.
Yes, that is really a shame, although it's been fully open sourced.
Yandex is aware of how the geopolitical situation is hurting them and are therefore building a new company called Double.Cloud, based in Europe, to work around the negative public opinion on Yandex, and thereby continue being able to sell Clickhouse cloud services.
Some projects have refused code from developers because they are Russian or Chinese. Just because it is open source doesn't mean it is unbiased, free from politics, and without racism.
Since the discussion started in the comments already, I have a similar question: any recommendations for a solution (don’t care if OSS or not) that has the best UX for nontechnical people to assemble some data and reports anyhow? I have Salesforce, some mariadb/postgres and (optionally) hubspot as data sources.
I can buy or manually provision anything, no technical hurdles or policies from that side. My absolute focus is the raw UX for business people.
Honestly Metabase has given best balance between allowing non-technical users to self-service and technical users to dig in and use raw sql if that's what they want. Also it OSS core so you can self host. It is super feature rich and has most everything in the OSS version as long as you don't need enterprise features like SAML auth, audit log, ...,etc
Metabase is the only tool I’ve used where I’ve managed to get non-technical users to actually engage and use to query building tools to answer their own questions.
I'm a co-founder of Dataland.io where we're building a powerful dataset viewer + search engine that can work on top of your Postgres or data warehouse.
We designed it specifically to provide an excellent UX to business users while reducing BI burden on the data team. We find that most business users often just need to search, filter, and sort instead of looking at charts to make operational decisions.
UX-wise, what sets us apart are:
- <1s full-text search (even on billions of rows of data), feels like Cmd+F in Google sheets, but faster
- Performance: we stream billions of rows into the web browser, seamless scrolling (no paging of 50 records at a tieme)
- Rich cells make tables easier to scan/read (enum strings => colored tags, numbers => color-coded based on value => checkboxes, timestamptz => clear date time pills)
If that fits what you need, happy to give you a demo.
arthur(at)dataland.io
Otherwise, I think the simplest BI (if charts are impt) could be something like evidence.dev or Metabase.
But I also think it's going to require some curation on your part. Can you reasonably expect business users to navigate the entire schema/table tree across these three sources? That's where I think the bottleneck often lies -- if your BI tool allows engineering to just expose a subset of curated core tables.
The problem is that developing the perfect UI for nontechnical people to assemble reports probably requires a bespoke frontend for your business, and one that likely lags behind the reality of its changes. Most businesses instead opt to just hire semitechnical people that can do a bit of data work and give answers to the report-writers, as they can accommodate business changes over time and understand how to construct new queries out of the overall business' data sources.
Maybe that'll change one day with AI, and when it does that will be bought by every big company in the world (-:
I dislike about your pricing that it tells me reasonable 29$ and then in the fineprint it says minimum 5 users. I get the reasoning behind your pricing logic, but I really dislike it. Now as solo business owner I'm gone.
