Because otherwise you'd have to convince AI-owners and select professionals to let go of their wealth to give a comfortable and fulfilling life of leisure to the unemployed.
More likely it will look like the current welfare schemes of many countries, now add mass boredom leading to unrest.
> Because otherwise you'd have to convince AI-owners and select professionals to let go of their wealth to give a comfortable and fulfilling life of leisure to the unemployed.
Not necessarily. Such forces could be outvoted or out maneuvered.
> More likely it will look like the current welfare schemes of many countries..,
Maybe, maybe not. It might take the form of UBI or some other form that we haven’t seen in practice.
> now add mass boredom leading to unrest.
So many assumptions. There is no need to just assume any particular distribution of boredom across the future population of the world. Making predictions about social unrest is even more complicated.
Deep Utopia (Bostrom) is an excellent read that extensively discusses various options if things go well.
> Not necessarily. Such forces could be outvoted or out maneuvered
Could.
> So many assumptions. There is no need to just assume any particular distribution of boredom across the future population of the world. Making predictions about social unrest is even more complicated
I’m assuming that previous outcomes predict future failures, because the forces driving these changes are of our societies, and not a hypothetical, assumed new society.
In this world, ownership, actual, legal ownership, is a far stronger and fundamental right than any social right to your well-being.
You would have to change that, which is a utopian project whose success has been assumed in the past, that a dialectical contradiction of the forces of social classes would lead to the replacement of this framework.
It is indeed very complicated, but you know what’s even more complicated? Utopian projects.
Sorry but I see it as far more likely that the plebes will be told to kick rocks and ask the bots to generate art for them, when asking for money for art supplies on top of their cup noodle money.
Well in Sam’s ideal world you’ll be using bots to keep yourself distracted.
You would like to learn to play the guitar? Sorry, that kind of money didn’t pass in the budget bill, but how about you ask the bot to create music for you?
Elites also get something way better than keeping people busy for distraction… they get mass, targeted manipulation and surveillance to make sure you act working the borders of safety.
You know what job will surely survive? Cops. There’ll always be the nightstick to keep people in line.
Tinfoil's chat lets you do that, add a bit of context to every new chat. It's fully private, to boot, it's the service I use, these are Open Source models like DeepSeek, Llama and Mistral that they host.
What is the lowdown on Databricks? Their bread and butter were hosted Spark and notebooks. As tasks done in Spark over a data lake began to be delegated wholesale to columnar store ELT, they tried to pivot to "lake houses", then I sort of lost track of them after I got out of Spark myself.
Did Delta Lake ever catch on? Where are they going now?
Capture enterprise AI enthusiasm by providing a 1-stop shop for data and AI, optionally hosted on your own cloud tenant. Keep deploying functionality so clients never need another supplier. Partner with SAP, OpenAI, anyone who holds market share. Buy anyone that either helps growth or might help a competitor grow.
Enterprise view: delegate AI environment to Databricks unless you’re a real player. Market is too chaotic, so rely on them to keep your innovation pipeline fed. Focus on building your own core data and AI within their environment. Nobody got fired for choosing Databricks.
You basically pay databricks a “fee” to choose the more appropriate and modern stack for you to build on, and keep it up to date. Never used it, but it handles with lots of the administrative bs (compliance, SLAs, idk) for you so you can just ship.
That does sound, as you allude, like IBM on its long downward spiral of globbing up products to stay relevant and touting them as an integral solution, while in-house development stuck to keeping legacy products alive for their Enterprise contracts. I wonder if they'll be foolish enough to start doing consulting around them, obliterating their economies of scale in the process; so far they are going with the "consulting partners" approach.
Oh well. Databricks notebooks were hella cool back when companies were willing to spend lavishly on having engineers write cloud hosted Scala in the first place, and at premium prices to boot.
A nice UI for a data lake house is underrated. I use AWS Athena at my work and it is just so bad for no good reason. For example, big columns of text are expanded outwards making reading the subsequent columns impossible.
Delta Lake is not catching on, but no worries, they bought Iceberg[0] (the competing standard).
I'm joking, but only a bit. Iceberg is open source (Apache), but a lot of the core team and the creator worked at Tabular and Databricks bought them for $1B.
It provides central place to store and query data. A big org might have a few hundred databases for various purposes - databricks lets data engineers set up pipelines to ETL that data into databricks and when the data is there it can be queried (using spark, so there's some downsides - namely a more restrictive SQL variant - but some advantages - better performance across very large datasets).
Personally, I hated databricks, it caused endless pain. Our org has less than 10TB of data and so it's overkill. Good ol' Postgres or SQL Server does just fine on tables of a few hundred GB, and bigquery chomps up 1TB+ without breaking a sweat.
Everything in databricks - everything - is clunky and slow. Booting up clusters can take 15 minutes whereas something like bigquery is essentially on-demand and instant. Data ETL'd into databricks usually differs slightly from its original source in subtle but annoying ways. Your IDE (which looks like jupyter notebook, but is not) absolutely suck (limited/unfamiliar keyboard shortcuts, flakey, can only be edited in browser), and you're out of luck if you want to use your favorite IDE, vim etc.
Almost every databricks feature makes huge concessions on the functionality you'd get if you just used that feature outside of databricks. For example databricks has it's own git-like functionality (which is the 5% of git that gets most used, but no way to do the less common git operations).
My personal take is databricks is fine for users who'd otherwise use their laptop's computer/memory - this gets them an environment where they can access much more, at about 10x the cost of what you'd pay for the underlying infra if you just set it up yourself. Ironically, all the databricks-specific cruft (config files, click ops) that's required to get going will probably be difficult for that kind of user anyway, so it negates its value.
For more advanced users (i.e. those that know how to start an ec2 or anything more advanced), databricks will slow you down and be endlessly frustrating. It will basically 2-10x the time it takes to do anything, and sap the joy out of it. I almost quit my job of 12 years because the org moved to databricks. I got permission to use better, faster, cheaper, less clunky, open-source tooling, so I stayed.
My stack atm is neovim, python/R, an EC2 and postgres (sometimes Sql Server). Some use of arrow and duckdb. For queries on less than few hundred GB this stack does great. Fast, familiar, the ec2 is running 24/7 so it's there when I need it and can easily schedule overnight jobs, and no time wasted waiting for it to boot.
You mentioned earlier about how long it would take to acquire a new cluster in Databricks, but you are comparing it here to something that's always on here. In a much larger environment, your setup is not really practical to have a lot of people collaborating.
Note that Databricks SQL Serverless these days can be provisioned in a few seconds.
> you are comparing it here to something that's always on
That's the point. Our org was told databricks would solve problems we just didn't have. Serverful has some wonderful advantages: simplicity, (ironically) cheaper (than something running just 3-4 hours a day but which costs 10x), familiarity, reliability. Serverless also has advantages, but only if it runs smoothly, doesn't take an eternity to boot, isn't prohibitively expensive, and has little friction before using it - databricks meets 0/4 of those critera, with the additional downside of restrictive SQL due to spark backend, adding unnecessary refactoring/complexity to queries.
> your setup is not really practical to have a lot of people collaborating
Hard disagree. Our methods are simple and time-tested. We use git to share code (100x improvement on databricks' version of git). We share data in a few ways, the most common are by creating a table in a database or in S3. It doesn't have to be a whole lot more complicated.
I totally understand if Databricks doesn't fit your use cases.
But you are doing a disingenuous comparison here because one can keep a "serverful" cluster up without shutting it down, and in that case, you'd never need to wait for anything to boot up. If you shut down your EC2 instances, it will also take time to boot up. Alternatively, you can use the (relatively new) serverless offering from them that gets you compute resources in seconds.
To ensure I'm not speaking incorrectly (as I was going from memory), I grep'ed my several years' of databricks notes. Oh boy.. the memories came flooding back!
We had 8 data engineers onboarding the org to databricks, it was only after 2 solid years before they got to working on serverless (it was because users complained of user unfriendliness of 'nodes', and managers of cost). But then, there were problems. A common pattern through my grep of slack convos is "I'm having this esoteric error where X doesn't work on serverless databricks, can you help".. a bunch of back and forth (sometimes over days) and screenshots followed by "oh, unfortunately, serverless doesn't support X".
Another interesting note is someone compared serverless databricks to bigquery, and bigquery was 3x faster without the databricks-specific cruft (all bigquery needs is an authenticated user and a sql query).
Databricks isn't useless. It's just a swiss army knife that doesn't do anything well, except sales, and may improve the workflows for the least advanced data analysts/scientists at the expense of everyone else.
This matches my experiences as well. Databricks is great if 1. your data is actually big (processing 10s/100s of terabytes daily), and 2. you don't care about money.
they are competitors and are similar. Snowflake popularized the cloud datawarehouse concept (after aws fumbled it big with Redshift). DB is the hot new tool.
BigQuery ELT, the org I went to was rather immature in their data practice, and I sold them on getting some proper orchestration (Dataform, their preference over DBT, and Airflow), and keeping the architecture coherent.
I'd have rather stuck with Spark just because I prefer Scala or Python to SQL (and that comes with e.g. being far easier to unit test), but life happened and that ecosystem was getting disrupted anyway.
If it's still like the RoR I used, it can never be considered "boring", just old.
It was a framework where code you dump in "conventional" locations is autoloaded everywhere.
With DSLs based on interpreting the method name as an expression, reflecting on them in `method_missing` implementations you get from inheriting classes.
Where state is shared between instance objects by way of reflection.
Where source diving is the documentation in many third party packages.
No, these were reasons why "rockstar" became a pejorative in the programming community for a while.
Your impassionate argument in favor of an ethnic cleansing with bombs, starvation and lack of sanitation lies in stark contrast to this outrage about the Uyghurs' culture being erased with development and draconian imprisonment for suspected extremism, while at the same time, the West Bank was reduced to a patchwork of ghettos with draconian checkpoints where life is cheap if you are suspected to be an extremist.
On the one hand, the slaughter of teenagers today in Gaza is justified because of the sins of their forefathers in electing Hamas, on the other, it's a genocidal atrocity that beards can only be so long in Xinjiang.
The US is only more righteous for supporting America's Greatest Ally at the cost of the lives of it citizens, of those of Arab countries and, now with the TikTok ban, to its founding principles, in your world.
I think you're raising an important point that I acknowledge, and I need to process that more. But my immediate thought is to note that the two situations are very different to me. Uyghurs are indigenous to Xinjiang (more than anyone else), and Islamic Arabs in Gaza or the West Bank are not indigenous to the region around Israel (Jewish people are, while Gaza's residents are outsiders since Islamic Arabs aren't indigenous to the region). The CCP/PRC is erasing Uyghur life in Xinjiang and forcing their population growth downward, while Islamic Arabs living in Israel have full protection of the law, can practice their religion and culture freely, gain wealth, have families without forced sterilization or abortion, and live without the threat of the state kidnapping their children. There's no evidence of Uyghurs broadly supporting terrorist attacks, while there's evidence of Gaza residents overwhelmingly supporting things like repeated rocket attacks.
There's more but basically there are enough differences that the situations can be treated differently. But at the same time, I do hope for a different, peaceful solution. Surveys show Islamic Arabs are generally quite happy under Israel, but that won't work for those in West Bank or Gaza perhaps. You are right that teenagers who aren't supportive of terrorism don't deserve violence. A two state solution still seems like the best idea, but Hamas has still not shown support for it, even though it would mean peaceful coexistence and could lead to a rebuilding of Gaza and West Bank.
What you call "Islamic Arabs" makes no sense. They are as native as the Jewish people. Mostly because they're nor more or less "Arab" than the Jewish people. Religion is what separates them, not race. Also, your argument ignores time. Even it it were true, for how many centuries have these "Islamic Arabs" been there? Why are they any less "from there" than Jewish people?
And there is an overwhelming support for the war in Gaza from Israelis, that many consider a gross violation of international law going beyond war crimes into genocide territory. If that is shown to be the case would Israelis deserve what they get in response? I doubt you'd agree.
Every genocide, every occupation and every conflict has its own historical conflict. You need to examine you own massive biases here, adjust your view to international law and human rights and you'd see the massive injustice here that is fueling the conflict and will continue to until its addressed.
Israel is not securing its own safe future here, there is no military solution here. Hamas will rebuild itself if it hasn't already because the crime they have committed and the people they have murdered will only boost the recruitment of Hamas and other paramilitary groups. This has always been the way the world over. Jewish history is itself has examples of this.
And I've never see a good faith attempt from Israel wrt this conflict that hasn't been completely derailed by the Israeli extremists and hardliners.
LLM companies aren't being funded by the hundreds of billions because investors expect science to be advanced by text and image generators.
I find it very unlikely that the commodification of knowledge work will be for the betterment humanity, I don't know if people are expecting here that just because the value of more people's labor becomes zero that we will do, what, do away with money? No, it will just mean that fewer people will have the chance to earn the right to use space and resources in a meaningful way.
There's always the basic drives that his mental illness place in his actions. If nothing else, attention, validation and social status are plenty enough for someone of his condition, that of narcissism.
Data collection is a worry of the previous decade, the recommendation algorithm is the battle ground, the US has decided that it much prefers having Meta push its white supremacist and gender wars drivel non stop than what was being shown in TikTok (Israeli atrocities)
While we are at it we should show the Hamas atrocities while we are at it. Would not be complete, without someone of the fake heart pull videos that Hamas traditionally put on as well.
They must be planning on milking what value can be had from the web to which they used to be the entryway, and clash with OpenAI, MS and Apple over AI trained on curated datasets, to layer some semblance of a business model over it. And I say milking because the relationship to websites is now parasitic for the most part.
More likely it will look like the current welfare schemes of many countries, now add mass boredom leading to unrest.
Sam Altman has expressed a preference for paying people in vouchers for using his chatbots to kill time: https://basicincomecanada.org/openais-sam-altman-has-a-new-i...
reply