What’s so special about Palantir? They seem to trade on this mystique of doing “secret government work” (spoiler alert most secret government work isn’t all that exiting). From what I’ve seen it’s fairly routing analytics and ML with a lot of humans deployed to deal with the usual messiness of any dataset. What am I missing?
Palantir was created by the US intelligence community to be their vendor. Palantir's first VC investment (outside of Thiel's self fund) was literally the CIA, through its In-Q-Tel VC firm.
Their moat is that they are trusted to do whatever you pay them to do and to not bite the hand that feeds them. And that if you're the kind of engineer to organize against a Project Maven, you really don't want to work for Peter Thiel.
In-Q-tel also had a hand in Facebook's rise. I remember being laughed at in about 2010 and called a conspiracy theorist for suggesting big tech was on intelligence agencies' leash, but it's pretty clearly true. Society is being shaped from the shadows.
Edit: I overstated the first sentence. It wasn't In-Q-tel, just someone who might have been sympathetic.
Always; yet at the same time, because of those shadows, almost all specific claims of some specific face being spotted in those shadows isn't correct.
Thiel personally, and the name "Palantir", gives off Bond Villain vibes (yes I know it's LOTR). How much of that is the actual threat, vs. an unmarked shipping crate owned on paper by "Dave's Shrimp Inc."?
The unmarked crate can blow up and destroy individual people.
A secretive governing process can overstep, undermining democracy or subverting constitutional rights.
If the crate blows up, generally the public will know about it.
If important calls are made without public oversight, then the public won’t know about them. This might deprive the electorate of a chance to shape those processes democratically or to go through the courts, where their rights could be defended or where they could be compensated for harms.
Fortunately, I can’t think of any examples of when secretive organizational processes have harbored abusive dynamics. Not even in more mundane organizations with far less authority, like the Boy Scouts of America. Public officials and their reports can always be trusted to do the right thing, so public oversight is never necessary.
I think we're on the same page on what's bad with secretive stuff.
Given that the opponents of your government can use the same information as the general public to determine the blind spots of your intelligence agencies, I'm not sure what to do about that, though I am hopeful that we're rapidly approaching the point where any motivated teenager can sucessfully spy on a government and at that point there's much less benefit for agencies to be secretive.
I hope.
But all that aside, my point was that conspiracy theories are often wrong even when there's an actual conspiracy in the same general area.
That was a good bit, but no, it was based on reporting about the connection mentioned in the link above, an investor having served on a board with CEO of In-Q-tel.
Almost every notable technology has it's roots in intelligence or defense, because government grants for projects as such has been the dominant way tech has been built. Corporations have their use, but predominantly, they're corner cutters[1]. Their function is to chase higher profits by reducing costs. This is not a skillset with no value; it's why you can now get an 85" television for less than the cost of a mid-range laptop; but it does mean that their capacity for innovation is inherently limited.
The way this always worked was the government would sponsor research into new tech, get it and use it, and then the patents and licenses would be disseminated into the private sector and be re-packaged as products. Think of the first iPhone; the LCD screen was a result of DARPA projects looking to reduce the beefy CRT displays to something more usable in vehicles and aircraft; the multi-touch screen was developed by CERN; the GPS system was originally conceived to track deployed materiel in warzones, etc. That doesn't mean Apple did no work of course, combining all of these things is no trivial matter. I'm just saying that most tech has it's roots in defense/intel grants that fund the research, because building new technology is fucking expensive and most corporations won't do it because it's terrible ROI.
Edit: I remembered the note I wanted to add for corner cutters:
[1]: The primary exception being a corporation that does not yet need to make profits, which is where you find investor-funded startups and incubator arrangements like Google had during their early time.
I think while lots of corps do business with the government and love the money, Palantir is next level and seems to be aimed completely at serving the government and especially intelligence communities as the go to, which means that if there is a swamp, there are in the most gassy part of the swamp, we are talking fartville, microbial smorgasbord indigestion version. That’s not a great way to have a healthy democracy; let the government do government stuff but mixing in too deep with corporations is another step to towards fascism and corruption.
That's pretty much what it is. Most of the humans are just doing ETL type work, and that's what you're paying for. The software itself isn't all that remarkable.
Palantir's entire MO is to create a digital twin of the socioeconomic sphere of existence. This is what exists between the lines when they claim such success in finding ISIS operatives / sleeper cells.
