This is just that 'reality has a surprising amount of detail' phenomenon all over again. Big tech simply isn't willing to engage with the detail, and increasingly expects the world to conform to its expectations. That said, this refusal is not new; my parents worked for a bank that was trying to adopt some IBM technologies in the 1980s, and they said that IBM couldn't accommodate the bank's requirements.
Using standardisation to improve productivity has been the backbone of the industrial revolution ever since we first standardised screw threads in the 19th century.
It's undeniable that we've lost some cultural richness in the process, but if that means I don't have to work a field for 12 hours a day just to get enough food to survive like my ancestors did, then I'll accept the tradeoff.
> I don't have to work a field for 12 hours a day just to get enough food to survive like my ancestors did
Not disagreeing with your overall point but this isn't correct. Medieval peasants worked about 150 days a year, and while they were obviously very busy during planting and harvesting times, they had lighter workloads at other points in the year.
These studies generally don't include the large amounts of domestic labor that had to be done in addition to farming. Lots of food preservation, home repairs, tending to fires, etc that we simply don't have to worry about today in nearly the same way. There was a lot less work in winter, but there was also substantially less food and with less energy for recreational activity, a lot of people's leisure time was occupied with sleeping.
Medieval peasants were required to work 150 days a year on their lord’s estate. That was how they paid for the land they had to farm the rest of the time to survive. They also had to spin, weave, and make clothes.
It really depends on a lot of factors, including time and place. The lot of a "medieval peasant" was not the same across time (1000+ years) and place (all of Europe) and varied a ton. Your link is a response to an article by an economist that notes periods of high wages in the fourteenth century in England, that's it. There were certainly times when peasants had much less labor to perform on their lord's lands, or might have been free peasants and have none, or lived in an area with no demesne-like system, etc.
Which just makes the whole thing of looking at the life of peasants and talking about how much free time they had compared to us all the sillier. The point about what else you need to do to live (cook, mend clothes, etc.) is important, but makes it even more difficult to compare to our modern lives, even variation aside.
I've read a lot on this subject, and unfortunately I think that all sources I've read are introducing a particular bias, which is usually either "capitalism makes people work more!" or "y'all are crazy, people worked way more before the Industrial Revolution". For example, here is a counter reference to your reference: https://groups.csail.mit.edu/mac/users/rauch/worktime/hours_....
My own personal take is that, first off, I'm not at all trying to make peasant life as some sort of panacea - far from it. If anything, life before the Industrial Revolution was certainly much more precarious: obviously tons of childhood deaths (not to mention deaths from things like smallpox and the plague), and all it took was a bad harvest to cause major famine. War was much more commonplace. But the lack of a lot of tools that the Industrial Revolution provided meant that, in many ways, there was simply less possible time to do productive work. When the lights went out, it's hard to do a lot by candlelight/torches. The work really did ebb and flow with the seasons. Yes, there was ton to do during the offseason, but I think the analogy with the modern world is that is like the time after a tough deadline and big release - lots of "OK, let clean up some of this code now that we have time".
Plus, a lot of the tasks that your reference mentioned would have been clearly divided by the sexes. There is a paragraph from your link that states:
> We might also point to the amount of household labour that had to be performed. Yarn had to be spun, cloth to be weaved. Cooking was over open fires: and that firewood had to be collected. Bread baked and so on and on. There was a recent report (rather exagerrated but still) which claimed that in the 1930s it took 65 hours of human labour a week to run a household. Today it takes 3. Things were worse back in medieval days.
And nearly all of that work would have been done by women while men worked the fields. There was certainly much more total work to be done to run a household, but the division of labor was more clear cut.
So again, I think the reality is probably somewhere in the middle. Life was certainly "harder" back then, but from everything I've read there was undeniably more "downtime", even if there wasn't a ton to do during that downtime. The "having to work 12 hours a day just to keep food on the table" that I was responding to clearly seems to be false from just about every source I've read.
I think there was a article here which stated that 1 person had to work for 1 month to get 1 pound of sugar or something. Today it is equivalent to 1 hours of work, something like that. You are narrowing your focus too much. The abundance is really the key here.
Then in the description it states "The French Livre, sou, and denier are equivalent to the pound, shilling and penny (Latin liber, solidus, and denarius)"
Meaning:
pound = livre = liber = L
shilling = sou = solidus = s
pence = denier = denarius = d
So redoing the calculations, a dozen eggs were a quarter day of laborer wages, and more than a full day of wages for a maidservant or swineherd. Still considerably more expensive than the price of eggs today.
And they were just farming. It had straight forward payoffs, rewards and risks (stress profile). Flash forward to where I have to constantly reassess my risk exposure in a corporate setting on repeat, daily. It almost makes communal farming based on biblical dogma and superstition appealing.
Society would be healthier if we retuned to seasonal labor (even in intellectual fields) and had agreed-upon travel seasons. Why must we all chain ourselves to the same geography year-round?
This is poignant in the age of LLMs. What cultural richness are we going to lose as a result? What societal cohesion are we going to lose? What wealth inequality are we going to create?
There's actually something interesting here in my opinion, which is that LLMs do not necessarily need to hinder standardization.
For example, we have standardized schooling/exams, because that's the proven scalable way we can have for children - essentially a factory spitting out different grade levels/seniority.
But LLMs can break this standardization by being able to tailor student needs in a scalable manner.
However this takes a huge amount of action, and that's going to be the pain point in the near future as we humans tend towards the easy/greedy paths.
Cultural richness - LLMs are also very bad in this regard due to the bias towards majority viewpoints. This reminds me of a recent HN thread [0] discussing how AI is hindering the adoption of new technologies. In some ways, this happens because AI tends to favor mainstream perspectives, making it biased against "new" or "fringe" viewpoints.
We will only lose our culture if powerful people exert dominance. That's how cultures were withered throughout history, a dominant violent culture comes and pushes its wicked pursuit. It takes decades or centuries to recover your culture after it happens. Shout out to Native Americans, keep trucking, you'll get it all back (truly sorry about all this).
People talk about the glory of Rome, but it was utterly vile. They either enslaved or exterminated whoever they conquered. If they had an LLM, they might keep you alive and brainwash you with it. Maybe.
Per gpt:
"In the military of ancient Rome, decimation (from Latin decimatio 'removal of a tenth') was a form of military discipline in which every tenth man in a group was executed by members of his cohort."
This was done to thwart desertion.
This is similar to:
- PIP.
- Email me what you did last week or your fired.
- Blanket 10% layoffs in high profit environments
- Blanket 10% cuts in general
I'm less scared about AI than the culture of dominance utilizing it. We are not the only people that ever lived on planet fucking earth, everything has been done already in some other form of matter. The wicked animals you see now days in power have lived many times over in past histories, we've seen these exact people many times. We simply get hoodwinked when they seduce us with casual talk about space, science, and the world. Decent people begin to think they are just like us. Nope, they are not like us.
Sometimes you just have to connect the dots and accept it. History precedes us and owns us.
That period was when all the lords had captured the land and forced people to toil on the land, without access to large land ranges the average person cannot live independently
Before your ancestors would have kept a large herd of mammals and lived nomadically moving to fresh pastures when the grass and vegetation was eaten
It's rare but there are some modern nomadic people who live like this camels and goats in the desert
Old diets were primarily carnivore and plants were struggle foods
That's why Europeans killed all the buffalo and why the Japanese on average are shorter because they had long periods where meat was restricted for religious reasons
Don't compare yourself to a people under exploitation, independence was taken by force
I have family that still live as subsistence farmers on their own land. Their life is hard and unforgiving. There’s a reason they wanted their kids to get an education and move to the city.
I’m not going to try to compare it to pre-neolithic nomadic tribes because nobody knows for sure why the neolithic revolution occurred, but my guess would be that climate shocks were more likely to wipe out small nomadic tribes over the long run.
Sometimes bespoke subtlety is bad or superfluous and sometimes its innovative. Ideally, you streamline a golden path as best you can but its also important to leave room for new good ideas.
I saw someone saying that it is as though tech read Seeing Like a State and took the wrong lesson
I think tech does have the drive to make things legible, and is falling into the same trap as described in the book where efficiencies or processes that cannot be described in the format desired at the top leads to them being discarded. And the legibility issues mean that the impact of discarding these types of things is not properly understood
> That said, this refusal is not new; my parents worked for a bank that was trying to adopt some IBM technologies in the 1980s, and they said that IBM couldn't accommodate the bank's requirements.
That misunderstanding is something that causes a lot of grief in SAP introductions.
When working with large enterprise software, customization is your enemy - and every time you have to customize something there, you should ask yourself if you shouldn't re-think your business processes instead. Often enough IBM, SAP or whatever have considerably more experience than you.
Ironically at my company, our custom software made us too flexible. There was too many crazy left field demands that weren't really that useful.
So when it came time to think about next steps. There was real appeal in being able to say, "No it's not supported in xyz software we just adopted". This prevents us from looking like the bad guy who's just getting in the way and should be laid off because we didn't want to spend 2 months implementing a hair brained idea that would only give us a net return of like four or five thousand dollars.
And that customization kills you over time. Heck, a big reason a lot of banks and other big enterprises eventually move over to one of the big vendors is because they've built up a hodgepodge of now unmaintainable crap that nobody can touch.
This is also why Salesforce was so successful and why it killed on-prem enterprise deployments in a lot of places. No longer a ton of different essentially unique versions to manage that make it nearly impossible to upgrade. Salesforce obviously supports customization but in a much more controlled fashion than was common at the time, and it's why it won out over Seibel Systems in the early 00s.
1: It locks you into a vendor. The vendor strongly encourages customisation for this reason (and for other reasons that are mostly against a business' interests)
2: onboarding staff is expensive because they need to learn your unique systems. If you're using a package/service off the shelf you can often advertise for people with knowledge of the package/service e.g. AWS versus inhouse e.g. known payroll system versus custom.
3: internal costs to upgrade. With stock standard systems your upgrades are cheap because the costs are amortised across many companies e.g. mobile app development in 2010's. With custom systems you pay for everything. The costs are often invisible e.g. horrific Mobile support (watching friends fight desktop websites on mobile is painful). Another cost is that internal development teams are often shit and certainly can never compete with Darwinian best-available-on-market software.
4: shitty custom internal systems. Sometimes the customisation provides business benefits but we have all worked with crappy internal systems where the benefits versus costs were slim.
Two examples of bad customisation requests:
A) HR systems in small companies. For some reason HR managers demand fractally complex systems and they all want different things so developing common features is really hard.
B) larger customers with demanding internal rules/regulations e.g. a client asking us to conform our UI to their UI standard. No way it was worth it for us or for them. Plus dealing with their UI despots would have been hell. I think their internal software teams decided to build inhouse instead (I bet the outcome was shit).
We got one of our largest and definitely most complex customers not long ago. Being rather small, we pushed hard against any customization that we didn't feel was necessary due to business demands, and asked them to follow the way we'd successfully used with our many other customers.
After the went live and the dust had settled, they thanked us and said they were glad we had pushed back. It had forced them to rethink how they worked but the result was much better and more optimized processes.
> every time you have to customize something there, you should ask yourself if you shouldn't re-think your business processes instead
I've heard this as "the best flavor is vanilla". It referred to a hospital aligning healthcare business processes with industry-standard software workflows.
And this has been true since states wrought the cadastral map, last names, the metric system and taxes on us.
The main issue today is that the techologies shaping reality are in the control of private, non-democratic institutions, selling this power to the highest bidder (including hostile foreign powers).
I had a fleeting hope for a while that the sheer complexity of reality would thwart these forces, a la https://tvtropes.org/pmwiki/pmwiki.php/Main/DystopiaIsHard -- but I'm increasingly worried that the explosion in performance of machine learning plus the fact that much of the world's population has willingly handed over their data in service of convenience will just make it a cakewalk. What is the modern digital equivalent to the War Machine (e.g. Deleuze) that can fight this? is it possible? Am I overreacting?
> This is just that 'reality has a surprising amount of detail' phenomenon all over again
Ever since that was posted[0] a week ago (reposted, but it was the first time I saw it), I've been filtering almost everything I see or do through that lens. It's really eye opening.
It's the complexity of reality sure - but that's not the only lesson.
Another lesson is that users can be taught to use appropriate protocols if those protocols are specified and the users believe in the mission/purpose of the site/organization/association.
>users can put in whatever tags they want. (Autocomplete is there to help, but they don't have to use it.) Then behind the scenes, human volunteers look up any new tags that no one else has used before and match them with any applicable existing tags, a process known as tag wrangling.
This is what booru sites have done for ages. Danbooru is the only booru mentioned in the article (and just as a passing mention despite being a few years older than Archive of our Own). I can only guess that AO3 is a bit more palatable than boorus for a general audience.
But this style of tagging really is the best of both worlds. The only downside is, as mentioned, the requirement for manual labor. I implement a similar system myself, obviously at a way smaller scale, for my own pictures. Basically I tag things as I feel like it at the time, and then every few months I "wrangle" the tags, as they do here.
It's mildly amusing that AO3 is considered more palatable than booru websites. Both AO3 and many of the boorus have content that would be hard for most folks to stomach. I guess AO3 has the advantage of being mostly text, (which seems to be generally less visceral to people,) but it is also nearly completely uncensored, since that's kind of the point of it.
Yes, overall I wouldn't describe either of them as particularly palatable to a general audience, haha. Your hunch is the same as mine -- text has much less potential for instant shock. On AO3 you have to, sort of, go out of your way (by reading, or searching for specific tags, etc.) to hit the unsavory stuff.
Even if you did find the unsavoury stuff on Ao3 it might take a while to get to it in the story whereas in an image you could be blasted with the sin of more than a thousand words all at once
A lifetime ago, on Stack Overflow, I was one of those people that monitored tags. I nuked needless new tags as often as I could, replacing them on each post with more correct existing tags. I ended up with a set of huge search bookmarks with regularly recurring bad tags to help keep things under control. Burninating bad tags became a delightful passtime. But eventually I burnt out on SO as a whole and dropped out. That was a decade ago now. I recently rediscovered one of the search bookmarks and lemme tell you things got really bad in my absence. The New Tag Deletionist Cabal is no more.
> Wrangling means that you don't need to know whether the most popular tag for your new fanfic featuring Sherlock Holmes and John Watson is Johnlock or Sherwatson or John/Sherlock or Sherlock/John or Holmes/Watson or anything else.
"Featuring". Yes Wired that is a technically accurate description.
They could have tried Steam, but I don't think it works as well. Although they did start with the pure freeform one and switched to a managed one the next day.
Exactly that, it's labor intensive, while tech tries to reduce labor intensivity by using magic or rules or whatnot. Related and larger scale projects are sites like Wikipedia or TVTropes, although Wikipedia makes use of a lot of automation as well.
You can totally use machine learning to classify stuff, that's one of the main things we've been trying to do with it, after all! But that automatic stuff often is entirely without soul and without contextual understanding. Useful as a starting point, perhaps, but the human touch is still needed to organize information like a human wants.
The huge number of communities that have elected to move to Discord, an information black hole, seems like a strong counterargument to the generality of this claim.
This seems to be a generational issue. It makes me sad, but I suspect it's because doing things "in public" on the Internet has become increasingly exhausting and people have retreated to gated communities. It's not a black hole, it's a "dark forest": you can't find their information because they're scared of you.
It's not super new though, people had private forums in the phpbb era as well, which you could only read after login. Or think about IRC, private torrent trackers etc.
I dunno if there were that many successful private forums; locking off access to unregistered users back then was a really good way of ensuring no new users finding the place back then. Very very possible I just wasn't aware of private forums enough though.
Beyond that though, information was still neatly catalogued into small focused threads whereas Discord's biggest issue is that stuff just vanishes in a gigantic chain of messages.
Was a bit late for IRC's heyday but am I right in thinking it wasn't guaranteed that all users would have access to the chat history? That would create an impetus to log the information elsewhere.
Private trackers such as what.cd tended to attract people with strong archivist streaks which I think largely resolved the issue there. Oink could go down and you'd know that a huge chunk of its catalogue would appear on Waffles within a few months.
I ran a private business forum for online marketing. The locked door approach meant it developed a legendary reputation and meant many more people talked about it than they probably would have if it had been open. The mystery effect.
This also happens with some private torrent trackers; the status of having an account means people put more effort into maintaining the place too.
Online marketing feels like a field that would _need_ a private place to talk tbh; anything public would be a beacon for self-promotion efforts by people who are quite good (or at least relentless) at doing so.
I think the sort of Discord group being discussed here counts as "in public". You can join interest groups by just clicking a link with an anonymous account. It's out of view of Google, unindexable, hardish to search and to collect organized information from, but it is more public than say, old newspaper archives in microfilm that anyone can look at in a library.
People are going to discord for that because it is convenient as a platform for a variety of reasons, not because they want to hide it from the public imho.
OSINT communities use discord to communicate and discuss information from mostly twitter and telegram and organize and share the packaged information.
Information organization is hard and lots of work. Not everyone (or every community) has the same need to organize information, and not everyone has the ability or inclination to do so.
Information getting lost on discord is a problem, but that just means the useful stuff needs to be archived elsewhere, and frequently there is another distribution network involved.
Accurate Problem description, which can easily be countered with .. [extra effort expended in idealized behaviours] .. [I could paste Lore Ipsum here everybody stopped reading at the idealization]
People who care deeply tend to do things better than people who don't care. Large organizations only organize information to the point of short term marginal profitability. If being better organized beyond a point doesn't get you more money in the next few months/years, it isn't worth it to most businesses.
Likewise, being extremely organized personally is probably unnecessary, but to each their own.
IME it goes deeper than that: caring is ruthlessly stamped out by crushing bureaucratic processes. Results don't matter, what matters is never being responsible for a mistake. Fans can fix minor mistakes - in corporate America, even the smallest misjudgment is fatal. The only way to win is to never actually do anything, but keep up the appearance of being busy.
They really do not care much about it. Lets say you sell a product. You have had 3 revisions of that product. Support on that item is very minimal. The current version is what you sell and that is it. The previous versions are just historical interesting things. Properly archiving it and cataloging it takes time and money. Does talking about your older items sell you more items? Maybe maybe not. It is just a thing you sell. Not any sort of historical artifact to be preserved. It is just how you make a living scraping the margin.
But to a collector or 'fan'. All 3 hold importance to your collection. All of the details/metadata are mapped out so you know why v1 is worse/better/interesting from v2/v3.
What is worse in many of the 'fan' cases is you find it is usually 1-2 people mapping that stuff out. They map out what they find interesting. Many also get heaps of verbal and legal abuse from both the companies and other people on the net. So they bail out and whatever 'fan' site came out of it, rots. Like one project I found a few months ago. Tons of stuff mapped out but some errors here and there, no big deal. I have a set of patches ready to go to fix it. But the orig author has ghosted. They got tired of tons of abuse from other 'fans'. Frankly what I see in the previous requests I want nothing to do with it. So I keep my stuff private.
I released a complete and comprehensive software package for free around fifteen years ago. I was threatened, accosted and cussed out on the phone numerous times for my generosity. I have since taught myself to be greedy on purpose, obscuring and ignoring my true nature.
Archive Of Our Own is not just any group of fans of course, the very name of the website refers to them deciding to write a better version of fanfiction.net. The award reflects the main goal of the website, not something that happened by accident.
Reminds me of Google's old motto, "Organizing the World's information" (is it still?). Even at their best, Google was never really "organizing" as much as "making it searchable". e.g. YouTube - for a given band, why can't I browse a list of their past concerts (with dates and locations) and see all videos from each?
And people realized it was easier to just search, just as they are now realizing it is easier to just ask, and have an LLM synthesize all the search results.
The upsetting part isn't that fans are better at organising information, it's that companies and governments are so incredibly bad at it.
Limiting this to "tech" isn't really fair, because most other organisations isn't doing much better. Right now the entire world is trying to avoid collecting, creating and organising information by feeding it through AIs, which pretty much depend on organisation having done exactly that in advance.
There's a huge potential of business and organisations that will do the dirty work and focus on information creation, collection and organisation internally. Just think of customer service, when was the last time an FAQ or self service guide provided any value? It happens so rarely because business don't care to keep things updated or even spend money on good writing. Nope, better to invest in an AI chat bot than updating your website.
To me that sounds backwards. Who knows your business better than you and your staff? How would anyone from the outside be able to manage your internal information better?
I mean based on experience I don't think your wrong. I've run into so many businesses who feel like they need to hire consultants and bring in outside help and products to get back on track because they honestly don't know how their own business work and don't have the skills to fix it. So on the surface they are "doing what they do best", but they're missing the middle part so they not actually doing their best.
Customer service is a different set of skills. Anybody and everybody can learn customer service if they want to, and get rid off the middle men with some effort and some investments. But most companies simply don't want to invest in it and leave it to middle men instead.
Also, most business owners do not have a clue of what they're doing. They inherited the business, or caught a ride on low interests etc. That's why there's still fierce competition and newcomers eating their lunch.
Sure, it works for your anime fan site, but what about when money is involved, like in a search engine? That attracts bad actors, who can use their power to abuse your site.
Solvable with a combination of robustness and stochasticity. If you say need two randomly selected people to approve an edit, and you flag users who make too many attempts at making rejected edits (either one user, lots of bad edits or lots of users trying to make the same bad edit), the only way for a bad actor to reliably make undesirable edits would be to gain control of a very large number of potential approvers. More generally, if the cost to effectively manipulate the system is greater than the perceived reward from manipulating the system, bad actors aren't an issue.
I noticed the article didn't speculate on why, but I think you nailed it. This system is probably incompatible with a commercial site. It requires too many volunteers.
It's because fans actually care about the data! Corporations do not care about data. Corporations will never put in anything beyond minimum effort to get it right. Usually they won't go beyond a good guess.
Of course anyone other than "Tech" (Corporations) is better at literally anything than "Tech." "Tech" doesn't care about providing utility to it's users, only extracting as much profit from them as possible, which is almost universally in opposition to the interests of users.
People who build things because they care about them and want to maximize the utility they provide actually design systems that allow users to effectively and efficiently access information or complete tasks, "Tech" on the other hand designs systems full of anti-user dark patterns solely intended to force users to view as many ads as possible or buy whatever service "Tech" is selling.
Is there any effort to organize scientific literature like this? I know journals often generate tags for papers but those can often be quite poor and restricted to the field The journal is in. I would happily join a volunteer effort to create tags and do some tag-wrangling for scientific literature in my research area.
> At a time when we're trying to figure out how to make the internet livable for humans, without exploiting other humans in the process, AO3 (AO3, to its friends) offers something the rest of tech could learn from.
I mean it makes sense. The biggest problem with the internet today is that all the big tech companies want to automate as much as possible, and that necessarily means quality and organisation will take a massive hit. Google services, Facebook, Twitter, Instagram and other social media services, Amazon and other marketplaces, digital storefronts like Steam and the console download game shops... they're all terribly organised and flooded with garbage because no one's manually doing anything to fix it.
Meanwhile, you've got fanwork sites like this one which have better quality and organisation standards than any large company site, simply because someone gives a toss about whether things are easy to find and of high quality.
All of this is no longer true. A modern AI will still get tripped up on details sometimes, especially with recent fandoms, but it's not going to make mistakes about common fan fiction terms any more.
> Another of the Tag Wrangling Chairs, Qem, also thinks that machine tag wrangling is unlikely, pointing to machine translation as a cautionary tale. “There are terms in fandom which, while commonly understood in context among fans, would not be when you take it out of the fandom context," Qem says. For example, seemingly innocuous words like "slash" and "lemon" do not refer to a punctuation mark or a citrus fruit in fannish contexts, and tag wranglers are already well aware that machine translation can only manage the literal, not the subcultural meanings
The problem isn't just knowing the fan fiction terms, it's being able to alias the hundred different variations of those fan fiction terms correctly.
AI sucks at this. It will alias incorrectly, it will alias when it shouldn't, it will create a new tag when it should be aliased, etc.
There are several cases where the human reviewer needs the context of the story (or image, in case of boorus) to make the correct call on whether to create a new tag, alias a tag, recognize a misspelling of an existing tag, etc. AI is not a good fit (yet, at least).
