The problem is that the open internet is dying. What used to be blogs and forums now is Reddit. Tens of thousands (hundreds of thousands?) of communities that had their own wiki, blog, forum, ... are now reduced to be a Subreddit.
The Internet has become fragile. One service goes down and everybody suffers. If the top 10 services went down most people would think that there is no Internet at all.
E-Mail is the last standing service that is way more open that the rest. But the raise of Whatsapp and equivalents are challenging that. One day all our communication will depend on a monopoly. We are starting to know what would have happened if AT&T have never been split.
It's getting worse by the minute, too. What used to be a subreddit might now be a Discord server, which is in my opinion even worse, keeping solutions/questions to certain issues within Discord's own walls.
You now have to enter a server, search for your specific problem, and, if not found, ask someone and wait for them to respond. If results are on the first page of google it's much more friction-less and convenient imho.
It’s not just your opinion. Discord is a black hole for content. There won’t crawlers for the content in these spaces. I’d like to see more activity in the ActivityPub & modern, self-hosted forum/blog spaces (where Atom feeds can aggregate), but history likes to repeat itself as we move from one closed, proprietary communications service to the next because there’s ‘no cost’ to join.
It's even worse for only the metric by which the efforts of a group should be indexed for the masses. But people use discord for group conversation and the goal is not the create content for consumption but to do the things the group does as a semi-private organization. Discord seems like a massive win on some dimensions of privacy even with the drawback of not being able to index all the things. I agree discord is bad if you want to create content for universal consumption but no one I know is truly using discord for that.
Pithy retort: that must be a parallel dimension to suggest Discord is good for privacy.
Letting ideas float around a chatroom is good but communities hopefully have motivated members to consolidate and publish some of the good ideas for posterity because we will lose them over time. Some places do a better job, but a lot of content is straight up locked out from the wider internet where users are required to give away credentials to a private American company just to get access to it; at least private American Reddit I could read without an account with answers to folks web searching for a quick solution. But we’re tech-savvy folk and know better, but it seems from even the Stack Overflow survey results yesterday that our industry is also trending in the wrong direction.
There's different notions of privacy. Discord isn't going to protect you from someone deliberately stalking your posting history, but it is more private in the sense that it is, first and foremost, comprised of communities who want to manage who participates. A Discord server is the equivalent of a local pub with a specific audience.
And in that context not being crawlable is very much a good thing because communities require spaces that are to some extent fenced off, nobody holds a meeting in a train station. In an age were publicly scrapable information becomes subject to automated surveillance or vacuumed up by AI systems it's not surprising and justifiable that people move to walled gardens.
Not to mention that a lot of discord chats are conversational. And the default for casual conversations is that they're ephemeral, for good reasons. When you have a chat in the analog world, you don't expect that to be scanned by everyone for posterity.
Sure. It's not a problem that people who want a casual chat with friends or interest groups use a platform for casual chatting. I say that even though I prefer the earlier world where everything was indexed and people liked to overshare on Facebook.
The problem is when you use Discord (or Slack, or IRC - yes, IRC is just as bad in this context) to run a community that's outward-facing. For example, when a large OSS project, or a foundation, decides to use Discord as its main support/hanging out channel. Participating in such group requires so big an investment of time and attention that it instantly shuts off anyone who has a day job or other interests and responsibilities. Again, this is not about causal chats - your "offtopic" forum thread probably is better off being a Discord channel. The problem is when contributing to, or receiving support from, such community requires being active on a chat group.
Remember "The Tyranny of Structurelessness"[0]? That seminal article from 1970s that, for some reason, just doesn't want to die? It was posted here just three days ago[1]. It actually talks about this. What it calls "elites" that naturally grow as an informal hierarchy in a group without a strong formal hierarchy - Discord, et al. are pretty much designed to create those. The people who are able and willing to keep up with the chat flow, following it all day every day, are the ones that become those "elite", and end up running the community.
>Participating in such group requires so big an investment of time and attention that it instantly shuts off anyone who has a day job or other interests and responsibilities.
depending on the kind of OS project, that may be more of a boon than a hinderance. There is always good talent put off by some factors in life, but there are always a lot more actors (some bad, some simply ignorant) that may make the project more of a strain than a collaborative effort.
----
I'll also emphasize that unless you are a part of an enormous hub without proper structure, the "chat flow" isn't as heavy as you'd expect at first blush. Its much more annoying to log on and read the day's messages, but the number of relevant messages may be in the dozens that take 10 minutes to skim, not thousands that takes a part time job to parse. But even then it makes sense; for those super large OS repos, there are in fact people doing it as part or full time jobs
Take a look at Zephyr RTOS, for instance. It's all behind a fucking discord. It's a pain in the ass compared to something like Nordic Semiconductor's Devzone. The fact that I have to join a tech support channel for my job with the same profile I use to join 18+ furry roleplay servers (discord requires a phone number that can receive texts and I don't have a work phone) is a sign that something here is a problem.
well yes, that's always an issue with popular social media. That's why I make sure my 18+ furry roleplay account is different from the professional one.
> I'll also emphasize that unless you are a part of an enormous hub without proper structure, the "chat flow" isn't as heavy as you'd expect at first blush.
For values of "enormous" starting at "a dozen people".
Elsewhere in the thread I wrote about my Hackerspace experience. At the point I started drifting away from the community, we had ~100 people active at least every now and then, some ~30 present and at least occasionally active on IRC (later Telegram), and 5 people generating 1000+ messages between 09:00 and 17:00 on every regular working day. By volume, 50% could be casual shitposting, 30% interesting technical conversations, and 20% something relevant to the community - but 100% of it was a bonding experience.
No surprise that the people generating most of the chat backlog volume were also seen as the "core" / most involved participants.
1000 messages in a work day sounds like a lot even for a decently popular game discord. That sounds like it goes beyond "active participants" and more like "eternally online" people.
You already made your choice, but there is something worth considering on if you even want to be a part of a group (even if you had all that free time). I find it interesting that this post came up later after I came back to this comment:
>motivated members to consolidate and publish some of the good ideas for posterity
I think there were some sensitive documents dumped from Discord (where noone noticed them for a good while) to the public internet. So I'd say there is some truth to it being semi-private. The "Discord Leaks".
There are a few open-source projects that mirror the Discord help channels to a external website so that it is easily searchable. More people should do this in my opinion. Discord is extremely convenient and it is always very active, so naturally people will be drawn to it, but we should push for more alternatives like using matrix with a bridge to discord and indexers that mirror content to be more open and transparent. The problem is that its expensive and not everyone is capable of doing it easily, or they dont necessarily care or know why that's a good thing.
Exactly! There is a market for something that more closely replicates conversations in a bar. Sometimes you want to be a blow-hard and say some crazy shit without it being indexed for all eternity.
I mean this was basically clubhouse which people loved until it grew large enough that they had to build out moderation tooling besides "kick this motherfucker out of the bar" (like automatically enforcing "if anyone talking in this bar personally blocked you, you're not allowed to enter the bar. Fine for a local dive, disastrous for larger rooms, imagine being banned from a conference because one person blocked you - they didn't even have to want to ban you from the conference, you're just auto-kicked from any room they enter !)
Except it’s often used in a way that goes beyond chitchat and bouncing ideas. I was pretty active in a programming community a few years ago, and then they decided to move official chat exclusively Discord which I disagree with on principle & was not going to create an account. Some (not all) decisions I would have previously expected to see on the forum got discussed and decided within Discord’s proprietary walls which cut some users out that previously were a part of the discussion.
It often happens unintentionally, and it poisons communities. Back when I was actively participating, and for a time even running our local Hackerspace, I insisted on a hard rule: our IRC channel is non-binding, anything of any relevance must be also discussed on the mailing group, and only decisions made on the mailing group are binding. This rule was keeping the community from going nova - shedding outer layers of occasional participants, and collapsing into a core group of friends that's in constant contact.
Even with that rule in place, when I graduated and had to focus more on my work, I realized I'm slowly becoming an outsider to my own community, simply because I'm not able to keep up with the torrent of random conversations on IRC. Back then, there were no LLMs available to summarize the 1000+ messages people sent between 09:00 and 17:00. And even if they were, it wouldn't have helped, because the problem was, people who wrote those messages were bonding together, becoming closer-knit every day, while I was becoming more distant just by missing out on this.
It's what eventually made me stop participating - and from what I talked to other people in and around our Hackerspace at the time, I'm not the only one. Which is why I say, chat groups are, by very nature, poisonous to communities - including off-line ones. They need careful managing.
It already is indexed, though. Any member of a server can run a search over the messages. Granted, it's not a particularly great search, but it's there.
Content created for universal consumption was the dream of the early internet. It powered several generations intellectually forward at light speed (or backward in some circles).
Segmentation, paywalls clickbait and SEO are, unfortunately, the Monday morning reality we’re waking up to.
When I was researching some DVD menu stuff a couple of years ago I was able to read through long dead forums from 2002-2005 via google and get almost all of the information I needed from there.
When I was trying to resolve an issue with my hacked PS3's BluRay drive a few months ago I was able to deduce the solution from a fairly terribly maintained wiki from 2012.
When I was trying to reverse engineer a board game app from 2018 a few months ago I was barely able to find any information because the official Discord banned such discussions and the unofficial Discord didn't seem to exist anymore.
What is the appeal of discord? I've tried to multiple times and to me it's just ephemeral... something closer to irc then a web forum. The onboarding and UX is horrid (or I'm just an old dude... which is true :)
The appeal is for small private and semi-public communities. You are right that it is more like IRC than a web forum, but it serves much the same purpose for many people with its server channels and channel threads.
I'm a member of a small community about 25-30 people that know it each from either university or work. We chat about games, politics, software development, organising IRL events, and lots of other interests. The advantage of discord over IRC is we get better message persistence, video streaming and voice calls, sharing media (pictures and video). It lets us organise this content in a way that say a group whatsapp could not. Right now three people are in a voice and video chat room talking about and playing Street Fighter 6. I could just jump right into that conversation with a single click and might do so in a moment. They are also sharing videos and twitter links in a channel dedicated to fighting games.
I'm also a member of a larger programming community with a few thousand members and we have automated help systems allocating space to get help, evaluating programming expressions to aide conversations, moderation tools, url shorteners for programming playground - much of which is powered by custom bots using discord apis.
The thing it is closest to is MS Teams or Slack, but the difference being that its more focused on soical interactions and gaming intergrations than it intergrations for work tools. Its also just frankly a better product than Slack or Teams - it just doesn't have the business intergrations that make it viable for business and doesn't offer business plans.
I host a Mattermost server for this purpose. It does what Discord does, just shittier, buggier, fewer features, and I have to pay a web host to have somewhere to run it. But it's -actually- private and our community owns the data, instead of whoever buys Discord next week.
I wish there were more self-hosting offerings to compete with Discord. RocketChat, Mattermost, Zulip, and Matrix/Element all leave something to be desired.
Discord is a proprietor-centric subject-specific usually-public chat room with a medium-length lifespan. Longer life than a chat window, shorter lifespan than a 2008 web forum. Where else do you get that?
Thinking Discord is "appealing" is not the right way to look at it's popularity.
The reality is more organic, the younger generation was exposed to discord through gaming and Twitch streaming and learned a natural workflow to create a chat community around themselves that tie-in to their other social media.
I like Discord because it solved one of my longstanding complaints with forums: the perpetual forum-dweller who is an otherwise shut-in person who monopolizes the discussion and likely has other flaws that precludes them from being a voice of authority on a subject anywhere but in their little forum space.
Discord is much more "you must be here participating to have a voice" and if you leave, someone else becomes the authoritative voice. More democratic, more humble, less belligerent.
> Discord is much more "you must be here participating to have a voice" and if you leave, someone else becomes the authoritative voice. More democratic, more humble, less belligerent.
And shutting out everyone who has a job or other responsibilities that prevent them from keeping up with the discussion flow 12+ hours a day.
I've never heard of "perpetual forum dwellers", but Discord (and Slack, and yes, IRC too) is a place that's structurally makes a community be run by close-knit group of people with too much free time.
oh I certainly have. We call them moderators often, but there are definitely power users who you will see comment on almost every post in a particular group.
And since we're on the topic of Reddit here: well, you will definitely see the abyss stare back if you look into how many subs the largest moderators run.
What's the point of chat rooms? Well... you see in the beginning there was only grunts and aggressive body language (kinda like how emojis are used today,) and then well, you know what just read the wiki https://en.wikipedia.org/wiki/Human_communication
I use Discord to talk privately with my friends. Sometimes I get answers there on specific communities faster, but I agree that the non-searchability is horrendous.
This, at least subreddits are relatively stable, public and searchable. There is so much knowledge hidden inside huge chat logs of semi-public Discord servers which might disappear at any minute and is hard to retrieve, unless that server maintains a wiki or something similiar.
So then someone should make a basic product/feature that let's you "save" (encapsulate, and then push) a conversation to a public page.
Let's say I run a Discord community around some product/service, like a rendering engine like Octane. Two users have a meaningful conversation explaining how to increase light samples. Let's say that information is not documented online. There should be a nice way to summarize + post that onto a website that can be crawled.
You could even automate this with GPT. It could constantly scan Discord for back-and-forth context-specific conversations and push summaries from them to a website. Then users could upvote or downvote them into oblivion asynchronously.
>someone should make a basic product/feature that let's you "save" (encapsulate, and then push) a conversation to a public page.
it's not impossible. And ultimately many discord groups will try to have channels dedicated to keeping a list of guides (usually some google sheets maintained stuff) that are slightly more publicly visible. But it still comes down to the will of the people to maintain that and make is accessible.
Main barrier with your approach is that I'm not sure how scrapable discord is. It (to my knowledge) never had an API that did more than let you automate posting and maybe scan recent comments to reply to.
I absolutely despise Discord. I understand that it’s essentially a chat room for communities where content is ephemeral, but so many communities use it slightly differently. For example lots of communities use it to provide help to other users but then complain when users ask the same questions over and over. (See Python discord)
What I like about Reddit or any forum is the ability to search for a topic or question you need help with and then see people discussing it and resolving it. Forums are a great Archive for discussions like this.
Where discord plays a good role is in fleeting chat. Water cooler type chat. Discord is fun if you’re heavily involved in a community, you can spend many hours a day in your favourite discord but for someone like me who just likes to pop in a now and again, it’s borderline useless.
I suppose what I’m getting to is that, for me, discord is for low effort throwaway chat, Reddit or HN for that matter is better for discussions that should be read again by interested minds.
I've seen some IRC channels archive their chat as search-able HTML pages that are also indexed by search engines. Wish Discord servers would do something similar.
Discord knows better, this is by design. Their investors would run screaming if they knew what was in those chat logs. The lack of searchability is their biggest defense against being hauled in front of Congress because the public just isn't aware of how much of the platform is extremism.
That's too bad, because a lot of open-source projects have moved to Discord, and it's an absolute pain that every time I want to search, I have to 1. login and 2. join yet another Discord server.
I find it easy to mute and put the ones I don't frequent often in a folder and only go there when I need to. I only actively visit and contribute to 2 Discord servers but that number might grow if the reddit communities stay dark (and I don't blame them for doing so).
I would take Discord over Slack or Gitter any day of the week. Slack only retains 90ish days of text chat which makes it an awful platform for open-source projects to use. It hurts my brain that CNCF and Kubernetes use this as a platform. At least in Discord I can search thru years of content and discover a discussion instead of asking the same questions over and over.
> I would take Discord over Slack or Gitter any day of the week.
Gitter had terrible search, but one advantage Gitter had over Slack and Discord is that it required no login to lurk, and chatrooms were indexable by search engines. You can still do so with Gitter's switch to Element, although Element leaves a lot to be desired on the UX front.
> I would take Discord over Slack or Gitter any day of the week. Slack only retains 90ish days of text chat which makes it an awful platform for open-source projects to use.
But who is to say that one day Discord won't start enacting limits on search history? The enshittification seems inevitable, in my opinion.
>But who is to say that one day Discord won't start enacting limits on search history? The enshittification seems inevitable, in my opinion.
If it's inevitable, the service doesn't matter. YCombinator can one day do to HN what Reddit did to itself, but I'll still enjoy the community while it lasts.
It's better to work more on a "good enough for now but keep backups in mind" state rather than "there is nothing perfect so I won't go anywhere" state. If your mindset is the latter, why value online communitie at all to begin with?
What I mean is that it's inevitable for a closed-source, proprietary solution like Discord. The service does matter, and there are better alternatives to Discord for online OSS communities.
Sure, just like how there were better alternatives for Facebook, and Twitter. Even when they were at their peaks. I think we've seen often enough in history that the best, most technically impressive tool is rarely the most popular one.
If you have the power to move the people you can help try to fight that aspect of society. But at large we can't even lead the horses to those theoretical oases (let alone make them drink)
I think you're missing the context of the thread. An open-source project doesn't need to choose a tool that competes with Discord in popularity -- popularity is not important here. The aim is not to attract millions of users, the aim is to provide a valuable and sustainable resource to its community that it can link in the project's README file. A tool like Discourse or Zulip is more sustainable than Discord because they are open source, history is exportable, and the software is trivially self-hostable in case the managed hosting solutions don't pan out.
It's different in that IRC is designed & understood to be short lived real time chat. Who cares if I can't find that type of content via Google? Discord is being used as a permanent info repository, like a replacement for discussion boards, wikis, etc. This is the type of info I want to search for.
LLMs perform lossy compression on knowledge, so they're not a good replacement for indexing and search. If anything, you want to have the latter, and expose it to LLM as a tool it can use, regardless of whether the LLM was trained/fine-tuned on that same content or not.
> It's getting worse by the minute, too. What used to be a subreddit might now be a Discord server, which is in my opinion even worse, keeping solutions/questions to certain issues within Discord's own walls.
To be honest, for the good. Discord search in "discord forums" sucks so bad it's almost unusable. Let's keep using Discord for instant messaging and voice.
Still open to any alternatives. But as we saw from the thread earlier in the week, none of the answers seemed to have achieved that network effect yet.
No real point in an indexable community if the community has no knowledge.
i keep seeing this sentiment about things moving to discord here, but my experience is i dont see that happening anywhere. i suspect there is some kind of experience bubble you are in. i really never hear about people using discord except for games.
Email is dying too. I have run my own email server for 20 years. A few years ago my original ISP was acquired, forcing me to switch providers and hence get a new IP address. Ever since then my server has been blacklisted by one major player or another as a source of spam. I even tried changing to a new IP address, but the result of that was just a different set of providers blacklisting me. I'm about ready to throw in the towel and switch to Fastmail. :-(
I had this exact experience, I had an email server running on the same ISP for 15 years, got great delivery. I got a new ISP, blacklisted nearly everywhere. IP reputation is super hard and time consuming to build.
So I made the switch to Fastmail, and honestly, it's the best descision I made. Migrating was super simple, and they've been rock-solid for me. I didn't realize how much time I was wasting administrating an email server with one user until I switched.
I spend pretty much zero time administering my server. Once it's set up, it pretty much just runs itself.
The reason I want to run my own server is that I want to run my own spam filter, whose algorithm requires seeing my outgoing mail as well as my incoming. (Though I guess I could do that through IMAP by looking at my SENT folder. But for me it's also just the principle of the thing. I remember the good old days, and I'm not going down without a fight.)
I would strongly advise anyone considering hosting their own email these days to do literally anything else. You'll be more fulfilled by abandoning email entirely and writing letters with a fountain pen, if you're looking for a new hobby. It's not a matter of if a private server will get blackholed anymore, it's when.
If you're planning to offload to a third party anyway, why don't you instead first try routing mail though AWS SES? You'll definitely get better delivery and at least you'll be able to control the rest of the stack.
(I actually already have a Fastmail account, and I use it to deliver mail to domains where my server is blacklisted. But even that little concession feels like defeat.)
Not sure about better, just different since it's strictly an MTA. But perhaps you can also use Fastmail as an MTA only? Biggest benefit I can think of is with SES, Amazon would never see any of your incoming mail. Well, unless you replied to something. Then I suppose they would have access to it indirectly.
> perhaps you can also use Fastmail as an MTA only
Yes, and that's what I actually do to solve my current problem. But one of the reasons I run my own email server is to maintain my claim to fourth-amendment protections in light of Smith v. Maryland so I really don't want my mail being routed through a third party.
there are also small companies which deliver the same service, if that appeals. i've been using mailroute.net for delivery and spam prevention for a small business for years.
In the end, what you need if you want really reliable self hosting is to own your own ASN and IP range, so you can tightly control the quality. Review how clean the addresses are before purchasing them and continuously review it for the duration you own them, as some been used for spam previously.
Unfortunately, you’re probably right. I’ve been self-hosting (email, web and DNS) for the past 10 years. So far, the only issue I’ve had was GMail rate-limiting my server when I had to email about the 30 members of a local voluntary group as part of secretarial duties. However, it made me aware that the delivery of emails sent by my server is more akin to a “privilege” than a “right” on the modern Internet.
By the way, I just upvoted a reply you previously made to another comment of mine: https://news.ycombinator.com/item?id=36032588 Thanks for providing that example of how homoglyphs can be used to “fingerprint” plain text. I also discovered that with Vim digraphs, I can easily input spaces that are look the same as regular space characters.
It's quite expensive, yes. But removes a lot of headaches (while introducing some new ones), is a great learning exercise and is most probably a solid investment when it comes to ipv4 at least, prices keep going up and it won't just disappear in the near future.
This is the way I think personally. I recently changed my mail server host, had to unblock the IP from MS and all that but since, it has been smooth sailing... for now.
Email is shrinking, for sure, but there are still a lot of options and providers and if you're hard core you can run your own if you're willing to pay for a clean IP address.
There's lots of smaller VPS and VM providers -- even local IT companies. Mythic Beasts and Bytemark have been good for me.
If you want to use Linode for your CPU time and disk, probably there are providers for outbound SMTP that you pay a small amount for to use as a smarthost.
I still use a couple "old school" forums, but I have to say - the reddit tree format is just so, so much easier. It is impossible to de-rail a thread, and there's a certain level of anti-spam and self-moderation mechanism to the reddit format. The older forums still need some hardcore moderation, or else many threads become unreadable. Nothing worse than when two users start arguing, and the thread turns into shit.
But that's just the how things have evolved. It's so much easier to just start a subreddit, than to host your own forum.
citation needed, I've seen many reddit threads degenerate into name-calling, logical fallacies, political posturing, and what have you. If they don't, my guess is it's a very focused subreddit that doesn't attract much general interest, or the mods are very on top of things.
But 95%% of attention will go to the "top" branches which are usually memes and "dumbass marathons" were people are trying to keep the original joke going.
Compared to how most of the attention in a traditional forum will go to people trying to respond to the TC/OP? Even in cases where 4-5 pages in the TC already resolved the issue?
The only benefit for a forum is that it "bumps" a topic when you participate in it, so you can keep a topic alive. But that isn't too difficult for a threaded style forum to replicate. That just wasn't how Reddit wanted topics to work (nor HN, in our case). Reddit never even implemented a way to subscribe to a topic despite a 3rd party extension having a feature for over a decade.
When you have to collapse a dozen worthless discussion trees to find one that is actually discussing the topic instead of spamming lame reddit narwhal memes, I think it's fair to call the discussion derailed.
Point is, linear boards like phpBB can't even support threads this large - without the tree, a thread goes to shit the moment someone posts a lame meme.
Also most old forums I know were ran by some guy on their spare hardware that could break at any moment or be shut down on their whim, as most eventually got. Reddit is at least bulletproof in that sense, even if the mods go full regard they still can't destroy any actual data and it can all be reverted by admins.
Nobody wants to host anything, because it costs money. Even reddit no longer wants to be the internet's free data dumping ground. Lemmy and Mastodon won't succeed because of this fundamental problem. Everyone hosting their own server? Literally the opposite of what 99.999% of people want.
So what's your solution? forums just die as a concept because no one wants to host text?
TBH I think reddit's hugest shot in the foot was that they DID want to host everything. For a decade the site was fine pointing to youtube videos and Imgur albums, but at some point they decided to bring all of that into the site.
> even if the mods go full regard they still can't destroy any actual data and it can all be reverted by admins.
it can, yes. But will they bother? What kind of sacred knowledge is really kept from some random post on r/pics that was made 5 years ago that got 100k points, especially when it is reposted every few months anyway?
I think what's happening with Twitter and Reddit is actually a good thing in that it's exposing the failures of centralized networks. The thing is, these failures aren't big enough yet for people to jump ship and I think the main problem is none of the decentralized social networks have provided a solution that's better.
Mastodon is theoretically a better version of Twitter, but normal people get lost the moment someone starts talking about "instances". It's this gap we need to cross somehow. How do networks provide all or most of the pros of being centralized without being centralized?
Maybe the solution isn't federation, but a nonprofit foundation with an open source and relatively simpler codebase like Wikipedia. Wikipedia is the 800# gorilla in that space, but there's no ads and no perverse incentives and they aren't trying to lock down to prevent "theft" from AI companies, and the decentralization is through people being able to relatively easily setup specialized wikis.
I'm not sure the problems of federation are ever going to be solved at massive scale. Usenet and IRC were always problematic back in the 90s. Not even e-mail works well due to spam.
social networks are a disaster. Many news websites used to have their own dedicated forum space, but they all died down over the decade, or moved to Discord/Reddit. It's just that big a headache.
Hosting user generated content is very costly. If not in server hosting, then in the emotional toll of moderating that forum and making sure it's not spammed to all hell, or worse. Bad apples ruin the barrel, as usual.
It is stunning to me that Mastadon still hasn't come up with a simple onboarding solution to sidestep the issue of "instances"
I completely agree with you. Mastadon even loses techies like me due to the decision fatigue of having to pick instances before I can even start exploring.
What’s shocking is everyone can wrap their heads around regular country post offices, emails, in general distinct entities interacting, but lose their minds when you tell them about mastodon.
Just tell them it’s email for twitter. Just like you can interact with anyone who’s email address you know so too with their mastodon handle. The only real difference is that everyone also publishes all their “emails” for anyone to see instead of sending them privately.
This also displays where the federation that mastodon uses, has difficulty.
If a rural post office closes (they do from time to time), unless you're using a very old "rural route" style postal address ( https://pe.usps.com/text/pub28/28c2_021.htm ), your address doesn't change when a post office changes.
Additionally, if you happen to live next to a person with unsavory views, the post office will still deliver all your mail to you.
When a mastodon instance shuts down unexpectedly (it also happens from time to time), you need to start up a new identity somewhere else and your address changes.
If your neighbor on an instance becomes problematic you may find that your messages aren't visible to others anymore as other instances start defederating from the instance that you are on.
For situations where people say "well host your own...", for most of the populace, this is similar in practicality as staffing your own post office.
Oh please. The consolidation has affected email equally to the point where "email" is synonymous to "gmail". No one is signing up for email by searching for other email clients, setting up SMTP and whatnot. They go to Google, search for "email" click on the sign up link to "gmail" and carry on. I hardly believe even more than 1% of email users even know about its federated nature
Unless Mastodon has similarly sensible, no frills defaults, it is doomed to the fringes. Normal users are not going to have the patience to look at a bunch of instances and make a choice that they really don't understand. Make a large instance for everyone and make it the default
> I hardly believe even more than 1% of email users even know about its federated nature
You're completely ignoring the corporate world, which may outweigh consumer email use (dunno, but it wouldn't surprise me.)
G Suite isn't even close to a monopoly, and I imagine there's large IT departments at anti-cloud corpos specifically for managing all their emails.
It's like, consumers don't know or care if they're using Linux, but they would definitely care if everything using it went away. Consumers aren't where Linux "won" either, it won on servers.
All this to say email is more centralized than it should be, but its decentralization is absolutely still important for its continued relevance.
If you take away the decentralization and Gmail were to become its own private messaging service, it would have the momentum to carry on for a while. But it would surely die. 5 years, maybe 10. But I have no doubt it would suffocate itself.
I know lawyers and founders who have Hotmail and AOL.com addresses, so I'd jump ship day one. One mass "Hey guys Gmail is bad here's my new email" to all my contacts (or as much as spam prevention allows...) and done.
I didn't say 1% of all email users are not gmail users. I said 1% know about its federated nature. Even if email allows for different email domains and there are entire departments are dedicated to manage domain emails, I thoroughly doubt even these people would even understand that this is what makes email "federated" or even call it that
gmail isn't a monopoly because of how they handle email. gmail is a monopoly because the common user doesn't know the difference most of the time
I mean, they may not know, but they'd understand a high level overview.
I on gmail can email someone at hotmail, outlook, yahoo, or any custom business email. I can do this because email is a common protocol that is shared amongst dozens of servers. That isn't deep technical knowledge, but they would get the gist of federation there if I segway'd to it.
>gmail isn't a monopoly because of how they handle email. gmail is a monopoly because the common user doesn't know the difference most of the time
Even my Grandma has a Yahoo account. She knows my Gmail address and can send me messages. I don't think Gmail is as synonymous with email as you think.
also as mentioned above: some 80% of people with a business email address probably knowws that there are more addresses than "@gmail.com".
You’re missing the point. Just like most people have no idea how their post office functions, really. And most people have no idea how their email service functions except at the most basic level, they understand enough to use it. For most nontechnical people I know, they’re more likely to be using yahoo or aol vs gmail.
You are missing MY point. If you put front and center how your social media is "federated" and then add friction in the signup process by asking them to "pick a server" then it's DOA for anyone except most tech users
From what I've seen, they've significantly changed their signup process to push unauthenticated users to a default feed and to create an account by default on their largest instance which is a massive win for getting new accounts. This is a good change
What I can't wrap my head around is trying to read a conversation with some bits silently missing because some parts haven't (yet) synced to my server. It makes it impossible to use Mastodon for anything serious, as there's too great a risk of talking at cross purposes due to missing context. (Or, simply repeating something someone else has already said.)
[false quote narrative removed, I had names mixed up]
Not in the slightest. You see, you can use mastodon.social for exactly as long as it suits you to do so.
If you're the type who will simply never wrap your head around federation, stay there. Sorted.
If you figure it out eventually, you can take all of your content and migrate it to whatever new instance you like. No hassle, no corporation making it difficult because they are trying to keep you coralled in their silo. The flagship instance would absolutely love to help you move off their servers while keeping every bit of your data.
I dunno, the experiment is in motion. The hope is always that people will open their minds and learn the concepts. But I feel just as lost watching a sports event as a lot of folks apparently feel picking an instance, so I get it.
Edit: the thing is, even the totally incurious might run up against a rule they don't like, or they might notice that every post that is relevant to them ends with @sexy.gazebos and get curious what that means. I am 100% positive that that curiosity plus a 100IQ is enough to get the idea.
I have to ask, though: if 99% of users do stay on mastodon.social (I'm still there, for instance) or whatever ends up being the main Lemmy instance, is that actually a problem in some way, which is not a much bigger problem on Reddit?
I am curious, why don't you like centralization? It seems like no matter the crowd, people need centralization. Atleast to begin with, then they are ok breaking off into subgroups. Like what happened with reddit. You start on the popular frontpage. Then the longer you stay, the more subreddits you start using.
historically speaking, I don't like putting all my eggs in one basket. We've seen sites rise and fall, some fall from grace and others outright shut down. And often times, no one is prepared enough to move. In many cases they may not even care about the lost knowledge and move on in life. Sometimes I do care tho and I spend way too much of my time trying to find any data hoarders who may have also cared. This is a huge hassle.
So in theory, a decentralized network should solve this. If one part goes down, you host another instance, migrate, and have that new instance rise, with minimal impact to your community since all of this is happening under the surface. They may change their URL, but they will still feel like they are on their favorite site, hopefully with most of the people intact.
>Like what happened with reddit
Yes, a great example of why I don't like centralization, given current times. While I support the blackout, if there was some important bookmark to a piece of content that I needed, I may not be able to access it. If reddit was federated, I coudl simply clone those important posts to a new instance ahead of the blackout and keep on keeping on. Maybe have some way to redirect the URLs as well (I'm not too versed in how configurable these instances are).
It's one of those battles you can't win: Ask people to choose up front, new users balk. Assign a default, the existing crowd absolutely riots about centralization, and half of everyone doesn't even notice the change anyways.
The difference there is, they're wrong about it being a centralizing move, because the user retains the right to move on at any time now or in the future. It's just a matter of continuing the conversation until it sinks in, I had to do it with a very forward-thinking friend who I only learned in that conversation, had no clue about FOSS stuff at all. I really did have it in my head that everyone who got on the internet also figured out how the internet works just by being here, but of course not, they've been using platforms that protect them from the real internet...
They lost a techie like me when their main web app refuses to even display without JS. And I say that as a JS guy. Open stuff with aims as big as Mastodon should work on as many devices as possible.
I agree that decentralization would be great for social media because anything successful would have to solve some interesting problems that would push social media forward as a tool.
Reddit, Twitter, Facebook, email etc all have fairly complex vocabularies for curating your experience:
These concepts caught on because they were presented in a gradual, simplistic way, even though they represent the workings of a complicated organism.
Basically what I'm saying is that Federation may be one more layer of complexity, but the vocabulary is already present for users- they've had a pretty long tutorial on social media. 99% of social media sites are aware that user growth requires removing friction and that includes typically dumping people into a "default" area, then allowing them to customize their experience. I think this kind of funnel would be necessary for federation, even though a "default experience" is kind of the antithesis of federation.
It's a good question. Regardless of Twitter, Kbin and Lemmy (and maybe others?) will be even more important for ensuring they get indexed properly. The organic/real nature of Reddit comments (at least those not astroturfed .. heh) were valuable to search. We (devs) need to make sure Fediverse stuff provides that value. At least in my view.
Is there something Mastodon/Kbin/Lemmy might be doing that could cause poor indexing? I'm working on yet-another-lemmy-thing and i'd like to not commit sins on this front.
I only know Mastodon: There's a large LGBTQ+ community who don't want to be indexed out of concern for their safety (because, for example, there are small far-right communities on other instances looking to cause mayhem). The ability to be oneself online without worrying about getting brigaded, doxxed, or whatever, is a selling point. Indexing runs counter to that.
Agreed, but is it a choice or a requirement from the software? I love choice, that instances (and users!) can and should get to choose the level of indexing. If it's software wide, not sure i can get behind that. Especially since .. doesn't ActivityPub sort of ruin that? Ie any instance you federate with could expose your data, regardless of Mastodon's rules.
Federated instance data visibility and agreed visibility rules of course come into play. But if i understand this right it would mean you can only federate with instances or software that agrees to your privacy rules. Which is totally viable, but that also then means Mastodon can't federate with, for example, a Reddit clone that wants to be indexed.
It's a complex web of requirements. Definitely something i'll be thinking on as i implement my own ActivityPub instance.
I don't know enough/anything about ActivityPub, but maybe different...apps? (Mastodon et al vs kbin/lemmy) could have different agreed-on rules. The forum-style ones would be appropriate in search results, so they could be indexed. People linking to Mastodon posts on lemmy might be unavoidable I guess, but there's at least a bit more separation.
One thing i do hope though is that if instances that want to hide content, or users hide content, that the instance also publishes that data during syncing. Which is to say, i want to write a custom instance but i don't want to mistakenly leak something someone wanted to be private.
In theory it's a well understood problem-space in the Mastodon stack, so hopefully they only federate/publish what they are okay with being public. But it's concerning regardless.
Pretty sure Mastodon doesn't allow indexing because that can be used to brigade users and they're trying very hard to actively avoid creating the dogpiles that twitter almost actively incentivizes.
Google absolutely does crawl quite a bit of Mastodon, but the community is largely against crawling and indexing so in many cases Google may be blocked.
the open internet is dying. What used to be blogs and forums now is Reddit.
How "open" were those blogs and forums, though in a functional sense?
Instead of an untrustworthy/unreliable corporation like Twitter or Reddit owning a data and community.... we used to have a zillion untrustworthy/unreliable individual "webmasters" owning those blogs and forums. "Your" "community" might be gone in the blink of an eye because the guy running it threw a hissy fit, or forgot to pay his hosting bill.
All other things being equal, the current near-monoculture does suck worse. But the old days sort of sucked too.
> we used to have a zillion untrustworthy/unreliable individual "webmasters" owning those blogs
True enough, but the chance of many blog/forum owners all having a hissy fit at the same time - the equivalent of what has happened to Twitter with Musk or Reddit with Huffman - was infinitesimal. Also, because there were many, competition worked. Don't like one forum? Go to another, often on the same topic, and losing all the friends you knew on other forums was never even a possibility. Thanks to network effects, that hasn't been the case with the every-topic megasites. Returning to a more decentralized model won't solve everything, and will even (re)introduce a few other problems, but it's still more resilient especially with regard to crazy CEOs.
Valid point, but only partially true. It was remarkably easy to simply take a database dump and clone the server. Or pay the bill again. Or have multiple billing admins.
There’s a very good number of forums still alive, proving they have some resilience to show for themselves. It’s also a form of distributed resilience, i.e. one forum going down isn’t as bad as all of Reddit.
It was remarkably easy to simply take a database dump
and clone the server
Huh? I'm not following. That helps the admin(s) and nobody else.
The other 99.99% of users are in functionally the same predicament as Reddit users: they have zero control over the data.
Theoretically a forum admin could make those dumps publicly available, I guess? I'm sure that has happened once or twice, but I've never heard of anybody doing that.
I don't know if anything else was possible, the IT need for communities is about the same and it makes total sense to have those on one place. It solves the infrastructure problem, it solves discoverability problem, it solves the user account management problem and it is all "free".
It comes at the cost of fragility, true, but as always convenience is the king.
Maybe the solution isn't decentralization, but a non-profit approach like Wikipedia.
Reddit already functions entirely on volunteer labor anyways, adding venture capital requirements makes much less sense than a just-keep-the-lights-on non-profit model.
It may be a better way to go for a public forum like Reddit. Don't kid yourself though into thinking the non-profit foundation won't also be rife with petty politics, hidden agendas, and the usual crap. There will still be issues, and how to handle NSFW content is still going to be a big conflict.
| Don't kid yourself though into thinking the non-profit foundation
Those aren't the problems I was expecting to solve. Those are other problems that exist regardless of funding model, because they're fundamentally social problems.
A non-profit funding model simply solves the problem of venture capital undermining the core product in search of profits.
>A non-profit funding model simply solves the problem of venture capital undermining the core product in search of profits.
It also introduces a larger problem of shutting down entirely if they can't pay for server costs. That's always going to be a bigger problem than VC's.
Wikipedia only does so because it is bankrolled by universities and other institutions. And we should note that Wikipedia doesn't have to host videos, and has strict requirements on how to host images. I can't think of those places doing the same for Reddit.
Reddit worked for years without hosting its own videos and even images. For many years, it was hosting nothing but structured text and links to external resources (which was its original raison d'être, in fact; discussion about external resources).
HN chose to be pure text and not embed external resources, but this is just an editorial choice, it wouldn't cost more resources to add them.
| shutting down entirely if they can't pay for server costs
Keeping the lights on is never a guarantee for anything. VC funding can also run out. Being for-profit locks an organization out of crowd-sourcing, institutional grants and other funding sources that non-profits have available.
Very interesting point. It'd be a challenge to execute, but I'd be glad to see a non-profit that lets online communities discuss topics of interest, solve problems, and expand knowledge. Maybe something like that exists and I just don't realize it.
Sorry to make this about AI, but it'd also be interesting whether such a non-profit makes its data fully open--i.e., for AI companies to scoop up--or has more restrictive terms that forbid AI "scooping" without a separate agreement. Lots of tricky issues, trade-offs, and interesting incentives involved. There could be alliances with orgs creating open source models, for example. If anyone is working on a nonprofit like this or just wants to chat about it, please reach out.
The AI aspect is interesting, it could solve the funding issue. AI scooping requires a (paying) licence. Or, the paying licence would be restricted to closed-source AI models, and it would be free for open-sourcec models, thus making them much more competitive.
Codicat is doing that for Stack Exchange. I don't know anyone doing anything similar for Reddit. There are apparently some Reddit clones on the Fediverse, but I have my own dislike of the Fediverse model.
Few users of reddit communities are dedicated enough to that community to be willing to use it via an app that doesn't aggregate their interests into one doom-scrollable UI, much less donate to a site for hosting costs to keep it afloat.
99% of phpBB installations were pwned and distributing malware. It’s a terrible model. If we want small distributed operation of software like that we need a revolution in software operability and quality.
It's not hard to to write significantly more robust software than what you get on a late '90s LAMP stack. What killed those forums was the constant upkeep needed to keep them secure. Most of that is historical baggage though. The software emerged in a time when the threat landscape was completely different, and things like data portability wasn't even a thing.
The problem that remains is spam mitigation. Bots are very hard to detect, and there really aren't any good distributed approaches afaik.
That's one of the reasons centralized model works so well nowadays, nobody wants to take care of things anymore, self hosting your own apps is becoming a small niche
> ... nobody wants to take care of things anymore ...
I mean, why would you? It is quite a bit of technical hassle, even for an experienced admin. There's quite a bit of cognitive load just deciding what software stack to use, even before you run an installer. And then you have to maintain it too!
Compare that to less than 3 minutes of sign-up and creation of a new subreddit. It is an orders of magnitude difference in effort and knowledge required.
You're talking about late 90's-early 00's tech there (basically, LAMP architecture). It wouldn't be that hard to develop an open source, generic hierarchical forum engine that would be easier to deploy than an Apache+PHP+MySQL suite from scratch (on a dedicated Linux server that you have to completely configure all by yourself), and that would work both on a web browser and through a smartphone app.
Security would be less an issue than back then, too. We learned a lot since early PHP and early Javascript's bad practices.
I'm lucky enough that my main hobby's primary discussion era is an old-school phpBB, and it's likely not going to migrate to Reddit or Facebook or somewhere else. It's great. Content is fully indexed by search engines and can be easily searched using those engines. Decades of FUD from cloud providers and centralized services have convinced so many that hosting your own server is difficult and expensive and scary and just not something normal people do. Online communities really need to get back to self-sufficiency.
I flip between Flask and Django depending on what I'm doing.
If I want to make something "from scratch" I do it with Flask, if I've got an existing database (or something that can be wrangled into a database) I do it in Django.
Ah yes, signing up to a dozen different forums. Don't want to go back to that, either.
Tbh we've kind of failed to unify software properly.
Hardware is semi unified, Apple's efforts besides, but things like Wifi/Ethernet TCP/IP, HDMI, USB C etc all tie things together decently well now (but it could still be better).
But when it comes to software...why don't we have a universal identity/login system that everyone & everything supports?
Why don't governments enforce/mandate hardware/software compatibility for (larger?) companies. It should be like "if you want to support your own custom login/identity you can but you must also support minimal x"
Wifi for example, as a standard isn't legally enforced, it's purely supported for business reasons. If Apple could get away with rolling their own that only works with iDevices I'm sure they would. But if the usage purely for economical reasons disappears, we need law to force companies to use a minimal standard for interoperability.
> signing up to a dozen different forums.Don't want to go back to that, either.
That will cost you 3 minutes if you want to ask a question, if not you don't have to sign up. in exchange we will preserve data and make it easy to search.
and not make single entity to rule every community.
A big part is that a lot of the internet isn't available to google anymore.
Lots of good forums just have a robots.txt with "Disallow: /". Google can be a pain. People use DMCA in flagrantly illegal ways, etc, so lots of places just opt out of being easily found.
Then, as others have said, there are the discord etc, or irc style, that have never been very searchable, but have lots of good content.
It’s also a clash between the profit motive of these platform runners / investors and the increasing value of data itself (in the “age of AI”).
Enthusiasts need our help as Usenet as is, phpBB, etc just don’t cut it anymore in terms of ease of use for admins and exploration and fun for users.
Todos then:
1. Centralized solution it ain’t
2. Remove (boundless, blunt) profit motive out of equation (don’t depend on this at least)
3. Actually care and be professional about UX. UX quality for centralized services has been decreasing over the last ~10 years (I blame Apple for continuously dropping their own ball there; role models are important) so it’s actually not as high a barrier as it used to be (relatively speaking).
The problem is that the open internet is dying. What used to be blogs and forums now is Reddit. Tens of thousands (hundreds of thousands?) of communities that had their own wiki, blog, forum, ... are now reduced to be a Subreddit.
Some people argue that blogs are the answer to a more centralized or impersonal web. I disagree. Reddit answers and comments are better than blogs, which are full of ads, SEO-spam, signup forms, and generic and vague copywritten filler . Reddit comments are written by actual humans who are either looking for answers, or are supplying incisive answers or explanations, not filler. And most importantly, based on unbiased, personal, first-hand experiences. Many bloggers are paid to promote products and are not fully honest, or do not use the products they endorse. If you want to know if interment fasting works, instead of reading a vague blog about the benefits of fasting or having to sign up to a newsletter, why not read actual people's experiences with it.
One might quibble about blogs being Reddit when Tumblr and Medium and Substack exist, or forums being Reddit when Facebook groups exist, but that would be missing the point. Whether the number is one or a dozen, or even several dozen, it's not like the thousands upon thousands of independent sites that used to exist. While openness and distributed ownership/control are not the same thing, it seems like it's hard for the former to survive without the latter.
That's why so many people are looking to the fediverse and its imitators to replace the falling monoliths. There are problems there as well, both technical and social, but at least there's hope that they can grow into solutions without being assassinated by a single rich psycho. All it takes is for people to stop thought-experimenting on yet another rich-guy-owned monolith and start actually hacking toward a permanent and public solution.
New users aren't going to spend the same amount of time and brainpower managing their internet presence as OGs do/have. So they will choose the services that enable them to reduce the overhead of using the internet. For many, this doesn't just mean using Reddit for their interest-based communities, but using Gmail for spam prevention, Apple products for their curated store, Steam for ease of access, etc.
This would only be solvable if you had government-forced interop so that e.g. Twitter posts were accessible on Mastodon timelines with no cost to the site operators, but even then any small barrier is something for a platform to optimize and sell in the form of a subscription/using their user base for ads.
Here is an idea: forums are easy to build. What if people got back into them and just hosted them on their domain and then there was a webring attached to all forums like sites were back in the 90s, except there would now be a main site that would serve as a directory of forums divided by interest. so everything would be owned separately: the directory and each forum, but all connected through a webring like it was before. It would "decentralize" the Web in some ways (except for there being a directory).
And what’s worse, the contents of subreddits are incredibly homogeneous; there is no place for dissent. A subreddit named ”music” or ”films” or ”games” (just conceptually, ok?) fancies itself as the reference for that specific thing, when it’s actually just a reflection of whatever political stance the mods have (usually simply reduction to average). Outside people will see that and think ”oh nice a centralized discussion for concept X” and that just makes things even worse.
And the public upvote / downvote system make it even worse. Soon, you're reduced to a small subset of opinion because any slightly dissenting opinion will be downvoted to hell and hidden at the very bottom of the page.
Not necessarily an issue for small communities focused on a non-controversial topic (mostly anything about jokes/memes, or very niche technical stuff), but bigger communities focusing on slightly political topics become hell. The "europe" subreddit is incredibly toxic for this reason. Not because of individual redditors, but because of the group effect.
HN doesn't suffer from this problem both because of the niche aspect and because downvotes are restricted.
Yes. The mob mentality is an issue on reddit and has made me refrain from giving my honest opinion on a lot of topics because of the fear of being downvoted to hell. I think an upvote or like only system like Facebook or twitter would be much better.
reddit isn't doing so well either. The locking of posts was a terrible idea. The old.reddit vs reddit shows many like the old style. Hate the app.
All the information on reddit is user generated yet the reddit owners seems to think they own it. My view is the "frontpage" doesn't own the story just because they printed it.
Back in the 1990s I feel users were much more cable and there was so many more pages or forums especially as you noted forums.
The trend that I hate the most is blogs without comments. I don’t know when it started, but a few years ago I suddenly noticed most blogs started to turn off their comments section one by one. I don’t know why they’d do that. Interacting with the author or other people was half the fun. Now there’s just silence. It’s what made me stop visiting blogs altogether. I used to browse a bunch of the same blogs everyday and now I don’t. I don’t even remember most of them.
Even when googling, I usually just stick to the sites that allow interaction like reddit, stack over flow or this site.
A few weeks ago I remembered the names of a couple of blogs I used to browse and it’s sad to see them become spam and/or low-content farms.
It isn't the 1990s anymore. The vast majority of comments on blogs will be spam, bots or, in the extreme rare case of an actual human reader, raving lunatics. The chance of an actual human being reading your blog and leaving a sincere comment in good faith is so low that you can safely assume it will never happen.
I know spam is a ver big deal, but that wasn’t my experience. I wasn’t around in the 90s so I don’t know what it used to be like. But just a little over half a decade ago the blogs I used to visit still had people interacting with each other and it was common for posts to have meaningful conversations. I miss that a lot.
The cause seems to be software from private companies gets more resources for features, feedback, maintenance etc.
If that is correct, the obvious solution is generating some substantive funding for important open source projects which develop software and protocols for billions of users.
Of course, there are organizations already doing funding for open source projects, but maybe moving the scale of funding by a few orders of magnitude can help.
Software development is easier than ever, CPU power and bandwidth are quite cheap, and there are working federated structures like Mastodon.
There's no reason why Reddit couldn't be turned into a federated system as well, in fact it almost is one as it is. Eg, just like on Mastodon servers can choose whether to interact with each other, on Reddit it's not uncommon to get auto-banned from somewhere if you post in a subreddit they dislike.
Yeah but also no? The skills required to be a basic "full stack" developer have multiplied to the point where I think that term is on the verge of obsolesence just as "webmaster" was before it.
There is of course nothing stopping anybody from simply writing PHP like it's 2005 even though it's 2023, but going against the currents has its own set of costs.
> just like on Mastodon servers can choose whether to interact with each other, on Reddit it's not uncommon to get auto-banned from somewhere if you post in a subreddit they dislike.
That feels like a very incomplete implementation of federation, to say the least. Being able to defederate is one part of that, but "just make alts for the various mutually defederated subsets of subreddits" seems like a pretty unpractical way of going about it. :D
That's Reddit for you, the actual site doesn't really serve the needs of many of its users, so they had to invent workarounds like "run a bot that auto-bans people that post in r/placewehate".
But the actual dynamic of that to me seems to resemble what you get under federation. It's just that Reddit doesn't let people do it easily, so weird workarounds are needed.
huh, I guess seeing reddit like this makes some sense. kinda like Ankh-Morpork on the disk world: not free in any real sense, but free enough in practice for people to get done what they want done.
Not the bright future we had hoped for, but the one we built...
I see this opinion often. I hold it myself. But I don't know how to have individuals hosting zero maintenance secure platforms. The amount to learn how to do a community administration is both difficult and obscure.
The normal used to be hosting a phpbb forum and moving on. Now that doesn't seem feasible to protect from bad actors.
I have a webserver I admin and I found a crypto miner happily running away on it one day. I still have no idea how it got there.
> The problem is that the open internet is dying. What used to be blogs and forums now is Reddit. Tens of thousands (hundreds of thousands?) of communities that had their own wiki, blog, forum, ... are now reduced to be a Subreddit.
As much as I try to explain this to newer generations, I am hit by the Cassandra syndrome. Nobody understands that siloing everything is bad.
As long as TCP/IP exists there will always be a way for people to communicate, it just won’t be at the scale of what we see right now.
As an example, use that news groups still exist. It’s a shadow of itself but it’s still there and it still works despite the fact that Reddit and all sorts of other form based communications, took over.
> The problem is that the open internet is dying. What used to be blogs and forums now is Reddit. Tens of thousands (hundreds of thousands?) of communities that had their own wiki, blog, forum, ... are now reduced to be a Subreddit.
try rdrama dot net, one last bastion of the old "bb board". style forum. and good OC discussions.
This is another reason why it's good for everyone if Reddit dies.
At those times I wish that I could be as eloquent as Cicero was, when he tells Lucius Catalina that his death has been determined by the Senata. I'd tell Reddit the same.
In the 80's, the OSI suite of protocols, owned by the telephone monopolies of the world, actually had pretensions of becoming what the Internet did become. OSI lost, but the spirit never died, apparently.
well I guess if I'm gonna start arguing semantics with you, this is a valid come-back.
so, "use" in my reply means "actively using, creating posts & comments" while "use" in your post means "view content". Is that a fair summary? I think I'll end the definition game here. :D
I thought your response was good thumbsup_emoji.jpg It's an interesting question "who is entitled to see reddit communities up and running at all times" and "how much public-interest is being affected"
My theory is that Google lost the spam war. All the leetcode in the world won't teach someone to build an ML model that can tell whether a website is spam. So Google outsourced the spam detection problem -- they heavily bias their results towards only the most popular sites, who either have human moderators or paid contributors, and those sites do the spam management that Google's automated approach is incapable of.
Google's recency bias in search results hurts them too. There are many older resources which are still valid out there, but you won't find them (easily) via Google Search. Instead, you get the SEO spam which doesn't match the search as well, but is newer.
All of the Google sections seem to be terrible for their use case. Images are lacking filters and are filled with Pinterest results which send people into a cycle when trying to get to the actual image to save. News will sometimes show old news when it's obvious there should be new information about whatever you're searching, leading you to have to specifically indicate sort by date. Videos will prioritize certain domains even if the video itself is irrelevant to your search (ie searching up an actress, it'll show an IMDB video first, even if the video is very old and is just a generic trailer for some movie they were in ages ago). If you search something relatively generic, the new search bubbles will now hide the other categories (ie searching adele will hide categories like shopping or books). The finance option literally just redirects you to Google Finance now, doesn't even retain your search.
They've really let the core search experience be deteriorated so extensively, that we can't blame all of it on SEO.
I haven’t given it much thought until just now but it really is surprising how terrible Googles image search is. There are so many images on the Internet yet the image search often fails to retrieve good examples even on straightforward searches. Never mind difficult searches, higher resolution images, or if you want to quickly download an image. It was never very good and it’s gotten perceptible worse.
Side-thought: I have at times wished Google had a time machine function, for example, "show me the results that this search would have returned in 2008".
It would open up a whole new world of chronological meta analysis; a new dimension of cross-referencing.
[I understand that Google has, or had, some limited incarnation of this, in the form of it's Time Range search, but - and without looking deeply into it - I suspect that is an algorithmically different procedure than the one I am describing. That is, I expect that applies a simple filter to a current search, rather than being The Actual Old Search Results]
there have been many times when i'm searching for things i'll go into the "tools" and limit results from 1990-2010 and find exactly it with a handful of results, vs if i don't a ton of blogspam and other crap hiding it
Is Google biased, or are it’s users biased? If someone is looking up something technical or diagnosing a problem, the most recent post on StackOverflow would be more relevant. Old stuff can be important, but if users searches mostly comprise recent stuff - what’s the point?
I have some old articles on my blog that always decrease in readership. I do a pass every year, change a paragraph here and there, change the alias/url, and then get 5x the readership for a few months. Rinse and repeat. Most of my traffic comes from Google. Also, most of my content is in such a specific niche that it doesn’t really age much in 5 years.
So recency bias kinda works against people discovering useful content. But whatever. Google is in decline anyways.
I just ran a Google search on some code I was working on 15 years ago, posted to blogspot.com. It was about card game X, using programming language Y. The relevant keywords appear in the blog post. However, it completely failed to show up in a Google search. Just... nothing, even when I scrolled through the "more results" stuff. There were relevant and more recent links to the same topic (and likely better implemented, but let's leave that aside).
I did find my page on a Bing search after getting it to not ignore an important keyword, so that's something.
There are way too many google searches where the best result was the top 10 years ago, and still should be the top. Instead it’s something from this year that has no information.
Googling is now like talking to that person who repeats your question back to you as a statement, as if they added to the conversations. The results are some social media manager who’s job it is to make “content” by writing filler.
Content is clearly being auto generated. Even the new Zelda game. Trying to search anything and it’s Tons of pages that all are extremely verbose with a consistent grammar style and not actually saying anything useful.
Most people aren't looking up techniques on how to work with cutting edge software. Sometimes, even usually, they're more likely to be looking for almost anything else.
I think it's user bias. It's shockingly hard to get Google to default to docs on Java 17 over java 8. Users want 8 so Google servers that even if it isn't as new.
Can you give an example? For many things (software, hardware, buying things,...) older resources might still be valid but dated. For more settled subjects, e.g. encyclopedic, I also have not had Google be worse than Bing.
Google results didn't start to decline to be worse than Bing, Google results declined irrespective of and without reference to Bing. Google results are far worse that Google results once were.
Doesn’t help that they just give answers to a query. Answers often being very wrong. Then instead of more results they show related questions, which are often non-sensical.
This is not even getting into censorship of any political thing that google employees dislike.
It is more than the spam war...a lot of content is just no longer produced and exposed to the open web (think about how much content goes into tiktok, discord, etc and you will never get that into your search results). Google has less useful content to index, algorithms can't fix that. There is more spam only because that's the only open content that gets added massively.
The winners of this battle will be the places where content is generated (or curated) - and reddit is perhaps the most important content hub generator (it's not just an aggregator anymore - comments about some news can be often more interesting than the news itself). Indexers (and language models) are useless without content to scrap.
I can't believe that. There's still so many personal blogs by real people. They just don't show up in the first 3 pages unless you query for them very specifically.
Seems most queries result in something like 30% SEO spam sites, 30% quora, 30% reddit, 10% other.
Edit: I don't disagree that Discord/Youtube/Other closed gardens have taken open searchable data away, but it's not like there's now no authentic searchable data at all. Perhaps Google also needs to learn to search those closed gardens better.
Google flourished because it could find forums (and blogs) and mine those, but much of that content has disappeared into Facebook and Discord (and YouTube - we must not discount how many things that would have been easily parseable blogs are now buried in livestreams and videos).
Discord is probably the worst of all. I'm not a gamer and I hate it so much that a lot of tech content is now locked behind private Discord channels. Even Facebook is more discoverable than that
Even when you are already on discord, search and trying to read old conversations is awful on discord, because that's not at all what discord was made to do.
So I've been working on a side project to make a Youtube channel I watch have its content be more discoverable through text. I've had great results by scraping the Youtube transcription, and running that for a few passes through GPT 3.5 with some prompts to essentially act as an editor. The original transcription was often terrible in some spots. Just whole phrases or multiple words mistranscribed throughout. For almost all of them, GPT 3.5 was able to clean them up and restore the original meaning through understanding the context of the monolog and fixing obviously incorrect words or phrases.
I've watched through a sample of about 20 of the 3,000 videos I'm working through, and the corrected transcription really did an amazing job at restoring the original meaning from the spoken words that was hard to understand from the original machine transcription.
That is exactly where LLMs are useful. (People thinking of them as "AI", meaning AGI is just so wrong. Writing legal briefs??) Using them to ex post facto adjust transcripts in order to make them available and searchable is great.
>we must not discount how many things that would have been easily parseable blogs are now buried in livestreams and videos
on the flip side, would those blogs have been created at all if they weren't financially motivated by streaming/video to provide the content?
there's a lot of discussion here about internet commuities, but this comment brings to question why blogs started to die down to begin with. At least with reddit you get clout if you share stuff (useless clout, but sometimes you just want a pat on the back).
Blogs are parallel to research papers in a sense. They're useless without peer review unless you're already intimately familiar with the source material and able to critically evaluate the contents.
So Blogs are more useful when they're aggregated through a site like Reddit, where users have already done the vetting on whether the linked page is valuable. Reddit comments are invaluable to pages by adding additional context. Noting when the content has become dated or inaccurate due to external changes, etc. Sites like Brian Kreb's blog are the exception as the author is well known and respected. But the general blogs? It takes time to earn that community respect.
Then beyond that, how often have you gone on the hunt for something obscure only to run across 3 or more blog pages which look entirely unique, but have the exact same article pasted to them? It isn't that the contents are bad/wrong/inaccurate, but rather who do you trust? How much effort are you going to put in to finding which blog was the original, written by the expert and which ones are bots copying the info?
>where users have already done the vetting on whether the linked page is valuable.
and ironically enough, if you post your own blog on reddit to be critiqued, there's a good chance it is removed for "self promotion". Funny how that "vetting" works, huh? So you get back to "how do I make my blog discoverable so it can be peer reviewed" and we're at square 1 again.
>How much effort are you going to put in to finding which blog was the original, written by the expert and which ones are bots copying the info?
A lot if it's important. Because as is I already have to do that muckracking on reddit to see who is trying to understand or even read the article and who just wants to soapbox their tangential pet rant. tracing a source back is child's play in comparison.
For me YouTube is always on the top, instead of the text pages where I can read the answer in a few seconds Google pushes me their video platform, probably in the hope of making money. I am logged in so I do not understand how those geniuses working at Google would think that videos in a language I do not know might be more relevant then text content.
For me YouTube is always on the top, instead of the text pages where I can read the answer in a few seconds Google pushes me their video platform, probably in the hope of making money.
To be fair, i have the same problem with Duck.
I wish i could backlist sites from my search results. YouTube and Pinterest are not helpful for the things i look for.
How great is your wish? If you host your own instance of Whoogle, which gives Google search results, you can set one of the environment variables to block particular websites from search results.
yeah, the GP really reads like it was regurgitating someone's notes that attended an internal Googs meeting on why they are ranking new higher as their mantra
a lot of content is just no longer produced and exposed to the open web (think about how much content goes into tiktok, discord, etc and you will never get that into your search results)
I see this all the time when trying to find information about old computers. So many of the good vintage computing resources are locked in social services or mailing lists that the information never shows up in search engines.
It feels a lot like the days when information was balkanized between AOL, GEnie, CompuServe, American PeopleLink, Delphi, etc.
Search engines were supposed to fix that and make all the world's information discoverable. They didn't.
There certainly is content. Often the ones I could find two years ago but now cannot.
That's because the web is full of juvenile sub-normie content such as geeksforgeeks (if you consider programming topics for example). It shadows the very specific queries with highly SEO'd Juvenile stuff.
I think this causes further problems as well. These big companies know they can easily be listed at the top of Google, and therefore they pump out low quality articles for every popular key phrase. They can write a "Top 10 X of 2023" list and bring in lots of traffic and referral sales, while a smaller site has no chance of doing the same. Then, the big site takes all that income, and pumps out more low quality articles. It's a rich get richer scenario.
If I search, "The top 10 movies on Netflix", I get sent to this page...
Is this really the best the world has to offer for my search query? I could spend a week putting together a far better page for that query, but what's the point? It's not going to rank anywhere notable in the Google results. There's no incentive for me to produce that quality content.
The real problem isn't that Google can't detect what spam is, it's that this spam drives so much of the traffic, and thus their revenue that they cannot remove it.
Google at this point is literally teaming up with spammers. Just go on a android phone and swipe left from homepage for "Google", count how many clicks it takes to get to an 100% AI generated spam page.
We have all those posts kvetching about how small email providers have become collateral damage in the arms race against email spam. I don’t think anyone is truly successful in that space.
The issue is that ML is very expensive compared to spam generation. And the moment another search engine becomes popular, then the spam cannons will be used against it instead.
for those of us without an android device to try this, care to enlighten us? the way it is described it sounds like a lot of of action required which seems like a good thing, but your point seems like it's trying to make goog's results bad. i'm really not following your point
This shows a bunch of articles based on your search history - many of which are incredibly poor quality and appear to be generated in some way.
I tried to curate mine for a bit and thumbs down useless sites or content I don't care about. I didn't really succeed in making this feed helpful so I mostly ignore it.
Google appears to think that I have one sole interest, which is The Lord of the Rings. And that I want to read endless poorly researched and sensationally headlined articles on differences between the novel and the films. It never recommends me anything else in that section.
I thought it was only me who thought that. Those articles are so much worse than, say, Google News app. Once in a while there’s something relevant, but it’s always clickbait stuff.
I wonder if Google wants to prioritize video to try to address this spam problem.
Sometimes when I search a topic all I get is video in the natural results, not a video search, and I suspect I'm part of an A versus B test.
I personally don't find it to be a compelling solution and click away because I'm not willing to sift through 10 10 minute videos to see if there is relevant content.
Google lost the spam war when Matt Cutts left. Google effectively lost its community outreach, and simultaneously pivoted to aggressive moat protection through AMP, Chrome, Jedi Blue, etc. It’s less that the spammers won and more that Google lost their aim at high-quality content.
For sure. They threw in the towel around '08 or '09. The prior years of yo-yoing between front page results containing ~60% webspam and 0% webspam settled into a permanent 60%, and they evidently just heavily downranked and/or started ignoring low-traffic sites to keep it from growing to 100%.
Their shifting from text-only "ethical" ads to being a more-ordinary web ad service, and putting ads inline with results, roughly corresponds with that, IIRC, which probably isn't a coincidence. Your ad-"results" probably get more clicks if much of the rest of first result page is crap. Many of the webspammers probably funnel money to Google one way or another, now. It screwed up their incentives to keep fighting that battle, I'd guess, which is likely part of why they stopped putting so much effort into it.
Expecting Google's code to flawlessly understand the truth and trustworthiness of all possible sources for every conceivable question is a fair/unbiased way for all topics past present and future is a very high bar. This is a really hard problem, and unlike many other hard problems (rocket science, neurosurgery, particle physics, etc.), Google has millions of financially motivated adversaries actively trying to trick them.
Google won the spam War, they were just fighting for the opposite side. I know it's trite but it bears repeating: Google only cares about user satisfaction if it positively impacts their bottom line.
As long as they can monetize remaining users more than what they lose from users abandoning search, the trend will continue
I mean I think you’re mostly right but to be fair to Google it’s an incredibly adversarial. Thousands of people have made it their full time job to try to trick Google into thinking their blogspam is good content.
Google can't index an internet that doesn't exist. The open web is truly dying.
I believe in the open web, but it seems like an ideological position, not something most people want to participate in. (Just like decentralization!) There's a few reasons:
1. Putting things on the web means your words are public, so they can be archived forever, and anyone can look at them. Do I want employers looking at my high school blog? (Hell, mine is still up, go look it up and you'll find things I no longer believe are appropriate!) Recall that keeping posts private was an important innovation w/ Facebook, over MySpace, which was largely all-public.
2. Most people don't want to bother with hosting their own website. It costs money and requires expertise to run. And it can easily go awry (DDOS attacks! getting hacked! losing your domain!).
3. If you don't want to bother with hosting your own site, you are at the mercy of platforms that always (always!) devolve into bloat, over-commercialization, and perhaps censorship. Medium, Reddit, Wordpress.com, Tumblr - you name it!
4. Most people don't want to put the effort into writing/posting anything beyond basic food/vacation photos or whatnot. Even the people hyper-engaged in politics online are a minority. How often did people write a diary, even, before the internet. Putting in this sort of effort is not something most people want to do. I don't blame them - life's busy and work/school takes the lion's share of everyone's energy. Look at me - all I do is post comments on HN! I haven't updated my two-page static website in years!
5. It's lonely running a website! There's no guaranteed engagement, as there is with posting an Insta story or a Facebook profile picture. It really is like shouting into the void. Yes, one can join a web-ring, but you could also make your own butter...
I think this largely hits nail on the head. Imagine I was particularly knowledgable about ballpoint pens, and I spent most of my time manufacturing ball point pens.
What motivation do I have to spend money to, build, design and host a website, deal with content moderation, work on distribution and SEO optimization and potentially buy ads. If I do this, I might as well expect a return and start selling something, which leads to spam. But if I just want to talk about the finer details of blue ink vs blank ink, and theres a community of ballpoint pen enthusiasts already, it's far easier for me to reply to a couple hundred comments over the course of the year.
I don't know where the "open" web goes from here. With moderation getting more and more time consuming I don't blame people for just running to discord where they can properly gatekeep information.
> What motivation do I have to spend money to, build, design and host a website, deal with content moderation, work on distribution and SEO optimization and potentially buy ads
Betterment of the shared social world of the web? Not everything has to make a profit.
> Google can't index an internet that doesn't exist.
Then it should index the internet that does, and stop trying to convince me that there are only 10 pages of results for any possible query.
> The open web is truly dying.
Of starvation, because the search indexes don't index it, preferring to index the part of the web that is aggressively marketing itself (and buying and showing ads.)
Even if it's possible to win the battle against SEO and junk sites (and I don't think it is), this is the best that commercial search engines will do, given their resources and incentives. No one else has been able to break in, for lack of quality, for lack of resources (it's expensive to index the web!), and b/c search engine brands are so sticky (that's why Google pays Apple ~$10 billion to be the default on Apple devices.)
If the junk web problem was so easy to solve, wouldn't another search engine be able to do so - like Bing with Microsoft's resources, or DDG with their unique outlook/guarantees?
So given the current economic system, the current governance regime, and the current culture, this is the best our civilization can do. Unless there's some sharp rift, the internet will stay the course.
I feel like this line of criticism feels a bit like screaming at the clouds.
Regarding 1: I firmly believe that in the future, what you say doesn't matter. What matters is what you can prove you've said.
What's the difference between the following two? (a) You say something and someone takes a screenshot of it, and (b) Someone just conjures up a fake screenshot of you saying something. These are indistinguishable. With LLM and SD, these fakes will become easier and easier.
The solution is some crypto that allows you to do two things: (a) Be able to prove that you said something, and (b) later revoke it, such that your prove doesn't work anymore and there's no trace it was ever used.
I noticed this too when I was trying to search for local opinions like affordable clothing in netherlands and for general recommendations like cloud storage with execellent file management.
Without reddit, outdated generic blogs with affliate links show up. They are just advertisement in disguise and not the genuine opinions which I'm looking for.
In a way this shows the value of Reddit. Even with their awful UI and app, their users seems to provide the most value to search queries.
I search for reviews of DIY tools and all Google shows me are review sites that are just thinly veiled affiliate link farms. I even read a post somewhere about a guy boasting he has SEO optimized his review sites to show up at the top of google searches and he makes a bundle on all the Amazon affiliate links people follow.
For me, the solution is not Reddit but Youtube - luckily, Google can now link you directly to the point in the video that matters otherwise videos are not random searchable, and I haven't seen all videos have text transcripts you can search through.
Video based solutions for many things are deeply ineffectual as they'll become outdated rapidly and are difficult to re-edit to add in missing context following publication
I think affiliate deals should be banned. I hate it that much. Whilst you’re at it, van advertisement without consent. You can use the kids argument if necessary.
A lot of these sites are probably already breaking FTC rules.
If one runs an affiliate link blog they're supposed to disclose that relationship when they show the links (and in a way that the reader won't easily miss it). [1]
It was a big thing in video game streaming that even stuff like free game codes given to content creators can require something like #ad or #sponsored put in the stream title.
So you want to live in a world where only companies with massive ad budgets breakthrough to end users? The ad empire only strengthens media/big app lock in.
I want the ads to be clearly labelled and only accessible when I've asked for them. Wasn't that the idea of Google AdWords? Show people results ranked by relevance, and insert a few related sponsored products at the beginning with a different presentation.
Enshittification was already causing reddit to ruin google results.
Reddit's new garbage redesign contains other posts from a subreddit in the page. So often times you'll google a term and get a link that looks like exactly the post you want, but it ends up being a different post in the same subreddit, and the one you were looking for was just in the suggested posts for that one.
Also, reddit has ruined filtering by time somehow. I'll often try to search for information on e.g. a new update for a game and find posts from years ago about old mechanics, despite filtering to "last 30 days"
You'd think that reddit would be smart enough to show web crawlers a cleaner version than real users, or is that not allowed? Maybe google needs whitelist some of the biggest sites on the web so they can feed searchable data into google in a more efficient way.
I think curated indexes, with associated search, will come back. Yahoo! was before its time. Now we need a human-curated search engine/index that only has verified human generated content. I guess everything goes in cycles.
Problem is that the curated index becomes so valuable that money speaks louder than the quality. As long as the company is on the stock market or is made using VC money, it will result in enshittification: https://pluralistic.net/2023/01/21/potemkin-ai
The only solution I can see is a mom-and-pop site like Craigslist which are owned by a family which don't want to optimize for money as they think they already have more than enough money.
None of it is due to organic interest anymore. Just gamification of attention to try and spread a viral sales pitch.
Paul Graham did some napkin math on Twitter a few months ago highlighting his approach is just try to get ahead of a trending but still not quite super popular idea.
It’s Shark Tank; film the show months in advance, get the viable product for sale before it airs, because you know copycats will be right behind you.
All to preserve a hallucination reality will collapse without a fiat currency game to play.
> Paul Graham did some napkin math on Twitter a few months ago highlighting his approach is just try to get ahead of a trending but still not quite super popular idea.
The stupid part is that profits are really not optimized by the enshittification profit campaigns - there is so much incompetence in isolated C-suites making these decisions.
I've said it before, but why do social media sites try to monetize through methods which provide no tangible benefits? A blue checkmark, some shitty NFT avatars, being able to badge someone's comment or post - I don't care about those things, definitely not enough to pay for them.
If Reddit wanted to monetize me, give me something tangible. I'd pay $5/mo for a 'pro' tier or something if it had unlimited third-party app access on top of some other real benefits. Instead, Reddit is making everything worse (like their new, awful, web view, or their official, awful, app) and then getting mad that people don't like the shit sandwich they're being fed.
Charge people for pro tiers, charge subreddits for 'pro' moderation tools (and actually build the moderation tools you say you're going to), introduce some kind of subscription service (YouTube Premium, Discord Nitro, that kind of thing) that actually benefits both users and subreddit moderators. Let subreddits provide perks to community members who are willing to contribute, or be able to make them subscription-only (or posts and comments only by subscribers) and the money will roll in.
Generally just make the site better and people will appreciate that. Don't isolate us from the apps we like and then tell us "we're doing it because we care", because no one believes that.
I ran /r/conspiracy for more than a decade - and when I was setting policy to have that sub be a-political, and some other mods complaing about their pecking order in the mod list, I was restructuring the mod list at the request of mods, then an admin's alt account 'complained' (and this mod was the actual mod who requested the org change) and an admin cam in, ousted me, and put said account in charge.
That account is assuradly an admin alt account throwaway.
I think the question is what are the tangible benefits people would actually pay for?
Social media companies have the incentive to figure out how to make more money from their product. I think the fact that none of them have really proposed any pro-tier that is adopted by a majority of users shows that most people aren't really willing to pay for anything they provide.
Reddit doesn't want to have any functionality that makes the site hard to use. They don't want to say, limit how much you can post because you're making content for free for them and keeping other people engaged.
Minor quality of life improvements are easily addressed with browser plugins like RES. It has to be something that can't be provided that way.
So what you get is very niche features like the ability to load more than 500 commments and a shiny badge.
I think it’s the same reason countries don’t simply get better as we expect they should: there’s no omnipotent entity with that exact goal in mind. Instead, we have a lot of people just going for their own interests, playing politics or corporate game, moved by vanity or greed etc. which often do more harm than good for the company’s supposed goals (not even mentioning larger companies tend to attract certain types of people who are more parasites than “builders”).
This why it’s funny that when the Victoria 3 game was released, people realized and it became news that Communism was more effective to make your country thrive. Well, no shit? You’re playing as an external entity which only purpose is to make your State greater in a century, so the more power over the economy the more effective you are at reaching your goals. Real countries don’t work like that, and the same logic applies to companies.
And Google itself is the primary instigator, with Google AdSense.
Google AdSense is a program run by Google through which website publishers in the Google Network of content sites serve text, images, video, or interactive media advertisements that are targeted to the site content and audience. These advertisements are administered, sorted, and maintained by Google.
I hope this is finally a wakeup call to Google to try to start to prioritize non reddit user content on the internet again. Web forums have been suppressed by their results for ages now and Google's choice to do that has been a big part of what killed the internet.
Many years ago, one of the search options on Google, alongside Images, News, etc., was "Discussions." It would explicitly show results from web forums. I defaulted to this when I was looking for, e.g., product reviews -- because it would bypass the endless SEO spam where someone found some highly-rated products on Amazon and published an article "reviewing" them.
Unfortunately, Google killed that search type. There are some browser extensions and bookmarklets that help, mostly by creating a very complicated search query that looks for specific things in the URL, but the discussion search is something I still mourn.
For what it's worth, my technique these days is to find a forum which has a large amount of discussion, then add a site filter to my Google query and only search that site. For something like product reviews, I can usually find the lesser-known products that don't show up on the SEO lists, at which point I now have a good keyword to find other enthusiast discussions online, which begins to open a lot of doors.
Unfortunately, I haven't quite refined the method for anything other than product reviews. At this point, I've started just using ChatGPT for factual questions before trying Google. :(
Same, if I am done reading reddit discussions I will just switch to 'forum' and occasionally stumble into some forum threads. Most are dead now though!
The Discussions page on google was fantastic. I was able to find information so easily. When they shut it down, there was a firefox extension to bring it back. Until Google shut it down completly. I've always wondered why they did that. Perhaps they wanted to bring attention to bigger websites with adsense? I have no idea.
They apparently[0] have plans to do this for their next "Helpful" content update. And that update should happen soon if we go by the current update metrics[1].
(I also manage sites myself and they've been doing some heavy A/B testing the last couple of weeks, but we have to wait and see how big of an impact it will actually have.)
For people who don't want to click the Twitter link:
> We're also improving how we rank results in Search overall. Helpful information can often live in unexpected or hard-to-find places: a comment in a forum thread, a post on a little-known blog, or an article with unique expertise on a topic. Our helpful content ranking system will soon show more of these “hidden gems” on Search, particularly when we think they’ll improve the results.
I ditched Quora when they had a similar policy change as Reddit, which I disagreed with (honestly, I cannot recall what the kerfuffle was about, but I deleted my Quora account and never went back.) I hate when quora pops up in results.
But google has engineered themselves, deliberately so, into the "black-box behometh in the corner nobody can poke or question, and it just does its damn thing despite any of your riff-raff's ramblings"
They didn't suppress web forums, they instead went to prioritize SEO factors that of course SEO spammers maximized to abuse. Of course the Google solution was to prioritize reddit because people were searching for "<topic> reddit" due to said SEO spam.
I think the primary issue is that no matter what Google prioritizes for, the SEO "blogs" will always switch over to attempting to look like it. Forums, whose goal is just people talking to each other, are never going to end up at the top of the list compared to people actively attempting to game the algorithm. Even if Google started actively promoting things that look like forums, SEO people would just make things that look like forums but are hyper optimized for whatever Google was looking for.
The only way around this is for google to start making a white list of known good forums, but people get (understandably) really pissed when a company of google's size starts actively choosing winners and losers instead of just tuning their algorithm.
bing accounts for your history in promoting search results. and has a different algorithm to google, inevitable really because google's algorithm is secret and unknowable.
the problem is that google WANTs to promote forums because forums are reliable sources of truth, but with automation all they can do is look for subject authority markers (incoming link count, keywords, text references on other highly authoritative pages, etc)
SEO as an industry is about optimizing those markers to appear above "organic" results in a search page.
changing the markers only means the SEO industry will adjust to target the new priorities, but forum operators are always the last to optimize. it's a fundamentally unwinnable game
I posted this a few days ago, and in-line with your thoughts ;
-
I use OLD+RES for MY consumption and data density - if you dont know how to configure these together to create a much faster, and more aesthetically pleasing (to me) UX - then that sucks.
I think 5-6 years ago, it was good. I enjoyed using it and experts wrote some amazing, no filter, content you can't get anywhere else.
From my experience, Quora failed because:
1. Quora decided to automatically mass generate fake questions. Someone genius at Quora saw that the number of questions correlated with engagement/traffic so he/she thought let's just mass generate questions for people to answer. Inevitably, this led to just pure spam. It's like Quora decided to spam itself.
2. Overtime, users figured out cookie cutter, feel-good, often fake news answers received many upvotes and Quora boosted those answers up. This led to an insane amount of garbage from users in India.
3. Quora decided to use every dirty tactic to get people to login/register and download their apps such as hiding content behind prompts, using unkillable popups, etc. Just like Reddit now.
I remember receiving an invitation to some Quora program to post questions for the website, in exchange for monetary compensation. Never took them up on it.
But this seems like the kind of thing, that will lead to low quality content on any platform.
This would have been a crazy idea that I (would have) like to see happen. The fact that we're talking about Reddit content in Google searches is an indicator that Reddit should have improved search on their own site.
In a different universe, I think the idea could be a valid attempt at dethroning Google. However, after seeing the changes to Reddit the past decade in our current universe, I wouldn't trust those same people to handle search for the world.
Google should be more pro-active in checking its results health and present cached versions if underlying sources are gone. It's not only about reddit, I also have plenty of websites which don't open in my computer (either because my country banned them or because they banned my country) and Google still presents those pages in my search results, wasting my time.
Google used to offer this functionality, but it seems to have disappeared over the past year. It was never automatic, but there were 3 vertical dots to the right of the list with a "more info" menu.
If you click on the three-dot menu, there's a small caret button you have to click to show more of the pill buttons at the top, and one of them is "Cached".
It feels like every time I need to use that button it's moved somewhere new.
However, this option is hidden for me on every result I’ve checked. Pages posted several years ago, pages posted within the past couple months, Reddit pages, StackOverflow pages, etc. Even pages from Google itself are missing a cached page option for me.
I've been bumping into this a bunch trying to research different neighborhoods of Washington, D.C., for an upcoming three-month stay. A lot of useful info is locked up in r/washingtondc right now, with much of it missing from Google's cache and the Wayback Machine. I still support the blackouts—I used to mod a default subreddit—but I wasn't expecting this particular effect!
(Tangentially, anyone know how the Mount Vernon Triangle area is these days?)
Where in Vernon Triangle? There's a big gradient between its eastern and western half. The western part, closer to downtown, has lots of restaurants and shopping, but it can feel a little dead further east. In general, an office-y vibe, not much green space.
Have been running into this the last few days and have realized just how much I use reddit in search results even though I don't visit the frontpage anymore.
Lot of pasting links into archive.org.
I've also been using Bing Chat to ask it what reddit thinks since it has most of reddit indexed.
Concerning though because it makes me feel like quite a lot of important information is rather fragile.
You can imagine various ways this data might be lost forever, I wish it felt more durable.
Currently struggling with searching for the usual developer queries, but not getting the answers I need :(
“Something something random package name is suddenly broken Reddit”
The backup is using the same query with “stack overflow”, but that knowledge has become dated.
Around the rise of Slack, and then Discord and Teams, knowledge started becoming transferred between open source devs within their corresponding chat. People would get their answer on a VueJs issue and move on… The end result was that Google never picked that up, and the next dev popped in the chat the next day with the exact same question.
In the last couple years, Reddit has become the place for devs to do q&a as well.
It’s also making it difficult for me to communicate with prospective users online for simple market research and feedback.
It probably would be a more effective campaign for subreddits to have their users wipe all their old comments (older than 1 year) than a temporary blackout for a few days.
It's the right of each user to redact their contributions. If that's the only bargaining tool then it seems entirely reasonable to use it when the other party refuses to negotiate or even be remotely transparent about their motives. If this was really about money then why do we have such little insight into their financials? Could it be that they fleeced the investors and will lie as hard as they can to not be outed for the crooks that they are? It's the most likely explanation from my perspective.
There's no reason to prop them up. The information will be rebuilt quickly or it won't.
People didn't post on Reddit for Reddit, people posted on Reddit in the hope that other people might read the post and get something out of it. Burning all of the books that I wrote for others to read is attacking myself to hurt Reddit. I don't care about Reddit enough to hurt myself to hurt it.
In that case, the script could be modified to archive all the comments into a local file -- you could restore all your books back to the library (Reddit) if you chose, once they stopped threatening customers with higher fees.
If people were really committed, they could "unionize" and have a centralized party which held access to their comments, but that would probably violate their ToS and would not set a good precedent for an open and free internet. Not to mention, Reddit could easily just block users from modifying/deleting old comments.
Humans built this wonderful common structure, where we can all communicate boundlessly.
The huge benefits only last briefly for humanity, until our inherent nature causes it to degenerate into a mess. Everybody goes back to doing their own thing.
No, Google, Facebook, and Reddit are the languages spoken on the WWW that annihilated all the others. Then God comes down and breaks them up. The Tower of Babel was the original silo.
Google is mostly a data mining operation. Reddit is an organic content mine. Google is a behemoth, while Reddit are struggling to go public. It speaks volumes about how the internet has evolved in the last 25 years or so.
Somehow after all these years reddit never got their search to work. You can paste a comment word for word in the search and reddit won't find it.
They're dependent on google.
Also if they somehow figure out search to work, it will be ruined because of users gaming the search system with SEO.
> However, the argument that Reddit can build a better search than Google is a flawed argument. And that flaw is the same flaw that makes Google results bad. And that flaw is money. [1]
What about Reddit search quietly powered by ChatGPT4
I’m day dreaming about Reddit telling OpenAI to foot the search bill or they’ll make the case that OpenAI needs to remove all of Reddit from its training?
I think it does. They're busy self-sabotating at the moment. They have never properly figured out search though, that aspect of Reddit has been mediocre since the beginning. There is reason to be skeptical that they could figure it out now, given their other product/feature development; they appear incapable of doing it effectively.
Or an opportunity for google to buy reddit? Seems google knows how to monetize and knows how to search. They do already have one huge successful online community (youtube). Yeah, I don't have much faith in google management and buyouts either, but maybe..?
the realistic opp for Reddit is to charge Google access to the API which would more than cover all of the other API users. If Googs can afford to pay Apple and others to have their search be the default, they can afford to pay to crawl Reddit too
I feel like the blackout isn't gonna be effective only because Reddit is just gonna forcibly unprivate the large communities and assign new mods. Pretty disappointing but not surprising. I know I don't have any interest going back there.
Bing should buy Reddit and kill Google. This blackout has really revealed how bad Google Is and give reliant it is on Reddit working. Buy Reddit for peanuts, sell Google api access or block them entirely from indexing
I just had this thought. Seems like user generated content is gonna be very valuable from now on, so wouldn’t it be good for companies like Microsoft to foster new websites like Reddit?
If it's perceived as profitable or core to the business, the latter more likely, I would not be surprised if quite a few companies try their hand at a reddit like endeavor. I hold out hope that who ever owns slashdot would add user generated categories and integrate the meta moderation system to keep check on the volunteer mods that it would require...
It would be too funny if Google now announced they will no longer index reddit because the API is too expensive. Maybe that would spur reddit to improve search, or their own app and web offerings.
It's surprising how this blackout has shown how important reddit is in general, and that is what reddit should consider in regards of their financial future
This really rings true - I was trying to do some semi-niche hardware/software research (wiimote on steam deck) and the only real sources for that information was on closed subreddits.
I ended up having to just view the cached pages and go on the Discord for help.
It really highlighted how dependent I've become on getting pretty good results from niche subreddits.
I knew that someday reddit executives would close old.reddit.com and that that day I would have to stop visiting because of how shitty the new interface is. What I never could have imagined is that it would be the actions of mods that would drag me to not being able to use reddit. It came as a surprise to me.
The internet is getting worse. Every comment on here is proof of that. We as tech-people MUST change this, because well, no one else will/cares/wants to.
Let's try to make it easier to make these communities without relying on monopolies.
This is an easy thing for spez to fix. Simply allow for external links to be followed to their destinations but block the view-all view of the subreddit when a sub goes private.
I've really gotten frustrated with google over the years and I've switched to Bing. Is Microsoft better? Probably not, but I'm a researcher, and I search fast and significantly. Google is constantly asking me to verify if I'm a human, Bing doesn't do that. I recognize that may be an individual issue, but on a broader spectrum, Bing works well.
Same search pattern as you. I have the verification problem only if I do it from the net of my research organization, never if I do it privately. Not sure why, just a pattern I observed.
I happened to notice this on Sunday evening. Was searching for some info on an Intel chip and some stuff from r/intel came up. I didn't even know there was an r/intel. The sub had already gone private at that point and even though I knew about the upcoming "blackouts" it still confused me.
Google + Reddit should form some sort of partnership to monetize results. Maybe instead of Reddit.com , users are redirected to a sort of landing page that incorporates Reddit answers but is not Reddit, and also has google ads on it. the landing page, using AI, incorporates the answers in a more readable format.
Create forums and message boards again. Give them topicality. Support RSS and open data transfer protocols. Manually append these places to the Google and Brave indexes.
I'm fine with Reddit AND Google disappearing. I'd much rather be on old forums that actually provided useful information.
Reddit is just an echo chamber for people to complain for the most part. I rarely find useful info there. I quite using Google search quite awhile ago and don't miss it at all.
Do people really find Reddit useful? I usually find lots of "hey guys has anyone got <same problem as I am searching about>" from 2018 with no solution but a few token "Me too!" Responses, or worse a response but "Never mind - I solved it. <No explanation>"
I skip over most Reddit links in the search because of that sort of thing being 95% of results. I genuinely have not ever found that much useful on it - I can probably count on one hand a time where it has been useful.
Is it just me that thinks Reddit is deeply overrated? Or is there some magic part of Reddit with people who know what they are talking about and who properly answer questions in a timely manner? I've never found it if so. Perhaps I need to have an account to access the best bits? But if so how would Google find it?
Yes, all the time, but quality varies heavily based on subreddit, and in my experience the more popular and/or less strictly moderated subreddits have lower quality.
The <no explanation> part isn't unique to reddit. See: XKCD 979
The Internet has become fragile. One service goes down and everybody suffers. If the top 10 services went down most people would think that there is no Internet at all.
E-Mail is the last standing service that is way more open that the rest. But the raise of Whatsapp and equivalents are challenging that. One day all our communication will depend on a monopoly. We are starting to know what would have happened if AT&T have never been split.