I really wish this kind of thing was more talked about, and better understood. Most people have no idea how these "bots" or "trolls" actually influence conversation online, and what impact their actions have.

One thing that frustrates me most is that even if people do reach the point of understanding the power these people have, it still won't matter much. We still trust online media such as Reddit, Twitter, Facebook, and even this website, as being the fountain of the mythical phenomenon known as "public opinion".

I can see in real life that people are becoming fatigued by the uncertainty of our modern existence, but everyone is unsure of how to pinpoint and react to the source of their tension. My expectation/hope/fear is that the younger generation who grow up in this environment will eventually form a nu-luddite movement.

My bigger expectation/fear is that they won't

History always provides context. In the late 1700s, the "printing press" gave new freedoms to publishers never before seen in all of history. How did our lawmakers react?

Yes, they passed the 1st Amendment, protecting the right to free press. But just a few years later, they also passed...

https://en.wikipedia.org/wiki/Alien_and_Sedition_Acts

> [The Sedition Act] criminalized making 'false statements' critical of the federal government

And an example of prosecution of people through the Sedition Act is Mr. David Brown. Here's his story:

> In November 1798, David Brown led a group in Dedham, Massachusetts, including Benjamin Fairbanks, in setting up a liberty pole with the words, "No Stamp Act, No Sedition Act, No Alien Bills, No Land Tax, downfall to the Tyrants of America; peace and retirement to the President; Long Live the Vice President."[17][19][20] Brown was arrested in Andover, Massachusetts, but because he could not afford the $4,000 bail, he was taken to Salem for trial.[19] Brown was tried in June 1799.[17] Brown pleaded guilty, but Justice Samuel Chase asked him to name others who had assisted him.[17] Brown refused, was fined $480 (equivalent to $7,300 in 2020),[19][21] and sentenced to eighteen months in prison, the most severe sentence imposed under the Sedition Act.[17][19]

-----------

Our 2rd President said at the time:

> “There has been more new error propagated by the press in the last ten years than in an hundred years before 1798”

----------

EDIT: It should be noted that Ben Franklin himself engaged in the creation of literal fake news, as a propaganda / negotiation tactic against the English Monarchy. Ben Franklin created a false story about Native Americans "Scalping" US Citizens on behalf of orders from the English Crown.

Our founding fathers were no strangers to propaganda, fake-news, and other techniques to rile up populations and/or control the dialog. Indeed, they themselves took advantage of it to successfully launch the Revolution!

I really, really wish there were spaces that required some sort of proof of critical thinking to enter.

The harder the test to get in, the better. Maybe even a non-refundable 50+$ fee. I would give a lot to have a single space on Earth where mere popularity is not the metric of correctness.

The fundamental problem of online "discourse" is that critical thinking is applied selectively according to emotional biases, not that participants are intellectually incapable of it. Malicious actors, doubly so. An attempt to control that can only devolve into testing for the "correct bias", and communities with sufficient moderator:post ratio or upvote/downvote type community moderation already perform that sort of self-selection.

As for monetary fees - demographic skew aside, that would significantly reduce the overall user population the more you charge, so a malicious actor would need correspondingly fewer sock-puppets to create the appearance of X% of the population saying Y, limiting its effectiveness as a deterrent. Amplifying minority views by a large factor or seeding new claims/statements doesn't require that many unique accounts in the first place.

> The fundamental problem.

You lost me. If some context was in there (as in "the current fundamental problem on my known platforms"), then everything else you said would follow. But there must be a way to do better.

I can personally think of many tactics currently unused by any social platform -- all of which would need testing to verify their efficacy -- but they leave the question open none-the-less.

Unfortunately, universities aren't looking so hot in this regard at the moment. Maybe the rationalist community?

>One thing that frustrates me most is that even if people do reach the point of understanding the power these people have, it still won't matter much. We still trust online media such as Reddit, Twitter, Facebook, and even this website, as being the fountain of the mythical phenomenon known as "public opinion".

I'm much more interested in who the moderators of social media are and who pays them. Going after 'trolls' is to think we still live in 2011 when this was the way to astroturf.

Reddit for example, on average has none of the content users post on their site reach and stay on the front page: https://www.reddit.com/r/RedditMinusMods/

HN is fairly straightforward in moderation: what dang doesn't like gets deleted, but the user flags have also been weaponised recently.

Twitter and FB I have no idea about. But I can't imagine it's hard to find out who the paid moderators are and pay them some more to delete content you disagree with and not delete content you agree with. The majority of them are paid peanuts to begin with.

For most big forums, Twitter, reddit, and Facebook, it's probably 2$ an hour workers in 24/7 facilities in India, the Philippines, and other low wage parts of the world. Automation is being used at an accelerating pace, and there's so much data being pumped online that only a vanishingly small percentage of flagged media gets reviewed by an actual human. A dozen or so motivated and clever humans is all it takes to get a person or channel suspended on any of the big platforms because automation is so embedded. Situations that a human world recognize as an error usually get corrected, but almost all social media is too much data by far to include humans at a meaningful scale. Moderation at scale is currently a very blunt instrument, compared to a well curated, civil forum.

Transformers and other recent models have enabled very nuanced classification of text, so we're going to see a confluence of economic factors - the quality to cost ratio of humans vs software at all levels of moderation and curation. Eventually the bots will be just barely good enough and cheap enough to run at scale that humans will be removed from the loop, from first pass sorting all the way up the chain. The humans will only set the parameters, and then we'll have a whole new bag of problems.

It'll be easier to manipulate software to trigger censorship than to manipulate humans following a set of guidelines.

https://news.ycombinator.com/item?id=23918763

Because of worker protections, minimum wage, and legal liabilities, it's better to find cheap multilingual offshore moderation farms than to hire in the US. It's a race to the bottom. The problem of moderation at global scale may prove to be intractable without human level ai.

Interesting podcast covering the influence of these computational propaganda campaigns and their effect on the human mind https://soundcloud.com/inpatientradio/neural-narratives-comp...

I like to think, somewhere, there are rooms full of patriots who pursued years of education and intense training for a role at a 3 letter agency - believing they would travel the world doing super secret spy craft. Now they spend their days sitting in dark, windowless, rooms in front of a computer screen slinging dank memes on the web.

I also believe there are some zealous patriots who joined the military, inspired by dreams of killing our enemies, only to be relegated to posting memes on The Army’s Instagram and TikTok pages to recruit other kids.

Or editing wikipedia, and arguing on talk pages.

arguing on HN posts

How's your shift, comrade?

Not bad. Sometimes socialism advances by inches, one argument at a time. But, I have to say the pay is abysmal: we need a union.

Imagine Marvin, the paranoid android working at a troll factory; I guess his trolling would even be kind of interesting! Interesting, couldn't they use GPT-3 for the troll factory job? What is stopping them from using real bots, instead of humans?

Good thing they shine a light on those evil troll farms. And not on the "internet activists" of political aligned NGOs in their own home countries.

This month Germany elects a new Bundestag and almost every party recruits their own online activists. E.g. they form "discussion intervention groups" to dynamically flood any online discussion, to ensure their narratives are protected.

Our current Mexican president has a very good grasp of social network manipulation. His critics were swiftly met with insults, profanity, or simply with offensive pictures, but now they have become more subtle. Their accounts got progressively harder to differentiate from common users because they appear to have other interests besides politics, even if pro-regime politics is the bulk of their content.

People need to relearn that something being on the Internet doesn't mean it's true, or even that it comes from a real person.

This is great journalism. Good job, and tbh, it comfort my theory that the twitter crowd, especially the loud twitter crowd, is small.

No parole for the trolling Poles.

If they direct you to a fake login page, they're phishing Poles.

"an activist for the conservative ruling party. . . He got the job – without even having a university diploma or a military background – thanks to his friends at the defence ministry."

I realize it shows my biases, but I am waiting for the first major troll farm associated with the left and with, I dunno, PETA or whatever the opposite of the military industrial complex is. Do they exist? They must, but it seems (based perhaps on my echo chamber) that 95% of the effort is from vested interests who prefer the status quo over fact.

Troll farms are associated with those who have more money than committed activists. Whatever you may think of the left, it tends to have the opposite. - maybe the vague center left at the level of Jeff Bezos could that but even here, you have money proportionate to committed followers.

the opposite of the military industrial complex

There's no money in anti-militarism. antiwar.com is a libertarian (ie right) website but it has survive on donations. Troll farm pay for opinions, that's a pretty specific thing.

Edit: Reading the article, this farm apparently supported some "left wing" causes. But basically it comes down to money.

I used to work in a team where everyone except me had studied at the now closed Central European University. My current personal manager also sudied at the CUE. Very clever and open minded people. And what is "your experience" based on again?

> regular people hearing "NGO" interprete it as shady-leftist-organisation

https://en.wikipedia.org/wiki/Propaganda

If you think Soros and his foundation are leftist, i might have something to sell you.

> that 95% of the effort is from vested interests who prefer the status quo

The incumbent power structure has both wealth and incentive to resist change. Underdogs usually don't.

Troll farms are just one form of social media manipulation. A far more common thing is a mailing list or a discord or similar used for covert coordination of things which aren't supposed to be coordinated: voting on social media sites, reporting campaigns, news coverage spin (or news blackouts), even infiltration and false flag posting.

It's been a problem for years. I think that online, Wikipedia was the starting point, since it was the first social site which was vulnerable to entryism and actually a tasty prize to have control over. There's no end to the "secret mailing list"-scandals of Wikipedia over the years. And Wikipiedia at least publicly acknowledges off-site secret coordination as a problem, unlike most other supposedly user-driven sites.

The less actual support a view has, the more it may tend to troll farming or corporate astroturf. But for views with some degree of popular support, left and right, backroom deals in the vampire castle is where the serious manipulation business happens.

Say what you want about the right, but they are a more unified and organized conglomeration. The left tends to dissolve into factions and disagree among themselves. Even in a Democratic held Congress and Senate, it's hard to get things passed where as the Republicans for all their faults seem to push through items after item, block judges from the Supreme Court, etc etc.

When it comes to these smaller organizations on the left, like "Antifa" they aren't even an organization, really. It's just a name, and you are Antifa if you say you are. The guy shooting at the WA state prison was just as much Antifa as the protestors in Portland, even though they had no connection or similar goals or values. This leads people to argue as to whether or not Antifa really exists. For some it does, and they believe they are in it. As a unified organization, not really.... welco.e to a lot of left "organizations".

It's hard to get things like a troll farm going when you can't even get yourself grouped with a single purpose.

It's fascinating that you say this, because I had only moments before read a conservative bemoaning that the right has so much infighting, unlike the united left. Maybe it's easier to see the division on your own side (if I may make the assumption).

You aren't wrong. I'm not particularly conservative on a lot of issues, even though I'm not a Democrat.

The level of inglorious buffoonery that we are seeing would not be possible if either side had any modicum of unity.

I call this the No Caucus vs the Yes Caucus. Unlike sportsball and war, defending the political status quo is always easier.

Building consensus in the Yes Caucus is very hard. Agree that something must be done. Then herd the kittens. Then buy off all the hostage takers and defectors. Then the caucus dissolves for every little victory (little potential for building momentum). Etc.

Whereas the members of the No Caucus only need to agree on "No."

It's unfortunate, or least weird, that "right" and "left" have been associated with establishment and reform. What ever one's opinion of movement Conservatives, 60 years ago they were the reformers, challenging the status quo.

They succeeded. And now they're playing defense, with all the inherit advantages.

FTA

> But the company does not only have right-wing clients – they work as well for ‘the other side’. On May 10, the company was ordered to suspend activities in favour of TVP until further notice. From now on the troll farm focused on the Twitter account of the deputy head of the Democratic Left Alliance (SLD), Andrzej Szejna. He is a successful lawyer and election candidate.

Trolls gotta troll.

I thought it was pretty widely known that the many troll farms work just to drive us all apart. They scream loudly with extremist views and convince everyone on either side that their opponents are plain crazy.

I was startled to find recently that the 70% of adults in the US have at least one shot. Any media outlet would have you believe it was 50/50.

Yup, I missed that, thanks for the correction.

"Correct the Record" is what you are looking for. Naturally, nobody in the media will call them a troll farm but you have your basic walks like a duck criteria there.

This place does seem to work both sides. This down towards the end of the piece:

"But the company does not only have right-wing clients – they work as well for ‘the other side’. On May 10, the company was ordered to suspend activities in favour of TVP until further notice. From now on the troll farm focused on the Twitter account of the deputy head of the Democratic Left Alliance (SLD), Andrzej Szejna. He is a successful lawyer and election candidate."

Many animal charities are trolls though. You can start a charity and your 'independent' board can just pay you and your brother who runs the advertising all the money. Send a few bucks to your cousin the lobbyist, some more to your other cousin who owns the office complex, and then leave 1% as grants for animal shelters. This is not far from what some PETA like charities do.

>vested interests who prefer the status quo over fact

I'd say that the Left is the status quo in the West at this point.

We probably need new terms for everybody.

We do need new terms for everybody. These terms from Revolutionary France are making enemies out of pepper who mostly agree with each other, and allies out of people who don’t actually agree on very much.

I don't think you can, and therefore you shouldn't use Twitter (or Reddit or Facebook) as an honest or useful place to discuss politics or anything of that nature.

To put it another way that's a little less accusatory but still encourages the right mindset - assume any comment on a service from a user without verified ID was written by a troll trying to provoke a reaction.

Anonymity is a poor heuristic. I rate people as trolls or not dependin gon whether they employ fallacious reasoning, and how they respond to being challenged. Not all fallacious arguments are trolling, as people can simply be wrong. But if a person is supplied with accurate information, or the form of an error explained and acknowledged, only to see them return the next day with the same schtick, then I regard that as posting in bad faith.

There are plenty of anonymous truth-tellers and plenty overt hypocrites and liars. It's important to remember that not all lies are meant to be believed; some are merely intended to upset, to bait, or to signal.

The problem isn't anonymity as such. I don't think there would be much problem with anonymous posts, provided we could be confident the anonymous poster actually was just one guy who spoke for himself, at a reasonable rate.

But you have anonymous posters who speak under hundreds of names to give their opinions the illusion of popularity; argue with their own accounts to set up straw men and steer discussion away from things that threaten them, and post hundreds of times more than the average users.

I think there must be solutions to this, I'm sad so few people seem to be working on it. Shouldn't there be a way for instance, using good old fashioned cryptography (I.e. NOT some tradeable token junk), to leverage a strong ID service to prevent sockpuppeting in a forum without revealing much to either the forum owner or the ID service?

Once we had that - a basic safety that everyone you engaged with on a certain forum was a real person, and this person (whoever he was) didn't operate under other names on that forum (or if he had earlier names, that they were irreversibly retired) - a lot of sensible things would become possible which are largely pointless today, such as speaking limits and distributed/allotted moderation.

> Not all fallacious arguments are trolling, as people can simply be wrong.

There's factually wrong, there's logically fallacious argument, and there's "I don't agree with you, so I will say you're wrong, and be condescending to impute your reasoning, but actually, I'm not the teacher, or the font of wisdom" wrong.

Yes, people can simply be wrong. Lincoln didn't write internet jokes online. But, oftentimes, "wrong" is actually "I don't agree with you, but saying you're wrong is more win"

I tend to all three (factually, logical reasoning error, and opinion) wrongs. So I'm used to seeing all three flung back at me. There. Flung. thats emotive. Probably casts (ha) things in to a specific mode of reasoning...

> But if a person is supplied with accurate information

Unfortunately, this is often a matter of opinion, but not perceived as such, often completely sincerely.

I'm with both you and OP.

There are concerned citizens. I'm one of them. But there are also trolls and organized, paid government & industry shills.

I think having verified IDs is an interesting idea on HN and other niche forums and would help parse out the intention.

Could probably do it without doxing the public facing comments. Would put all the trust in YC though.

Maybe that's an interesting product idea, a way to establish real identity trust without any chance of exposing personal data to any parties, even the verifier.

I think it would help discussions.

I have my opinion there are shills here, specifically China related content and I think it would be interesting to know comments are minimally form a single human. Coordination is still possible though.

Any state actor and many corporates can generate fake ID on an industrial scale with trivial ease.

The real problem is more that FB and Twitter refuse to moderate known bad actors. It's not difficult to analyse posting patterns, but the FB/Twitter-plex often refuses to act on that information, and (...Cambridge Analytica) has often been complicit in its weaponisation.

I don't think it's a solvable problem. Or rather, it's solvable with good government and legislation against any form of organised public deception. Unfortunately getting the former relies on the latter being in place, which makes it a chicken and egg problem.

And it's harder than it looks because the worst kind of organised troll posts are weaponised for emotional triggering, not facts or logic. They're based on known psychological techniques and they cannot be out-argued directly. Critical thinking is no help, because the techniques are designed to bypass it.

So it would be hugely useful if there was some kind of online emotional literacy training which would explain and dramatise the techniques so people could be inoculated against them.

That aside - HN definitely has shills. I expect all comments about one particular corporation to attract downvotes if they're even remotely critical.

This is a pointless and unnecessary waste of everyone's time, but it's at least possible a PR firm somewhere is trying to justify its existence.

Other operators are less predictable, so it's harder to tell.

This is often a problem. I came across an old acquaintance on FB this week making some very troll-ish points on a topic. If I didn't know her I would have assumed the account was fake. But so far as I can tell it's genuine, and - unfortunately - she really believes what she posted.

Oh yeah FB & Twitter just shows the way that targeted crazyness (oftentimes from otherwise smart people with an agenda trying to manipulate) spread and create true believers. Is it trolling if that's what you truly believe and have passion for?

And I agree. The education in this country is sad. Critical thinking just doesn't exist, i've experienced it with otherwise intelligent people too. A huge % don't think to google something or learn themselves. They need specific guidance hand holding.

In the US it's hard because one person's organized deception is another's political campaigning.

Though personally I think there is a line of malicious intent, kind of like libel if a person knowingly spreads BS to achieve some goal. Maybe we have to look at the damage of those words. Like the modern day version of yelling fire.

Maybe HN could do something like a verified badge, still without revealing true identity unless someone chooses too. Sign your github repo or linked in or something.

> specifically China related content

But from which angle? Pro- or anti-?

I don't think it's likely that most/any HN commenters are paid trolls, there are lots of Chinese and Americans in this industry that will naturally have different perspectives.

My opinion, i obviously have no data.

But every single thread on CCP in particular gets overloaded with pro-china comments which use classic troll tactics. Often taking stances like for instance on privacy that are 180 from almost every other non CCP thread.

False comparisons, whataboutism, etc. I just have a hard time believing some of the more extreme stuff; like if someone really does have such a crazy opposite worldview where they think oppression and interment of millions is acceptable for whatever <excuse> among other stuff. Maybe my hard time believing is just a naive misunderstanding of the world but it rubs me wrong.

And lots of downvotes - which seems to go against HN spirit it's not reddit to downvote what you disagree with.

Maybe that says more about your bias.

There's an anti-china thread on HNs front page like 3x a week. Imagine if you were an immigrant and you see that shit. Would it be infuriating?

If a comment comes from outside your frame of reference, it might sound like bad faith to you while you sound bad faith to them. The country that spent the last 20 years killing millions of Muslims really cares about Xinjiang now? Really? And if they point that out, you cry whataboutism?

the problem i have is the frame of reference is so crazy extreme, in such abundance. but yeah i'm sure there are real people who really believe that. after all even if there is a good amount of organized shills, it supports a government/leader who probably buys into their own stuff.

It's very different from your frame, it's normal to them.

You and I grew up with American history and civics classes teaching an American view of things. There's context and complexities to American misdeeds. China's are presented in a vacuum, with undue focus and a touch of exaggeration.

Yeah maybe. I definitely have a strong opinion on CCP & authoritarianism specifically.

But I don't think interning millions of your citizens & committing genocide needs much context or has much complexity. Even considering the attacks that are used as an excuse to perpetrate the crimes. Even disregarding reports of 3rd reich behavior like forced sterilizations.

I just can't buy into there being hundreds of millions of informed people who think 'yeah that's acceptable' or even good AND that they are so widespread on english language western sites actively pushing back with troll behavior and downvoting etc.

Maybe there are and I just don't want to live in that world..

Also informed is interesting in that context.

I agree in some of our biases. Like we view China as pretty locked down and indoctrinated 24/7, but it does seem lots of commenters on HN talk about having more freedom & individualism than we assume.

FYI there is as of now no evidence for genocide in Xinjiang. No mass casualties. Sterilization is a universal policy for exceeding child allotments, which were actually higher for minorities until they raised it to 3 universally.

That's exactly the kind of thing I'm talking about.

Concerning oneself with poster's identity (honestly, feels like a form of ad hominem) can't be exactly the solution.

It could be that relationship to the source matters - rather than author's identity. I mean, there is a subtle difference between "who wrote this?" and "do I trust this?". But the only implementation I can think of is some sort of web-of-trust and all attempts at implementing WoT I know about had utterly failed.

Either way, I believe some form of memetic immunity - not getting provoked - rather than attempts to identify and shut down "troll farms" and "fake news" - is an ultimate solution.

This is how I approach the Internet as a whole. Treat unverified anything as if they are the enemy trying to hurt you. Be skeptical. It's a real shame too because the Internet is one of mankind's crowning achievements but it has this taint.

Could you get any more “verified” than a recent president or be any more of a troll?

I'd say anonymity is sufficient but not necessary :D