PhysicsForums and the Dead Internet Theory (2025)
j / k
|
Down or up a comment |
h / l
|
Previous or next link |
?
|
This help |
Using real member accounts to post the fake AI content is so much worse.
I can’t even imagine the thought process that led to thinking it was a good idea to generate fake posts but assign them to real users. If they were going to fake posts, why not fake the users, too? They turned one bad decision into a much worse decision.
It must be sad to see your proud little corner of the internet decline in popularity, but this move took whatever positive historical vibes it had and lit them on fire.
Seemingly that can be found here: physicsforums [dot] com/threads/official-response-to-allegations-of-ai-misuse-in-pf.1078532/
> During the past several months, to counter declining engagement, I experimented with generative AI, intending to add value to "thin threads", especially those with 0 replies.
> For experiment #3, I used real accounts that I considered to be decade-old spammers as best I could tell, but I can't be 100% certain. I categorically regret and reject such a shameful and unethical decision.
(that website also modifies your clipboard when you copy stuff, big fuck you to the site owner, seemingly lacking any sort of guiding compass, good riddance to them)
I’ve seen niche forums in the past insert affiliate links, so when somebody clicks on that HP graphing calculator they go straight to Amazon and the site operator gets a 2% commission. This was the approach of a website that plagiarized Obscure Sorrows and was discussed here on HN (https://news.ycombinator.com/item?id=48611823).
And then there’s tinfoil hat territory: massive influence operations that span multiple forums and Wikipedia and random blogs to support specific points of view about scientific claims, preferred technology approaches tied to commercial platforms, and even historical claims such as who or what developed a scientific concept.
Yeah, that part seems not just ethically bankrupt, but also really really stupid and counterproductive?
It all made a bit more sense to me when I learned that the guy who did all this is not, in fact, a physicist like I had originally assumed, but rather an "SEO lead at Shopify". [1]
[1] https://www.mkedmc.org/people/greg-bernhardt
I already have my own AI accounts. Several of them, really. Some of them paid. For work and for personal. If I want any kind of take from AI, which is in fact a very frequent occurrence, I can just prompt for one. For free even! And with my choice of model based on how important or complex the topic. There is zero value-add for me when some product decides to squirt an extra helping of AI somewhere it wasn’t needed. Summary of a long piece of text (while still giving access to the unabridged text)? Probably fine. An AI answer to a question someone else posted on a forum? Or injected above my web search results (looking right at you, Google)? No thanks!
You’re basically saying that if “you have a problem with something, then it’s your problem” - well shit - that’s exactly the meaning of having a problem with sometimes and you’re literally restating it saying it’s in fact not a problem— presumably because you want to be contrarian on the internet, or why is that not a problem?
There is complaint, like someone complaining that they don't like your new haircut, but it is ultimately not their haircut to worry about. How is your apparent poor choice in style a problem to the onlooker?
Let's imagine that a tapeworm is living inside your body. You might complain. But I'd see nothing wrong. Only users with tapeworms, who also objected to being parasitized, would have grounds for complaint. (Keep in mind, many of them might not have any symptoms yet.) So, with no reason to complain, maybe the rest of us should conclude that tapeworms are beneficial and everybody should get one.
So I think some commenters here are saying, no, tapeworms sound like a health problem. But they're revealing more about themselves than about their understanding of the big picture. To me, no judgement, but those commenters come across like human beings. If I'm a worm, these "health problem" complaints are the problem. The solution is for people to keep quiet.
Right. Your new hairdo comes into question because we trusted you to look good, which is why we wanted to be around you, and you violated that trust. However, it is your hair to fashion as you please, no? What is the significance of us complaining about how you are now ugly? What does it mean in the grand scheme of things? What makes it more than idle gossip that will be completely forgotten next week?
No, you're not describing the problem @nyeah explained in their comment.
If you really want to torture your hairdo analogy into describing the problem, here's what it would look like:
You cut hair for people, take photographs of their hairdos, and put them up in the newspaper ads for your salon. Now your ads are suddenly showing photos of people who used to go to your salon, with Photoshopped hairdos and forged haircut dates. Your ex-customers are upset.
That reads more like the second half of the original comment, while this discussion, as quoted, has clearly always about the first bit: "Using AI to fake posting activity is bad." Let's not change the subject. A discussion cannot be had if you start inventing new context out of the blue.
In an effort to try to desperately salvage your salon analogy, this is like the salon showing photos of wigs and you becoming upset foolishly thinking it was someone's natural hair even though you like what the wig looks like and it has inspired your next, not so ugly haircut. Why is that so upsetting? If you really don't like what you see, why not move on and go the other salon next door? What is to be gained from crying over a business?
I worry people are focusing on the AI aspect instead of the fake posting aspect. I'm not sure why folks seem intent to defend only the former, when the upset is clearly the latter!
If you were right about the change of context, I would tell you to take that up with @nyeah, because my reply to you was about how you misrepresented their comment.
That said, there was no change of context. Your assertion that the discussion "has clearly always been about the first bit" is factually incorrect. Both @xp84 and @nyeah made it clear that the issue they've raised is not limited to the "fakeness" of the posts, but that it also includes the "realness" of the users who were being impersonated.
> In an effort to try to desperately salvage your salon analogy [...]
No, I don't think I will continue to engage with your attempts to misrepresent this discussion. If you had done that once or twice, it could have been an honest mistake, but the pattern you've established indicates that you're arguing in bad faith.
Then you kept doubling down on there being nothing wrong with passing off AI activity as human activity. At every point, you either ignored or failed to understand any attempt to explain that, no matter what words were used.
To top it all off, you are now claiming that the point about taking over human accounts to deepen the deception is somehow "unrelated to the discussion" and brought up in bad faith.
I can't think of a way to respond to that while conforming to the HN guidelines of "being kind" and "conversing curiously", so I'll stop here.
The original comment asserted: "Using AI to fake posting activity is bad." So, yes, I agree that "bad" is ill-defined, but our discussion, aside from your ridiculous diversions, has been about trying to break down what that really means. If it was already well understood there wouldn't be a discussion to have here.
Now, we don't know what you think it means since you don't seem to want to participate in good faith, but what it doesn't mean is some idea of 'taking over accounts', as the comment in question goes on to talk about that later as a separate idea.
> Then you kept doubling down on there being nothing wrong with passing off AI activity as human activity.
I'm not sure what you're getting at here. Not only does it not seem to have any relevance to the discussion, we currently lack the technology to make AI look human, so, while that may become a future problem, there is currently not a concern. Currently we only ever see AI in places where one is able to see, without ambiguity, that they are looking at the product of software.
Whether or not that hypothetical future needs to see passing AI activity off as being human as being "wrong" is really for the future to decide. We cannot realistically imagine what other conditions might exist in that time to consider the full picture. AI being able to appear human, if we ever get there, will not be the only evolution in technology and society to take place. It won't be today's world with one variable tweaked. Looking at it through today's lens is foolish at best.
The key point is that nobody cares, and nobody should care, whether the humans all get infected by tapeworms. That's the worms' business and nobody else's.
A good product is a good product. The implementation details don't matter.
Are you excited to generate an account and communicate with experts such as the ones in the article today?
Where? I admittedly hadn't heard of PhysicsForum before this, so I don't know what it looked liked before, but at time of writing it seems to be structured a lot like HN or most other online forums where there is no semblance of human activity, just computer avatars. Last time I checked human activity comes with humans. Anthropomorphizing software does not make it human.
> Are you excited to generate an account and communicate with experts such as the ones in the article today?
If it is interesting and enjoyable, why not? I don't care about how the software works. I know HN accounts tend to get caught up in comments expressing things like nobody will use software unless it is written in Rust and stuff like that, but interested parties of a product don't think like that. They evaluate a product for what it is, not how it works beneath the sheets.
If you are asking if PhysicsForum in particular is a good product, I cannot meaningfully answer. It is not something that would have floated my boat at any point. From my point of view it never was, but it is apparent that sentiment hasn't been shared, although apparently there has been a meaningful decline in interest so it seems it hasn't held up as a product over time. Key here is that whatever controversial changes were made were made because interest in the product had declined so much already.
What's curious is that nobody was complaining about how bad it had become amid that long decline. They just walked away when it wasn't good anymore. Yet here we are, where you can still walk away all the same, but somehow that's the end of the world?
In my mind this destroys the core value proposition of the site: reputation. If this is a clever pivot than I could use some help understanding the intended direction.
The question was: Where is anything on PhysicsForum presented as human activity? Because I don't see it. It is modelled like most other online message forums, including HN, which explicitly remove any suggestion that there could be humans involved behind the scenes. Which is the whole point! Online forums are for when you want something other than the human experience. When one wants the human experience they go where humans are found.
> If this is a clever pivot than I could use some help understanding the intended direction.
Allegedly the site was effectively dead and this was the operator's attempt to resurrect it. Was it clever? Probably not, but if the operator was clever it wouldn't have reached the point of near death in the first place. What are we expecting here? A miracle worker to come along and give the operator supernatural powers to abate his natural limitations? Not everyone is going to be as smart or capable as you are.
Right, that's the question I struggle to interpret as good faith, but I'll try to answer! For me, the whole concept of online forums is interacting with semi-anonymous humans (and I believe that's why many (most?) ban bots), but also the slogan is "science by humans" and that seems to be their general brand direction:
https://www.physicsforums.com/
But I worry the focus is on the word "human" too much. My complaint is attribution, which applies to any account on the site, human or otherwise.
Since you continue to sidestep the implications of my rhetorical question, I'd ask it non-rhetorically:
Why would anybody make an account on an "expert discussion" site to be misrepresented? Why would anybody, be it you or me or whoever you like, want that?
This is where things break down for me. You want to take humans out of the equation by making any humans that may exist in the process anonymous, to disassociate their involvement, but then worry that the anonymous computer avatars may not be human?
I could at least have a bit more appreciation if you were talking about Facebook Groups, where there is greater effort to make those computer avatars appear human (real names, photos, etc.), but PhysicsForum, HN, and the like chose, seemingly on purpose, to not go in that direction.
If you really want to have discussions with humans, which is valid, why not go outside and meet them where they are found?
> but also the slogan is "science by humans"
If an automotive forum's slogan was "All About Cars" you would disavow any human who uses it? Science by humans can be discussed by non-humans, no? Something and the discussion of something are logically not the same thing.
> Why would anybody make an account on an "expert discussion" site to be misrepresented?
What do you believe is being misrepresented when AI content is posted on a forum? There is no promise of human activity on the typical online forum. To return us to the top, since maybe you've lost sight of what we were talking about, "Using AI to fake posting activity is bad." — but it remains unclear to me why it is bad. I can see how it might end up being low quality "slop", but that is for the users to decide. There is the possibility that users don't find it to be of poor quality and actually enjoy it.
> Using AI to fake posting activity is bad.
This is not what I'm saying! I think I've been quite clear on this point. It's also not the intent of the original comment you replied to, which made clear points beyond the sentence you quoted. I'm not really sure who you're talking to.
It is not what you said, but it is the context that defines what we're talking about. I reiterated it to remind you of what the original reply isolated from a bunch of other ideas that were also thrown around as it was starting to look like you had forgotten what we were discussing, perhaps confused by parallel discussions that branched off in different directions. This seems to confirm that you had forgotten, although perhaps that is a misinterpretation on my part. So that we're clear, are you implying here that you've been trying to change the subject in bad faith?
It's clear you're either unable or unwilling to communicate as equals. Have a good day.
The larger context is a completely different discussion, and one that isn't particularly interesting to have, at least on my side. It wasn't narrowed arbitrarily. I can appreciate that you might find that other topic interesting, but there are plenty of other parallel discussions taking place about that. You don't need to shit up this one with more of the same in order to engage with it. Let's leave the bad faith at the door.
https://news.ycombinator.com/item?id=42816284 - Jan 24, 2025 | 154 comments
It's ironic because the article (and many comments in the previous submission) are about how one shouldn't mess with user data to show fake engagement!
In theory as the models improve their ability to ape human writing the effect should diminish, though iterative reingestion of their own output is probably still detrimental long term.
For millennia we created documents supporting the geocentric model, all wrong! How'd we break out of that rut?
I don't know if AI are currently equipped to deal with this as humans did, but maybe they can be?
I only talked to Greg remotely and never met him. This is a huge betrayal of trust. Older moderators have died since then. This is a very longstanding community. Even if we didn't stick around forever, many of us became real-world friends. Adding autogenerated summaries is one thing, but inserting comments in a real person's name that they never made, when you have no knowledge of what became of that person, who may even now be dead, is beyond the pale. You're rewriting history. The fact that this site still exists at all is extremely gratifying. It's a huge part of what has made me who I am today. It's a big part of why I don't really pontificate all that much, don't have a blog. I read what I wrote there nearly a quarter century ago, and it's not all bunk. A lot of what I said holds up. Plenty of it was raw factual data never in dispute. But when it came to politics and economics and forecasting of the future, I said things so wrong and thing I so strongly disagree with now that having that record teaches me how flimsy and full of shit so many of our opinions are, no matter how smart we are and no matter how strongly we hold those opinions. I've 180'd things I was completely certain of during the first Bush administration. Heck, you can add voting for Bush to that list. There is no way in hell I would do that today.
History matters, Greg. It can both humble and enlighten us. But only if it's true.
> Adding autogenerated summaries is one thing, but inserting comments in a real person's name that they never made, when you have no knowledge of what became of that person, who may even now be dead, is beyond the pale. You're rewriting history.
> History matters, ... It can both humble and enlighten us. But only if it's true.
Reminds me of George Orwell's writings. Specifically Notes on Nationalism - https://en.wikipedia.org/wiki/Notes_on_Nationalism Here "Nationalism" is to be interpreted as more akin to "Tribalism" i.e. defending one's group at all costs and against all evidence/facts/truth as needed.
All nationalists have the power of not seeing resemblances between similar sets of facts. ... Actions are held to be good or bad, not on their own merits, but according to who does them, and there is almost no kind of outrage – torture, the use of hostages, forced labour, mass deportations, imprisonment without trial, forgery, assassination, the bombing of civilians – which does not change its moral colour when it is committed by ‘our’ side.
Also this is a good example of the trend that has already been ongoing on HN since 2022. I think one other long-lasting HNer is working on a similar analysis of HN at the moment.
Edit: can't reply
> Appreciate some more details...
It's their story to tell. Not mine.
There was also an experimental period where they’d banned all political articles, and then they reversed it.
Appreciate some more details.
Due to non-linear effects of propaganda, a single/small-group of people working in a concerted manner can easily change group dynamics specifically its "fairness perception" and thus ultimately lead to the detriment of the group itself.
But ultimately a site moderated by up and downvotes that wants to keep a relatively light hand on moderation is going to end up like what HN is today. Changing the voting structure on the site or the moderation POV on the site would be a radical change and could change the entire site's culture drastically.
The stock answer to the above charge is that "this is more of the individual's perception and not reality" and "HN is just representative of what is going on in the society". Also;
Ask HN: What is Hacker News's business model? - https://news.ycombinator.com/item?id=23671531
Humans will game everything they can to gain an advantage even if it were only for silly imaginary things like "upvotes". Add in our psychological propensity for "Tribalism" and you can see we have to be a bit paranoid when it comes to social networks. Introduce anonymity into the game and things get even more worse.
So it behooves the HN owners/mods to have some sort of system to analyze everything and do some inferences for insight even if they don't act on it immediately.
Some users have already developed some models and shared it on HN; eg:
1) Show HN: Exploring HN by mapping and analyzing 40M posts and comments for fun - https://news.ycombinator.com/item?id=40307519
2) Mapping Hacker News to find who knows what in the HN community - https://news.ycombinator.com/item?id=41069527
3) Show HN: Hacker Atlas - A map of what Hacker News talks about - https://news.ycombinator.com/item?id=49844497
4) Hacker News Data Map - https://news.ycombinator.com/item?id=42035981
PS: There are a bunch of books on "Social Network Analysis" which one can consult. A good one is the book of the same name by Tanmoy Chakraborty.
I've been reading HN since 2013 and the quality has gone down, kind of like eternal September.
HN essentially became mainstream after the New Yorker profile, Reddit's degradation due to the mod blackout and API changes, and Lemmy flopping.
I've gotten my HN back to the pre-2022 consensus but it takes severe filtering, with around 200 posts being hidden a day. And it's those posts that drive the majority of discourse on the platform now.
The fact that throwaways like you are so normalized is a huge reason.
HN has also gotten divorced from tech and water-cooler conversations in much of the tech industry, and a lot of that seems to be because HN no longer has the same plurality of Bay/NYC/Seattle/Austin techies that it used to, and the userbase is way older than much of tech now (I was always younger than average on HN, but the amount of 1980s to early 2000s references shows that the platform has frozen in time demographically).
Ignore Reddit - HN has functionally become Slashdot (the fact that I even know what slashdot is highlights how the age demographics are so skewed on HN).
Edit: can't reply
> 2022 is when tech funding dried up...
Tech funding didn't dry up then. It just coencided with the time period that most vintages that originated in early 2010s were closing out and those that originated in 2016-18 were hitting their 5 year mark.
This meant a lot of dry-powder didn't get deployed until 2023-24.
> stands to reason that we ran out of hacker topics...
They are still being posted at the same rate based on my filtering. The difference is a massive deluge in non-"traditional HN" related content being posted, and the younger generation of techies and founders (those below 35) not using HN.
HN was a product of its time.
--
> They are still being posted at the same rate based on my filtering.
"I rewrote X in Rust" isn't a hacker topic. Being able to draw some kind of connection to the tech isn't the same thing.
What tools do you use for filtering if I may ask?
> I don't want certain types of users to know anything about me. I wouldn't have felt this way, say, 5 years back...
Frankly, most people DGAF about random HNers and (a general or meta-comment targeted at the reader, not you OP) if someone needs to use throwaways to post negatively polarized statements on HN, then they shouldn't even be on HN in the first place.
It is never most people you need to worry about.
> if someone needs to use throwaways to post negatively polarized statements on HN
If it were only read by other HN users who understand that a user account is not a human then it wouldn't matter, but the world is not so simple. There are people, who do not understand that forum accounts are not humans, who are working hard to associate internet activity with humans and will use it against them.
Of course, we already know how to identify individuals by their writing style, so a throwaway account probably isn't the screen the OP hopes it is. Still, it is the best source of making it as difficult as possible we seemingly have.
Which is to say the Internet can be "dead" without being filled with undetectable or detectable fake posts, not that fake posts help things.
If the dead internet theory were true or real, Google and Facebook would not generate so much revenue. Advertisers would notice that they are not getting any conversions and stop spending.
Are people responding to submissions, reposted by bots, a genuine interaction? It is one kind to know there are some bot accounts on reddit, and other to experience it first handed. For example, recently some karma farmer posted half of original post, and comments that appeared under it still referenced second page, because entire threads also got copied verbatim from previous submission, using different accounts.
It really feels like reddit admins don't do enough to combat that behaviour. Maybe, just like admin of PF, they think fake activity is better than none at all?
Many years ago I did a search for my name and turned up a LOT of things I did not actually write. This wasn't LLMs, this was scrapers doing a poor job of scarfing up old content--comparing what I saw vs my recall of the discussions there was a lot of stuff that lost the quoting, words were attributed to me that were actually quotes I was responding to.
Heck, you could even engagement-farm if you have to, by taking science stories in the news and say, generating a poll to try to spur discussion.
But what they actually chose to do is super gross, helps nobody, hurts the site’s reputation, and even devalues the site tremendously as possible training data for AI. (Which, some AI detractors might actually be happy about, but I think isn’t really a good thing.)