HackiNews

• woodrowbarlow
  in a post-truth world where the internet is full of non-obvious ai-generated mostly-but-not-quite-true information, academic journals should be more important than ever; unfortunately, the companies that run them had already become complacent and negligent long before LLMs came on the scene.

  • 10111001
    > in a post-truth world where the internet is full of non-obvious ai-generated mostly-but-not-quite-true information, academic journals should be more important than ever; unfortunately, the companies that run them had already become complacent and negligent long before LLMs came on the scene.

    There will have to be institutional adaptation to this, but it will be painful and I think the end-result will be worse that what we had pre-LLM.

• Lerc
  >When one model refuses or stalls, I switch models, rephrase prompts, and ask what the next step would look like “hypothetically.”

  I wonder If the hypothetically criterion gives the models carte blanche to make up references.

• Kim_Bruning
  The opposite is more interesting, (and the author points it out too). SOTA agents can help run small experiments and help you do the writeup. Should you publish it? I'd personally want to keep a human at least partially in the loop, or have very solid checkers. But for one off personal research it's at the same point as one off personal code: make sure you have a way to test what's valid; but if you do, it often Just Works(tm) Now.
• jszymborski
  Sidenote: way to flex your "writing studio".

  • lifeisloving
    A shed, but a cool shed ig.
• purplekohav
  Even though these experiments got rejected, I feel like if he had used better AI versions, they might have worked.

  Also, kinda scary to think about how many scientific papers are probably written using AI these days. The system might have to change.

• MarkusQ
  This whole issue seems to hinge on a fundamental disagreement about what science (or mathematics) is. Papers aren't science, they're the receipts. But too many people apparently can't tell the difference, so we have the present blizzard of forged receipts (first from paper mills, now with new lemon scented AI++).

  Mathematics and science represent two systems for sorting truth from BS. An exponentially increasing supply of BS makes that even more necessary.

• runamuck
  Whenever I read "Here's" in a title, I immediately assume that AI wrote the content.
• psychoslave
  You can't build trust over science, that's just an error of category. You can only build trust in some people on some topic. One can measure reliability in a process, but if no specific human is involved, there is no literal trust engaged.

  Now of course, to each its definition and metaphors are useful tools.

  But when one starts a discussion on truth, trust or reality as core topic, it's better to start with what is meant with these words and in which context to which scoped boundaries.

  Because of course, science can back a contemporary etnologic account that asserts there is no (longer) different races in homo sapiens, just as much as it can serve as frame for phrenology and whatever supremacism claim.

  So no, we didn't build trust in science over centuries, and not even in any scientific person because we don't have that much humans to hold the role of such a trustable entity over such a long period of time.

  Social institutions can carry unchanged names over centuries, but that tells nothing about how trustable are the people that inhabit them over time.

  LLMs certainly come with their own challenges, but the erosion or scientific institutions with its bizarre scheme of publication process have been there basically as far as I can remember.

• dist-epoch
  There is still no paper or notable result in gender studies published by an AI.
• kirth_gersen
  Save yourself a slog. The automated checks found that the citations didn't exist and flagged the "papers". A better title would have been "Automated Systems seem to hold back the AI Slop papers. For now." Sweet writing studio, though.
• scotty79
  In other news people prompted a model to pick a problem to solve, solve it and write a paper about it. And it did and the paper contains novel contributions.

  https://www.reddit.com/r/OpenAI/comments/1xgqv27/two_physici...

• paxy
  TLDR - both papers got flagged and rejected by automated integrity checks.

  • red75prime
    The first paper (free-tier chatbots) due to fake references, the second paper (a specialized AI-research tool) due to overreliance on preprints.
• VCFundedGenYer
  Please, even if it is an experiment, don't do this. We don't need more AI slop in the world.

  • pizza234
    This experiment was agreed with the managing editor of the multidisciplinary and open-access journal IEEE Access, and IEEE itself - it was red teaming, essentially.