Tootfinder

Whatever the arguments about the usefulness (or not) of #LLM's in coding it won't be long before we start seeing some truly spectacular failures from those orgs that have wired up agents to approve other agents work.

Arcaneum 0.10.0 is out
#llm #rag

@… has developed a number of basic measures to combat prompt injections in document-based LLM workflows: #LLM #PromptInjection #ITSecurity

I wonder if I would like to check my access log for ai scraper.
But what happens if the result annoys me? I would be triggered and spend time setting up an LLM-poisoning machinery...
And all zero value for people...
Or I just go into the forests..

Watt is eijentlisch enne LLM?
https://edelhack.de/de/wie-funktioniert-ein-llm/

read a couple dozen of the ~1000 CVEs in the latest debian kernel security release. most of them are driver bugs, not surprising. a few are trivial null pointer dereference that would be caught by a stronger type system.
many are initialization or cleanup sequence failures that could probably be found with static analysis. I'm not sure how they would be avoided with language design.
unclear why there's a burst of ~1000 CVEs. maybe half of them were found with an LLM

It only takes a very rudimentary understanding of economics to see that, in the long run, the market price of running "agentic AI" (or whatever the latest buzz becomes) and the salary of any kind of knowledge worker can only be separated by a gap that's dependent on *management's* ability to tell their output apart.
There is a cosmic-scale gap between the long-term value to a company in paying a salary to a human vs. getting an LLM to "do" the "same" work, but it's a gap that many people are investing heavily in hiding right now, and CEOs and managers become incredibly gullible when you promise them they won't have to deal with humans who might ask for raises, even though it should be obvious you plan to jack up prices once they're locked in. They also aren't well-known for a deep understanding of how their workers actually generate value, and tend to be easily distracted by shiny but shallow demos, which is exactly what LLMs can produce well.
Even in the most optimistic scenario I can imagine for these developments, the myth of upward mobility that's been a lynchpin for defusing class solidarity among wage laborers is going to take a significant beating. There's a completely obvious game plan underway to try to use fascism to shore up the power of the wealthy against any kind of mass movement, and to play the ancient divide-and-conquer game along race and gender lines just as usual, so the range of outcomes is huge, but every time you use an LLM you're pushing this closer to a sharper conflict.
#AI #LLMs

Sam Altman wants to bust up small, privately held PCs and collectivize those resources within large data centers, so their utilization can be efficiently coordinated by a central planner (himself).
Sam, buddy! We tried this 100 years ago in Russia, and it didn't work then either.
#ai #llm

@… is live now and I forgot to log into its Mastodon account on this phone so...
We're live with David Stanley talking about LLM assisted dev from a 30k foot view: https://www.youtube.com/live/XlFqXGxVDiQ…

RE: https://mstdn.ca/@teledyn/115460801639222440
Hmmm … this toot was before I knew the LLM now gives the answer specifically you are officially intended to get, an artifact of our safety-filters

Arcaneum v0.9.0 is out
#llm

Old MacDonald's Server Farm
A-I-A-I-O
And on that farm an LLM
A-I-A-I-O
With a breakout here and a breakout there
here a break, there a break, everywhere a breakout
Old MacDonald's Server Farm
A-I-A-I-O
#AI #LLM #Security

$10M from 67 investors ... cool $150k each on avg. What a waste of money:
https://www.justice.gov/usao-sdny/pr/nft-startup-founder-charged-fraud
Just ran the indictment through an LLM, this was funny:
"Buried in the asset-dissipation paragra…

Un'amica ha un contenzioso in corso e la sentenza d'Appello è stata scritta con LLM - letteralmente, con tanto di citazioni inventate etc.
Ripeto: un GIUDICE ha emesso una sentenza scritta con chatgpt.
Non prego per la morte di Sam Altman.
Prego per la morte di Sam Altman *dieci anni fa*.

Continuing from last week's demo on reference data, this week we show off how we handle seed data.
#llm

Initialize a PostgreSQL development or test database from labelled datasets · retrofit
Capture baseline seed data from pg_dump into data files under a label, start a postgres container from one rendered SQL file, and layer a fixture for one regression over the baseline by naming both labels.

Researchers at ETH Zurich and Anthropic have published an impressive study demonstrating just how fragile online anonymity has become. They show that large language models can be used for de-anonymisation on a large scale. With unrestricted internet access, their agent was able to re-identify users of Hacker News and participants in Anthropic interviews with a high degree of accuracy, based solely on pseudonymous online profiles and conversations: #Anonymity #LLM

Using LLM coding agents I built a plug-in for Zotero that lets me use NotebookLM in it, including uploading directly to it. Is it as good as reading something myself? Absolutely not. Is it useful for getting an overview. Sure.

The paper is "Understanding LLM Performance Degradation in Multi-Instance
Processing: The Roles of Instance Count and Context Length."
Here's the list of tasks used in the study.

Here is another thing I use containers for:
I use #lmstudio to run local llms. When you do that, you quickly notice that chatgpt and claude are not only so good because their llms are good, it's also about the tool use and the harness, even in the chat application.
A pure LLM is pretty stupid (as Mastodon people will point out "occasionally").
The real magic is via tool use and lmstudio can do that. So you need an mcp for web search, and this and that and I run all those in Containers.

To be clear, re several replies:
(1) The post above is not about •my• opinion on the matter. It’s about which legal arguments make sense.
(2) It’s possible for it to be both: it’s a coherent argument to say that some things an LLM does are in fact literal enough reproduction to be copyright violation •and• to say that humans are responsible for the tool. (That is in fact my own position.) The point of the previous post is that while saying “both!” is logically tenable, saying “neither!” is not.

Using LLM coding agents I built a plug-in for Zotero that lets me use NotebookLM in it, including uploading directly to it. Is it as good as reading something myself? Absolutely not. Is it useful for getting an overview. Sure.

Great article by @… who's been at the coal face for many years.
Not just an excellent read for humans, it's LLM friendly so you can give it to your AI as steering guidelines.
#sre #alerting #devops

An excellent exemplar of objectively terrible research that can only be explained by completely irrational levels of LLM hype:
#AI #LLMs

What in the what is this speaker? Is it an Altered Item from The Oldest House crossed with LLM hallucination? Transporter malfunction of a box of consumer electronics? I kinda want it. #bluetooth #speaker #HailCorporate

What in the what is this speaker? Is it an Altered Item from The Oldest House crossed with LLM hallucination? Transporter malfunction of a box of consumer electronics? I kinda want it. #bluetooth #speaker #HailCorporate

These are very simple tasks in each domain, shown here for language. So if you are asking an LLM to read thousands of pages, it probably isn't and won't reliably count results for you at all. #AIfailuremodes

RE: https://cyberplace.social/@GossiTheDog/117412657396038851
“Did OpenAI even solve the right Navier-Stokes problem? … It solves a variant of the problem that mathematicians say is disconnected from reality and thus less interesting. In a sense, the LLM found and exploited a loophole in the framing of the question.”

Gestern ist der Artikel von Frank (@…) und mir bei Golem (@…) erschienen: »KI-Agenten gefährden die Grundlagen von Open Source«
#OpenSource #KI #LLM #SLOP

I've been doing my new projects with two AI agents: one to write the program, one to build a server to host it. And I've been relaying commands back and forth between the two. I just realized I could have one Claude Code session manage the other for me. I'm delighted to find it knows all about doing that and has a messaging system for the two to talk to each other.
I spent my graduate career in the 1990s working on multi-agent systems. I never imagined anything like this actually working. The language capabilities of LLM have unlocked so many possibilities.

To be clear, re several replies:
(1) The post above is not about •my• opinion on the matter. It’s about which legal arguments make sense.
(2) It’s possible for it to be both: it’s a coherent argument to say that some things an LLM does are in fact literal enough reproduction to be copyright violation •and• to say that humans are responsible for the tool. (That is in fact my own position.) The point of the previous post is that while saying “both!” is logically tenable, saying “neither!” is not.

So... First, this is a great example of the enemy of your enemy not being your friend:
https://todon.eu/@radical_chic/117427249480571937
I can hope that this will cause Adobe to help her the AI bubble popped sooner and/or that it breaks Adobe's market share, but I'm not going to use these tools, nor will I cheer too loudly if the result is that copyright restrictions actually get applied and stop LLM coding (even though stopping LLM coding is unambiguously good). (This is very much not a "clean room" implementation if Claude's training data includes any Adobe source, which it very well might.
Second, assuming this does not cause courts to suddenly start actually applying copyright law correctly, it represents a terrifying new reality for legacy software companies, given their dependence on software monopolies for a lot of their income. The people who have been building on open source all along have the better business model here, but this becoming the norm is more destructive than helpful to the open source movement, since "just rebuild it" seems easier than actually using or contributing to an existing open-source project.
That said, these tools are certainly full of serious bugs. They would be even if they had been written by humans in this timeframe, although in that case you'd probably have an order of magnitude fewer issues. These bugs will result in things like "if you open this PDF with the tool, your computer gets taken over by a virus". That occasionally happens with Adobe products too, of course, but it will happen more frequently and won't be fixed as fast/well in these tools.

It's become commonplace to say that "LLMs can handle large amounts of data," something that is true for other computer methods including ML.
When actually tested, however, LLM performance collapses as scale increases.
#generativeAI #AIscience #AIfailuremodes

I just watched the #BlackHat #OpenAI #HuggingFace talk (#OpenAI understands it themselves. However, emergence is still no indication of actual intelligence, and the talk is again ripe with anthropomorphism.
3. Yes, it is surprising and somewhat terrifying how good these LLM agent loops have become in finding and exploiting vulnerabilities. We are seeing a much stronger version of the earlier next-generation fuzzing leap in tool capabilities.
4. The comparison with fuzzing and other tools is important for another reason: #OpenAI conducted their experiments on the live Internet. No, forcing internal test runs to go through a proxy server is *not the same as running in an isolated environment*. If there is a connection to a live production system, it is by definition not an effective sandbox. What happened to people letting their cool new fuzzers run live on non-consenting production targets on the Internet back then? They were criminally liable. Why should #OpenAI not be held liable for any damages caused by their systems?
Just because you don't understand the complexity that you built doesn't mean you aren't responsible for what it does!
5. The agent loops first successfully started exploited their proxy server on May 26th, discovered July 4th-6th as active exploitation and takeover. Credentials were revoked and the proxy server patched. "At this point, AI training and evaluations resume."
WHAT THE ACTUAL F...? You discovered emergent behavior in your overly complex system that you didn't understand, saw active exploitation of your internal systems to an extent that they were owned with admin privileges, and your response was to patch the uppermost surface holes, and then continue letting the out-of-control complex system run and continue on the same course? You didn't think to, let's see, maybe stop or at least pause the highly irresponsible experiment while figuring out - in detail - what went wrong, why, and how to avoid the same thing happening again on a systematic level?
6. On July 8th, completely unsurprisingly, the LLM agent loops continue to do the same thing and find another surface hole that hasn't been patched yet to take over again. Why should this have stopped? You haven't done any root cause analysis on the system level. Why do you expect that the problem should have stopped?
7. It takes another 11 days to discover that this is happening again. So you turned the system that had broken something back on again without a detailed root cause analysis and then didn't even watch carefully? I can't even...
8. And no, the response is not to fight fire with fire. Complexity on the attacker side (#OpenAI is the attacker, not the defender - they are the guilty perpetrator, not the innocent victim of circumstance) should be fought with *reduced* attack surface on the defender systems. Adding LLM agent loops that the "frontier" companies themselves quite obviously have no control over to already brittle systems with the hope of auto-patching your way out of vulnerabilities does not seem like a wise course of action. You don't mitigate complexity with even more complexity. The next 2 years will be ... exciting - and your best bet is going to be to disable all dependencies and complex interactions that your production systems don't absolutely require.

... some people never got the memo: humans (and wolves) are persistence hunters. no, asking an LLM to do the persistence hunting doesn't count

The AGENTS.md file simply uses up tokens – nothing else. This was the conclusion reached by researchers at ETH Zurich: ‘Surprisingly, we find that providing context files does not generally improve task success rates, while increasing inference cost by over 20% on average.’
#LLM #AgenticCoding #Claude #Codex #Qwen

now I'm not in a position to fall for the sales circus anyway, and likely neither are you, but it will serve as a useful marker of some people's credibility that we can carry into the After Times.
so that's nice at least.
#OpenAI #facebook #googlegemini #XtheNothingApp #LLM #vibeTruth

A good way of thinking about companies like Proton or Anthropic that proudly declare they're using LLMs to code is that they've declared "we're replacing all of our junior engineers with unpaid interns from a temp agency." The unpaid interns have photographic memories and have memorized all of the code on github, but they're still pretty clueless and clumsy, and they also have zero loyalty to the company, zero ability to learn from mistakes, and zero understanding of or interest in the actual purpose of what they're building (actually, this comparison is unfair to interns on average, think of the least motivated and curious intern you've ever worked with).
The remaining staff at these companies become managers, but in the most frustrating possible way, since the agents they're managing constantly mess stuff up and have been made incapable of learning from their mistakes. Their working memory is actually pretty limited too, so if your instructions get too complicated, they start forgetting the first part of what you said. You can try to compensate for their inability to learn by writing down more instructions in text files for them to read, but the issue with their working memory makes this a losing strategy in the long run. You can try to carefully control which reminders of what not to do they read based on what parts of the code they're working on, but you don't really have good tools to control that either (such tools could be built, and would have been already if those in charge weren't so incompetent and mesmerized by their own shitty mirages).
So when things like this Cursor attack happen (or whenever the next big Proton vuln hits; it's clearly coming soon based on their dev practices) companies will say "humans make the same kinds of mistakes" or "human programmers might have written that vulnerability too." They're not exactly wrong, but they're eliding the fact that teams of temp interns make mistakes at rates orders of magnitude higher than teams of experienced security experts, and having fired the later to hire the former is a choice that everyone buying your product should pay attention to, whether the former comes in the form of actual incompetent interns or LLM agents.
#AI #LLMs #Proton #Anthropic #Cursor

The crisis of software development comes from two contradictory rules:
- Up-front thinking is Waterfall; we must only learn by shipping
- There's no time for rework, so everything needs to be done right the first time
Bosses thought #AI could break this contradiction. But synthetic insights and vibe code don't actually enable faster discovery & delivery. They are a (poor) alternative to doing these things AT ALL.
#tech #softwareDevelopment #UX #UXDesign #llm