2026-03-26 17:53 UTC
Replies (11)
-
@jamey@toot.cat 2026-03-26 18:21
@regehr@mastodon.social If we build tools that actually give us zero degrees of freedom, surely there are more efficient and reliable ways to use them than LLMs? Given that, as you note, zero DOF is only aspirational, I would love to see more work along the lines of the Termite project for synthesizing device drivers. Version 1 took the provided constraints on the behavior of both the device and the OS, did a bunch of computation, and tried to spit out C source without human intervention. Termite 2 took the same inputs, but gave developers an IDE that would auto-complete large chunks when there were no valid alternatives, then prompt the programmer for the few decisions that were left. I think there are lessons I'd like to see more people learn there.
-
@sree@ublog.thirdlaw.net 2026-03-26 22:49
@regehr The concluding sentence is exactly what I'm excited about. Software will almost certainly get better with better tooling, AI agents or not, but now there's going to be a ton of money chasing better SE tools (I hope).
-
@regehr@mastodon.social 2026-03-26 20:51
I do enjoy using these things in ways that don't plagiarize and don't inflict slop on people. for example, when debugging an LLVM pass, this kind of thing can actually work. like, this actually found a miscompilation bug just now, and may find more if I ask it to keep going. it's an absolutely fresh LLVM pass, there is no chance that the LLM was trained on anything it's seeing here (much less on bug fixes to it).
-
@bob@feed.hella.cheap 2026-03-26 18:31
@regehr I had a similar thought and I was thinking writing constraints in something like JQF might work for making writing oracles that encode knowledge about your problem low friction
-
@whitequark@social.treehouse.systems 2026-03-26 18:35
@regehr meanwhile i'm reconsidering the kinds of tools i'm building. if i don't add types to my project it will result in fewer slop submissions => maybe i should do that?
-
@JamesWidman@mastodon.social 2026-03-26 18:51
@regehr it seems like people who work in biology departments could have told us that this was coming: LLMs are to random mutation as oracles are to natural selection (like how a fast predator effectively selects relatively-faster prey for breeding).
-
@kbeyls@mastodon.social 2026-03-26 20:09
@regehr Thanks for writing this! It feels close to how I started thinking of LLMs as "Kahneman system 1" thinkers (stochastic, intuitive, ...) . So you need "Kahneman system 2" tools (deterministic, analytical, controlled, ...) to control, verify or shape their output. It sounds like "executable oracles" are exactly one such kind of "Kahneman system 2" tool...
-
@jawnsy@mastodon.social 2026-03-26 20:56
@regehr I'm really curious what alternatives fit: "By this point, most of us who have experimented with ________ have noticed that the current generation of these can sometimes do good work, at superhuman speed, when given some kinds of highly constrained tasks." Fill in the blank: 1. Using child labor 2. Supervising graduate students 3. ???
-
@kworker@kolektiva.social 2026-03-26 22:34
@regehr wow, a refreshing look at LLMs that isn't simply marketing hype! Really appreciate it. Interesting to see the kind of work and strict scoping this takes. Also seems like IMO LLMs still really don't outweigh the negatives & cost
-
@mirth@mastodon.sdf.org 2026-03-26 23:01
@regehr In a way this idea of zero DoF is like the proverbial sculpture in a block of marble. You must carve away the space of tool-accepted code until the remainder has the shape you want. Very different from manual coding which often feels like assembling the thing by gluing grains of sand together one by one, or normal templates and scaffolding which have a sort of rigid and fixed shape.
-
@lritter@mastodon.gamedev.place 2026-03-26 23:39
@regehr article ok but. i don't understand why these metric-providers need to be called "oracles". it's the same pretension as the use of the word "story" in ticket systems. oracles deliver divine prophecies. what these things do is underwhelming in comparison. it irritates me greatly.