The tool on its own networks
The requirements for each learned part are a subgraph of their own: one use-case model per part, with the stats it reads and records as a closed list inside it, all joined in one graph of 3,298 nodes and 8,863 edges, where 30 stats are shared across parts. The loop then ran on each part, breaking its code, and each part was run on a graph of itself.
What it measured, on 30 October
Measured No part beat the plain rules on its own code. Who owns new code: 0.65 against 0.83. Which tests catch a break: 0.83 against 0.92. Which steps carry a requirement: 0.66 against 0.81. Which breaks get past the tests: 0.15 against 0.19. Two were too thin to say: which breaks change nothing, and where to add a test. One seed each, so these are not means.
Measured 48 requirements, 9 of them inferred. 144 stats, used 233 times across the parts, 32 of them read by nobody. 1,174 of 1,816 breaks caught, 642 got past. In the first, partial run, 115 of the 369 catches came only from new guards written from the requirements, because the tool's own selftests had passed those breaks.
Measured Every break has a verdict: none is left unrun. The first run left most of them unrun, so its counts were lower bounds; these counts are not.
What it found, none of it fixed yet
- With the guard that keeps the answer out of what a network reads removed, the network's own selftest still passes. Only the new guard catches it.
- One win is probably not one: which breaks change nothing, on the tool's own code, learned from labels written before a fix to how breaks are compared. Inferred, not re-run. On both projects every example it learns from is a break a pass already removes, so its score says nothing about the survivors a person reads.
- The bar for showing a proposal has no margin, so 0.51 against 0.50 passes, and it leaves out the plain rule that actually wins on one project.
- The run plan reads only the last row, and the row names no seed, so two losing seeds and one winning one switch a part on.
- The code that writes a score is not held by a test: 32 of 32 breaks of it got past.
- The Python adapter only breaks statements inside functions, so the thresholds and the guard list themselves were never broken by it.
Running the rules on the improvement model itself, a level up, is still open.
Made by RJ’s TNT Tools. TNT Tools’ own work, not a Heavy Iron product.