Goals List

What the learned parts are trained for, and whether they got there. Trying for a goal is not reaching it: a goal counts as reached only when it beats a plain rule by more than 0.02, on code it has not seen, at most seeds. We are looking for patterns in which kinds get there.

10goals on the list
5trained so far
5 of 10results reached the goal on paper; 2 are doubted
3of those results flip with the seed

The goals

Who owns new code

Learns from: maps to. Model: graph network. Since 10-29.

Asked for by the project owner, for linking code to requirements.

Tried with calls as well: The tool's own code 0.66, One game feature 0.76.

Tried with caught by as well: The tool's own code 0.63, One game feature 0.78.

Tried with calls and caught by as well: The tool's own code 0.62, One game feature 0.77.

Its own selftest still passes with the guard that keeps the answer out removed. Only a new guard catches that.

Retired. station code_to_step is done by step_trace_route (tested code), owner_head (untested, with branch text) and owner_question (untested, nothing to read); the network never beat the plain rules on both projects; retired 11-03 by the owner's ruling (failed parts retire where a passing part does their job)

Not reached The tool's own code won 0 of 5 seeds
This goal
0.65
Plain rule
0.72

The plain rule here: what each method is seen to do.

Not reached One game feature won 0 of 3 seeds
This goal
0.77
Plain rule
0.84

The plain rule here: a count of which tests catch its breaks.

Which tests catch a break

Learns from: killed by. Model: graph network. Since 10-29.

Asked for by the project owner, when asked what else a graph network could do.

Retired. station what_tests_catch is done by break_testing: the kill matrix of running the tests says which tests catch each break, so nothing is guessed; retired 11-03 by the owner's ruling (failed parts retire where a passing part does their job)

Reached, but it flips with the seed The tool's own code won 4 of 5 seeds
This goal
0.72
Plain rule
0.63

The plain rule here: the tests that catch the most.

Not reached One game feature won 0 of 3 seeds
This goal
0.82
Plain rule
0.88

The plain rule here: what each method is seen to do.

Which steps carry a requirement

Learns from: binds requirement. Model: graph network. Since 10-29.

Asked for by the project owner, when asked what else a graph network could do.

Retired. station step_slice is done by the model rules (U12: a step that binds no requirement is refused, and its author binds); retired 11-03 by the owner's ruling (failed parts retire where a passing part does their job)

Not reached The tool's own code won 0 of 1 seed
This goal
0.44
Plain rule
0.69

The plain rule here: shared words.

Not reached One game feature won 0 of 1 seed
This goal
0.40
Plain rule
0.56

The plain rule here: shared words.

Which breaks change nothing

Learns from: the breaks proven the same program. Model: small model on the break's features. Since 10-29.

Asked for by the project owner, when asked what else a graph network could do.

Doubted. The tool's own code: Its labels come from before a fix to how breaks are compared. 88 of its 94 examples are statements holding a comprehension, generator or lambda, so it probably learned the statement's shape, not the break. Inferred, not re-run.

Doubted. Every example it learns from is a break a pass already removes, so its score says nothing about the survivors a person reads.

Retired. break_testing proves an equivalent from compiled code (L19) instead of guessing one; retired 11-03 by the owner's ruling (failed parts retire where a passing part does their job)

Reached on paper, doubted The tool's own code won 10 of 10 seeds
This goal
0.66
Plain rule
0.41

The plain rule here: how often that kind of break survived.

Reached on paper, doubted One game feature won 7 of 10 seeds
This goal
0.62
Plain rule
0.58

The plain rule here: how often that kind of break survived.

Which breaks will get past the tests

Learns from: the breaks of earlier passes. Model: small model on the break's features. Since 10-29.

Asked for by a brainstorm tick.

The code that writes this score is not held by a test yet. All 32 breaks of it got past.

Retired. station break_order is done by break_order_head: HD-2 on the shops (53.8% of survivors in the first quarter, random 25%), held on fresh truth in HD-30 on the selfmodel (33.3%, its shuffled control 24.1%); retired 11-03 by the owner's ruling (failed parts retire where a passing part does their job)

Reached, but it flips with the seed The tool's own code won 9 of 10 seeds
This goal
0.30
Plain rule
0.25

The plain rule here: how often that kind of break survived.

Reached One game feature won 10 of 10 seeds
This goal
0.50
Plain rule
0.31

The plain rule here: how often that kind of break survived.

On the list, no score of their own

Patterns so far

Tried, and did not work

Turning a goal into a vector that points at something good

A goal became a list of numbers measured before any training, and a direction was fitted to what paid off, then tested on graphs it had never seen, with its bars and its predictions committed before the results. There is a weak direction for what a network can learn, and it held up on goals sealed in advance, 9 graphs of 10. There is none for what saves work. The two point about 95 degrees apart, so learnable and useful are different questions. The direction could also be gamed by copying a goal's sources, and it ranked a random relation high. It rests on 13 graphs in 5 domains, which can show a direction is not general but cannot show one is. So no goal is picked this way, and a person or a tick still decides.

Where new goals come from

Goals ask the stats

A goal that has not got there asks the stats list for what beat it.

  • Who owns new code lost to a count of which tests catch its breaks. It asks for killed as an input.
  • Who owns new code lost to what each method is seen to do. It asks for effects as an input.
  • Which tests catch a break lost to what each method is seen to do. It asks for effects as an input.
  • Which steps carry a requirement lost to shared words. It asks for the words on each node as an input.

The graph offers goals

Relations with enough links that nothing predicts. It proposes; a person or a tick decides.

  • documented in: 390 links
  • checked by: 134 links
  • links to: 64 links
  • asserts: 55 links
  • stated by: 45 links
  • next step: 30 links
  • infers requirement: 25 links

How the list has changed

  1. One game feature, pass 4: 0.71 against the best plain rule's 0.48
  2. One game feature, pass 5: 0.63 against the best plain rule's 0.48
  3. Trained to: Who owns new code
  4. One game feature: reshaped what it reads, 6 variants: kept nothing: the full graph's MRR 0.737; the best variant (without the model relations, 0.749) beat it by less than 0.02
  5. One game feature: reshaped what it reads, 8 variants: kept nothing: the full graph's MRR 0.737; the best variant (without the model relations, 0.750) beat it by less than 0.02
  6. One game feature, pass 6: 0.79 against the best plain rule's 0.67
  7. Trained to: Which breaks will get past the tests
  8. Trained to: Which tests catch a break
  9. Trained to: Which steps carry a requirement
  10. Trained to: Which breaks change nothing
  11. Its recorder written: What a change will break, before the run
  12. Its recorder written: Which methods change together
  13. Trained to: Where to add a test
  14. More seeds behind the verdicts: 20 more runs
  15. More seeds behind the verdicts: 10 more runs
  16. More seeds behind the verdicts: 6 more runs
  17. From trained to retired
  18. From trained to retired
  19. From uses another to retired
  20. From trained to retired
  21. From planned to retired
  22. From trained to retired
  23. From trained to retired
  24. The graph now proposes a new goal: predict `documented in`
  25. The graph now proposes a new goal: predict `links to`
  26. The graph now proposes a new goal: predict `infers requirement`

Why this list stays true

The goals are a data file and every number here is read from the run that made it. A job in the records that no goal claims fails the check. A new goal is a new row; a goal that stops stays, with its reason. Each run logs what moved, so the history above keeps growing on its own.

Made by RJ’s TNT Tools. TNT Tools’ own work, not a Heavy Iron product.