Growth

How the closed loop has grown, drawn only from the dated records it keeps about itself. Every point is a reading in a log row that carries a date, and nothing is estimated, interpolated or filled. A line is drawn only through 3 or more readings, and is broken wherever what was counted changed (the kinds of break tried, the criteria checked). With fewer readings the page shows the readings and says so. Every reading here was taken between 10-29 12:02 and 10-31 22:07, so most of what the loop records has few readings; that is itself the finding.

796dated points drawn
44logs read
66series long enough for a line
14series with one or two readings, shown as dots
31series flat
7series fell

What moved, plainly

Fell (7)

Went up and down (8)

Rose (22)

Not comparable across a change (11)

Flat (31)
  • Breaks that got past, the kit’s C# example: flat at 3 over 6 readings (down is better).
  • Breaks that got past, the kit’s Python example: flat at 3 over 14 readings (down is better).
  • Breaks that got past, one game feature: flat at 55 over 2 readings (down is better). Two readings, so no line.
  • Unexplained breaks, one game feature: flat at 35 over 2 readings (down is better). Two readings, so no line.
  • Requirements in the model, the kit’s C# example: flat at 4 over 6 readings.
  • Reasoning steps in the graph, the kit’s C# example: flat at 10 over 6 readings.
  • Requirements in the model, the kit’s Python example: flat at 4 over 14 readings.
  • Reasoning steps in the graph, the kit’s Python example: flat at 10 over 14 readings.
  • Requirements in the model, the tool’s own code: flat at 20 over 2 readings. Two readings, so no line.
  • Reasoning steps in the graph, the tool’s own code: flat at 32 over 2 readings. Two readings, so no line.
  • Requirements in the model, one game feature: flat at 27 over 2 readings. Two readings, so no line.
  • Reasoning steps in the graph, one game feature: flat at 32 over 2 readings. Two readings, so no line.
  • Requirements in the model, learned part: Which breaks will get past the tests: flat at 9 over 3 readings.
  • Reasoning steps in the graph, learned part: Which breaks will get past the tests: flat at 13 over 3 readings.
  • Requirements in the model, learned part: Which breaks change nothing: flat at 6 over 3 readings.
  • Reasoning steps in the graph, learned part: Which breaks change nothing: flat at 12 over 3 readings.
  • Requirements in the model, learned part: Where to add a test: flat at 5 over 3 readings.
  • Reasoning steps in the graph, learned part: Where to add a test: flat at 12 over 3 readings.
  • Requirements in the model, learned part: Who owns new code: flat at 15 over 3 readings.
  • Reasoning steps in the graph, learned part: Who owns new code: flat at 16 over 3 readings.
  • Requirements in the model, learned part: Which steps carry a requirement: flat at 6 over 3 readings.
  • Reasoning steps in the graph, learned part: Which steps carry a requirement: flat at 12 over 3 readings.
  • Requirements in the model, learned part: Which tests catch a break: flat at 7 over 3 readings.
  • Reasoning steps in the graph, learned part: Which tests catch a break: flat at 12 over 3 readings.
  • Stats recorded in a pass, learned part: Which breaks will get past the tests: flat at 47 over 3 readings.
  • Stats recorded in a pass, learned part: Which breaks change nothing: flat at 47 over 3 readings.
  • Stats recorded in a pass, learned part: Where to add a test: flat at 47 over 3 readings.
  • Stats recorded in a pass, learned part: Who owns new code: flat at 47 over 3 readings.
  • Stats recorded in a pass, learned part: Which steps carry a requirement: flat at 47 over 3 readings.
  • Stats recorded in a pass, learned part: Which tests catch a break: flat at 47 over 3 readings.
  • Measured outcomes: flat at 0 over 4 readings.
One reading only (1)
  • Owner network score, one game feature: one reading, 0.7900 (10-29 18:50) (up is better).

Kill rate, project by project

The kit’s C# example6 readings, 10-29 15:26 to 10-29 21:42
0.500.751.0015:2610-2915:2915:3115:3415:3421:421.0000
The kit’s Python example14 readings, 10-29 17:52 to 10-29 21:40
0.500.751.0017:5210-2917:5321:3721:3821:401.0000
Learned part: Which breaks will get past the tests3 readings, 10-30 12:23 to 10-30 17:54
0.500.751.0012:2310-3013:0017:540.6030
Learned part: Which breaks change nothing3 readings, 10-30 11:59 to 10-30 17:17
0.500.751.0011:5910-3012:5817:170.8077
Learned part: Where to add a test3 readings, 10-30 11:48 to 10-30 16:58
0.500.751.0011:4810-3012:5716:580.8017
Learned part: Who owns new code3 readings, 10-30 12:53 to 10-30 20:52
0.500.751.0012:5310-3013:0420:520.6453
Learned part: Which steps carry a requirement3 readings, 10-30 11:53 to 10-30 17:05
0.500.751.0011:5310-3012:5717:050.5211
Learned part: Which tests catch a break3 readings, 10-30 12:10 to 10-30 17:39
0.500.751.0012:1010-3012:5917:390.5682
Projects with fewer than 3 readingsdots only; an open dot is an earlier reading, a filled one the latest
0.500.751.00The tool’s own code0.5777 then 0.6042 (2 readings)One game feature0.9651 then 0.9653 (2 readings)

What it is. Of the breaks the loop made in the code that a test could judge, the share a test caught. Breaks shown or declared to change nothing are left out, and so are breaks with no verdict.

What a rise does and does not prove. A rise proves a larger share of the judged breaks was caught, or that more survivors were explained. It does not prove the tests got better.

In the kit’s Python example the kill rate rose from 0.9565 to 1.0000 while the count of breaks caught stayed at 66 on every reading, so the rise came from breaks being explained, not from more being caught.

The rate leaves out breaks with no verdict. The largest share is in one game feature: 52 of 1,104 breaks had none.

The axis starts at 0.50 where no reading is lower.

The tool’s own code: its two readings are not comparable (the definition changed: the Python adapter now decides by bytecode which breaks change nothing, so breaks once set aside as equivalent are judged).

Every reading in this chart (42)
  • Kit C# example, 10-29 15:26:47: 0.9474
  • Kit C# example, 10-29 15:29:20: 1.0000
  • Kit C# example, 10-29 15:31:55: 0.9818
  • Kit C# example, 10-29 15:34:30: 1.0000
  • Kit C# example, 10-29 15:34:40: 1.0000
  • Kit C# example, 10-29 21:42:35: 1.0000
  • Kit Python example, 10-29 17:52:20: 0.9565
  • Kit Python example, 10-29 17:52:46: 0.9851
  • Kit Python example, 10-29 17:53:07: 1.0000
  • Kit Python example, 10-29 17:53:09: 1.0000
  • Kit Python example, 10-29 18:25:35: 1.0000
  • Kit Python example, 10-29 18:25:51: 1.0000
  • Kit Python example, 10-29 21:37:22: 1.0000
  • Kit Python example, 10-29 21:37:41: 1.0000
  • Kit Python example, 10-29 21:38:08: 1.0000
  • Kit Python example, 10-29 21:38:44: 1.0000
  • Kit Python example, 10-29 21:39:23: 1.0000
  • Kit Python example, 10-29 21:40:00: 1.0000
  • Kit Python example, 10-29 21:40:16: 1.0000
  • Kit Python example, 10-29 21:40:28: 1.0000
  • Tool’s own code, 10-29 17:34:54: 0.5777
  • Tool’s own code, 10-30 16:28:41: 0.6042
  • One game feature, 10-29 18:50:13: 0.9651
  • One game feature, 10-30 16:28:22: 0.9653
  • Learned part: Which breaks will get past the tests, 10-30 12:23:24: 0.6119
  • Learned part: Which breaks will get past the tests, 10-30 13:00:07: 0.6119
  • Learned part: Which breaks will get past the tests, 10-30 17:54:25: 0.6030
  • Learned part: Which breaks change nothing, 10-30 11:59:24: 0.8367
  • Learned part: Which breaks change nothing, 10-30 12:58:43: 0.8367
  • Learned part: Which breaks change nothing, 10-30 17:17:48: 0.8077
  • Learned part: Where to add a test, 10-30 11:48:19: 0.8333
  • Learned part: Where to add a test, 10-30 12:57:09: 0.8333
  • Learned part: Where to add a test, 10-30 16:58:15: 0.8017
  • Learned part: Who owns new code, 10-30 12:53:27: 0.6783
  • Learned part: Who owns new code, 10-30 13:04:00: 0.6783
  • Learned part: Who owns new code, 10-30 20:52:37: 0.6453
  • Learned part: Which steps carry a requirement, 10-30 11:53:16: 0.5135
  • Learned part: Which steps carry a requirement, 10-30 12:57:49: 0.5135
  • Learned part: Which steps carry a requirement, 10-30 17:05:52: 0.5211
  • Learned part: Which tests catch a break, 10-30 12:10:27: 0.5536
  • Learned part: Which tests catch a break, 10-30 12:59:27: 0.5536
  • Learned part: Which tests catch a break, 10-30 17:39:17: 0.5682

Breaks that got past, and those nobody explained

The kit’s C# example6 readings, 10-29 15:26 to 10-29 21:42
012315:2610-2915:2915:3115:3415:3421:4230
The kit’s Python example14 readings, 10-29 17:52 to 10-29 21:40
012317:5210-2917:5321:3721:3821:4030
Learned part: Which breaks will get past the tests3 readings, 10-30 12:23 to 10-30 17:54
05010015012:2310-3013:0017:54106
Learned part: Which breaks change nothing3 readings, 10-30 11:59 to 10-30 17:17
0510152011:5910-3012:5817:1720
Learned part: Where to add a test3 readings, 10-30 11:48 to 10-30 16:58
010203011:4810-3012:5716:5824
Learned part: Who owns new code3 readings, 10-30 12:53 to 10-30 20:52
010020030040012:5310-3013:0420:52382
Learned part: Which steps carry a requirement3 readings, 10-30 11:53 to 10-30 17:05
01020304011:5310-3012:5717:0534
Learned part: Which tests catch a break3 readings, 10-30 12:10 to 10-30 17:39
02040608012:1010-3012:5917:3976
  • Breaks that got past
  • Unexplained breaks

What it is. Breaks no test caught, and of those the ones nobody has yet explained as changing nothing. The unexplained ones are the open work.

What a rise does and does not prove. A fall in the unexplained line is closed work only if the survivors were caught, or shown to be the same program. A fall that comes from a person saying a survivor changes nothing is a claim, not a proof.

In the kit’s Python example the number that got past stayed at 3 and the number caught at 66 on every reading, while the unexplained part fell from 3 to 0: nothing more was caught, 3 were explained, as the same program or by a person.

The tool’s own code: its two readings are not comparable (the definition changed: the Python adapter now decides by bytecode which breaks change nothing, so breaks once set aside as equivalent are judged).

Every reading in this chart (84)
  • Kit C# example, 10-29 15:26:47: 3
  • Kit C# example, 10-29 15:29:20: 3
  • Kit C# example, 10-29 15:31:55: 3
  • Kit C# example, 10-29 15:34:30: 3
  • Kit C# example, 10-29 15:34:40: 3
  • Kit C# example, 10-29 21:42:35: 3
  • Kit Python example, 10-29 17:52:20: 3
  • Kit Python example, 10-29 17:52:46: 3
  • Kit Python example, 10-29 17:53:07: 3
  • Kit Python example, 10-29 17:53:09: 3
  • Kit Python example, 10-29 18:25:35: 3
  • Kit Python example, 10-29 18:25:51: 3
  • Kit Python example, 10-29 21:37:22: 3
  • Kit Python example, 10-29 21:37:41: 3
  • Kit Python example, 10-29 21:38:08: 3
  • Kit Python example, 10-29 21:38:44: 3
  • Kit Python example, 10-29 21:39:23: 3
  • Kit Python example, 10-29 21:40:00: 3
  • Kit Python example, 10-29 21:40:16: 3
  • Kit Python example, 10-29 21:40:28: 3
  • Tool’s own code, 10-29 17:34:54: 647
  • Tool’s own code, 10-30 16:28:41: 648
  • One game feature, 10-29 18:50:13: 55
  • One game feature, 10-30 16:28:22: 55
  • Learned part: Which breaks will get past the tests, 10-30 12:23:24: 85
  • Learned part: Which breaks will get past the tests, 10-30 13:00:07: 85
  • Learned part: Which breaks will get past the tests, 10-30 17:54:25: 106
  • Learned part: Which breaks change nothing, 10-30 11:59:24: 8
  • Learned part: Which breaks change nothing, 10-30 12:58:43: 8
  • Learned part: Which breaks change nothing, 10-30 17:17:48: 20
  • Learned part: Where to add a test, 10-30 11:48:19: 7
  • Learned part: Where to add a test, 10-30 12:57:09: 7
  • Learned part: Where to add a test, 10-30 16:58:15: 24
  • Learned part: Who owns new code, 10-30 12:53:27: 37
  • Learned part: Who owns new code, 10-30 13:04:00: 37
  • Learned part: Who owns new code, 10-30 20:52:37: 382
  • Learned part: Which steps carry a requirement, 10-30 11:53:16: 18
  • Learned part: Which steps carry a requirement, 10-30 12:57:49: 18
  • Learned part: Which steps carry a requirement, 10-30 17:05:52: 34
  • Learned part: Which tests catch a break, 10-30 12:10:27: 50
  • Learned part: Which tests catch a break, 10-30 12:59:27: 50
  • Learned part: Which tests catch a break, 10-30 17:39:17: 76
  • Kit C# example, 10-29 15:26:47: 3
  • Kit C# example, 10-29 15:29:20: 0
  • Kit C# example, 10-29 15:31:55: 1
  • Kit C# example, 10-29 15:34:30: 0
  • Kit C# example, 10-29 15:34:40: 0
  • Kit C# example, 10-29 21:42:35: 0
  • Kit Python example, 10-29 17:52:20: 3
  • Kit Python example, 10-29 17:52:46: 1
  • Kit Python example, 10-29 17:53:07: 0
  • Kit Python example, 10-29 17:53:09: 0
  • Kit Python example, 10-29 18:25:35: 0
  • Kit Python example, 10-29 18:25:51: 0
  • Kit Python example, 10-29 21:37:22: 0
  • Kit Python example, 10-29 21:37:41: 0
  • Kit Python example, 10-29 21:38:08: 0
  • Kit Python example, 10-29 21:38:44: 0
  • Kit Python example, 10-29 21:39:23: 0
  • Kit Python example, 10-29 21:40:00: 0
  • Kit Python example, 10-29 21:40:16: 0
  • Kit Python example, 10-29 21:40:28: 0
  • Tool’s own code, 10-29 17:34:54: 647
  • Tool’s own code, 10-30 16:28:41: 648
  • One game feature, 10-29 18:50:13: 35
  • One game feature, 10-30 16:28:22: 35
  • Learned part: Which breaks will get past the tests, 10-30 12:23:24: 85
  • Learned part: Which breaks will get past the tests, 10-30 13:00:07: 85
  • Learned part: Which breaks will get past the tests, 10-30 17:54:25: 106
  • Learned part: Which breaks change nothing, 10-30 11:59:24: 8
  • Learned part: Which breaks change nothing, 10-30 12:58:43: 8
  • Learned part: Which breaks change nothing, 10-30 17:17:48: 20
  • Learned part: Where to add a test, 10-30 11:48:19: 7
  • Learned part: Where to add a test, 10-30 12:57:09: 7
  • Learned part: Where to add a test, 10-30 16:58:15: 24
  • Learned part: Who owns new code, 10-30 12:53:27: 37
  • Learned part: Who owns new code, 10-30 13:04:00: 37
  • Learned part: Who owns new code, 10-30 20:52:37: 382
  • Learned part: Which steps carry a requirement, 10-30 11:53:16: 18
  • Learned part: Which steps carry a requirement, 10-30 12:57:49: 18
  • Learned part: Which steps carry a requirement, 10-30 17:05:52: 34
  • Learned part: Which tests catch a break, 10-30 12:10:27: 50
  • Learned part: Which tests catch a break, 10-30 12:59:27: 50
  • Learned part: Which tests catch a break, 10-30 17:39:17: 76

Holes of ours

The kit’s C# example6 readings, 10-29 15:26 to 10-29 21:42
05101515:2610-2915:2915:3115:3415:3421:420
The kit’s Python example14 readings, 10-29 17:52 to 10-29 21:40
05101517:5210-2917:5321:3721:3821:400
One game feature3 readings, 10-29 12:02 to 10-30 16:28
025507510012:0210-2918:5016:2810-30not joined77
Learned part: Which breaks will get past the tests3 readings, 10-30 12:23 to 10-30 17:54
02040608012:2310-3013:0017:54not joined66
Learned part: Which breaks change nothing3 readings, 10-30 11:59 to 10-30 17:17
010203011:5910-3012:5817:17not joined26
Learned part: Where to add a test3 readings, 10-30 11:48 to 10-30 16:58
010203011:4810-3012:5716:58not joined21
Learned part: Who owns new code3 readings, 10-30 12:53 to 10-30 20:52
010020030012:5310-3013:0420:52not joined257
Learned part: Which steps carry a requirement3 readings, 10-30 11:53 to 10-30 17:05
010203011:5310-3012:5717:05not joined27
Learned part: Which tests catch a break3 readings, 10-30 12:10 to 10-30 17:39
020406012:1010-3012:5917:39not joined44
Projects with fewer than 3 readingsdots only; an open dot is an earlier reading, a filled one the latest
0100200300400The tool’s own code252 then 387 (2 readings)

What it is. Places where the loop’s own checking still lets a mistake through and the fix is ours to make, counted over the criteria it checks. The number of criteria is recorded with each reading. Fewer is better.

What a rise does and does not prove. A fall means holes were closed. A rise does not mean the code got worse: the tool is changed between readings, and when it learns to look for another kind of gap the count rises on code that did not change.

The largest rise in the kit’s Python example went from 0 to 12 between 10-29 18:25:51 and 10-29 21:37:22, while the tool changed from 5,835 to 5,994 lines. The code under test was not what changed first; the tool was.

One game feature, holes of ours: the line is not joined between readings 1 and 2 (criteria checked 14 to 17).

Learned part: Which breaks will get past the tests, holes of ours: the line is not joined between readings 2 and 3 (criteria checked 17 to 18).

Learned part: Which breaks change nothing, holes of ours: the line is not joined between readings 2 and 3 (criteria checked 17 to 18).

Learned part: Where to add a test, holes of ours: the line is not joined between readings 2 and 3 (criteria checked 17 to 18).

Learned part: Who owns new code, holes of ours: the line is not joined between readings 2 and 3 (criteria checked 17 to 18).

Learned part: Which steps carry a requirement, holes of ours: the line is not joined between readings 2 and 3 (criteria checked 17 to 18).

Learned part: Which tests catch a break, holes of ours: the line is not joined between readings 2 and 3 (criteria checked 17 to 18).

Every reading in this chart (43)
  • Kit C# example, 10-29 15:26:47: 11
  • Kit C# example, 10-29 15:29:20: 5
  • Kit C# example, 10-29 15:31:55: 7
  • Kit C# example, 10-29 15:34:30: 0
  • Kit C# example, 10-29 15:34:40: 0
  • Kit C# example, 10-29 21:42:35: 0
  • Kit Python example, 10-29 17:52:20: 9
  • Kit Python example, 10-29 17:52:46: 7
  • Kit Python example, 10-29 17:53:07: 4
  • Kit Python example, 10-29 17:53:09: 0
  • Kit Python example, 10-29 18:25:35: 0
  • Kit Python example, 10-29 18:25:51: 0
  • Kit Python example, 10-29 21:37:22: 12
  • Kit Python example, 10-29 21:37:41: 12
  • Kit Python example, 10-29 21:38:08: 12
  • Kit Python example, 10-29 21:38:44: 9
  • Kit Python example, 10-29 21:39:23: 6
  • Kit Python example, 10-29 21:40:00: 5
  • Kit Python example, 10-29 21:40:16: 4
  • Kit Python example, 10-29 21:40:28: 0
  • Tool’s own code, 10-29 17:34:54: 252
  • Tool’s own code, 10-30 16:28:41: 387
  • One game feature, 10-29 12:02:22: 0 (dated by the commit that held its report)
  • One game feature, 10-29 18:50:13: 77
  • One game feature, 10-30 16:28:22: 77
  • Learned part: Which breaks will get past the tests, 10-30 12:23:24: 63
  • Learned part: Which breaks will get past the tests, 10-30 13:00:07: 63
  • Learned part: Which breaks will get past the tests, 10-30 17:54:25: 66
  • Learned part: Which breaks change nothing, 10-30 11:59:24: 23
  • Learned part: Which breaks change nothing, 10-30 12:58:43: 23
  • Learned part: Which breaks change nothing, 10-30 17:17:48: 26
  • Learned part: Where to add a test, 10-30 11:48:19: 13
  • Learned part: Where to add a test, 10-30 12:57:09: 13
  • Learned part: Where to add a test, 10-30 16:58:15: 21
  • Learned part: Who owns new code, 10-30 12:53:27: 86
  • Learned part: Who owns new code, 10-30 13:04:00: 85
  • Learned part: Who owns new code, 10-30 20:52:37: 257
  • Learned part: Which steps carry a requirement, 10-30 11:53:16: 24
  • Learned part: Which steps carry a requirement, 10-30 12:57:49: 24
  • Learned part: Which steps carry a requirement, 10-30 17:05:52: 27
  • Learned part: Which tests catch a break, 10-30 12:10:27: 44
  • Learned part: Which tests catch a break, 10-30 12:59:27: 44
  • Learned part: Which tests catch a break, 10-30 17:39:17: 44

The owner network’s score, project by project

Projects with fewer than 3 readingsdots only; an open dot is an earlier reading, a filled one the latest
0.500.751.00One game feature0.7900 (1 reading)

What it is. How well the network that guesses which requirement owns a piece of code ranks the right one, as a score from 0 to 1 (mean reciprocal rank: 1 means the right requirement is always first). Only a pass that ran the learned part writes it.

What a rise does and does not prove. A rise means the network ranked the right requirement higher on that pass’s graph. The graph and the model change between passes, so two readings answer slightly different questions: it does not prove the network got better.

No reading of this stat: The kit’s C# example, The kit’s Python example, The tool’s own code, Learned part: Which breaks will get past the tests, Learned part: Which breaks change nothing, Learned part: Where to add a test, Learned part: Who owns new code, Learned part: Which steps carry a requirement, Learned part: Which tests catch a break.

Every reading in this chart (1)
  • One game feature, 10-29 18:50:13: 0.7900

Requirements and steps in the graph

Latest reading of each modelno time axis: this is size, not growth
010203040The kit’s C# example4 requirements, 10 reasoning stepsThe kit’s Python example4 requirements, 10 reasoning stepsThe tool’s own code20 requirements, 32 reasoning stepsOne game feature27 requirements, 32 reasoning stepsLearned part: Which breaks will get past the tests9 requirements, 13 reasoning stepsLearned part: Which breaks change nothing6 requirements, 12 reasoning stepsLearned part: Where to add a test5 requirements, 12 reasoning stepsLearned part: Who owns new code15 requirements, 16 reasoning stepsLearned part: Which steps carry a requirement6 requirements, 12 reasoning stepsLearned part: Which tests catch a break7 requirements, 12 reasoning steps
  • Requirements
  • Reasoning steps

What it is. How many requirements and how many reasoning steps each project’s model holds, at the latest reading it has.

What a rise does and does not prove. More is not better by itself. A model with more requirements has said more, not said it more correctly; it has grown only if it now covers more of the product.

Within every project the counts were the same on every reading: a model's size is set when it is written and did not change in the readings kept.

Every reading in this chart (84)
  • Kit C# example, 10-29 15:26:47: 4
  • Kit C# example, 10-29 15:29:20: 4
  • Kit C# example, 10-29 15:31:55: 4
  • Kit C# example, 10-29 15:34:30: 4
  • Kit C# example, 10-29 15:34:40: 4
  • Kit C# example, 10-29 21:42:35: 4
  • Kit Python example, 10-29 17:52:20: 4
  • Kit Python example, 10-29 17:52:46: 4
  • Kit Python example, 10-29 17:53:07: 4
  • Kit Python example, 10-29 17:53:09: 4
  • Kit Python example, 10-29 18:25:35: 4
  • Kit Python example, 10-29 18:25:51: 4
  • Kit Python example, 10-29 21:37:22: 4
  • Kit Python example, 10-29 21:37:41: 4
  • Kit Python example, 10-29 21:38:08: 4
  • Kit Python example, 10-29 21:38:44: 4
  • Kit Python example, 10-29 21:39:23: 4
  • Kit Python example, 10-29 21:40:00: 4
  • Kit Python example, 10-29 21:40:16: 4
  • Kit Python example, 10-29 21:40:28: 4
  • Tool’s own code, 10-29 17:34:54: 20
  • Tool’s own code, 10-30 16:28:41: 20
  • One game feature, 10-29 18:50:13: 27
  • One game feature, 10-30 16:28:22: 27
  • Learned part: Which breaks will get past the tests, 10-30 12:23:24: 9
  • Learned part: Which breaks will get past the tests, 10-30 13:00:07: 9
  • Learned part: Which breaks will get past the tests, 10-30 17:54:25: 9
  • Learned part: Which breaks change nothing, 10-30 11:59:24: 6
  • Learned part: Which breaks change nothing, 10-30 12:58:43: 6
  • Learned part: Which breaks change nothing, 10-30 17:17:48: 6
  • Learned part: Where to add a test, 10-30 11:48:19: 5
  • Learned part: Where to add a test, 10-30 12:57:09: 5
  • Learned part: Where to add a test, 10-30 16:58:15: 5
  • Learned part: Who owns new code, 10-30 12:53:27: 15
  • Learned part: Who owns new code, 10-30 13:04:00: 15
  • Learned part: Who owns new code, 10-30 20:52:37: 15
  • Learned part: Which steps carry a requirement, 10-30 11:53:16: 6
  • Learned part: Which steps carry a requirement, 10-30 12:57:49: 6
  • Learned part: Which steps carry a requirement, 10-30 17:05:52: 6
  • Learned part: Which tests catch a break, 10-30 12:10:27: 7
  • Learned part: Which tests catch a break, 10-30 12:59:27: 7
  • Learned part: Which tests catch a break, 10-30 17:39:17: 7
  • Kit C# example, 10-29 15:26:47: 10
  • Kit C# example, 10-29 15:29:20: 10
  • Kit C# example, 10-29 15:31:55: 10
  • Kit C# example, 10-29 15:34:30: 10
  • Kit C# example, 10-29 15:34:40: 10
  • Kit C# example, 10-29 21:42:35: 10
  • Kit Python example, 10-29 17:52:20: 10
  • Kit Python example, 10-29 17:52:46: 10
  • Kit Python example, 10-29 17:53:07: 10
  • Kit Python example, 10-29 17:53:09: 10
  • Kit Python example, 10-29 18:25:35: 10
  • Kit Python example, 10-29 18:25:51: 10
  • Kit Python example, 10-29 21:37:22: 10
  • Kit Python example, 10-29 21:37:41: 10
  • Kit Python example, 10-29 21:38:08: 10
  • Kit Python example, 10-29 21:38:44: 10
  • Kit Python example, 10-29 21:39:23: 10
  • Kit Python example, 10-29 21:40:00: 10
  • Kit Python example, 10-29 21:40:16: 10
  • Kit Python example, 10-29 21:40:28: 10
  • Tool’s own code, 10-29 17:34:54: 32
  • Tool’s own code, 10-30 16:28:41: 32
  • One game feature, 10-29 18:50:13: 32
  • One game feature, 10-30 16:28:22: 32
  • Learned part: Which breaks will get past the tests, 10-30 12:23:24: 13
  • Learned part: Which breaks will get past the tests, 10-30 13:00:07: 13
  • Learned part: Which breaks will get past the tests, 10-30 17:54:25: 13
  • Learned part: Which breaks change nothing, 10-30 11:59:24: 12
  • Learned part: Which breaks change nothing, 10-30 12:58:43: 12
  • Learned part: Which breaks change nothing, 10-30 17:17:48: 12
  • Learned part: Where to add a test, 10-30 11:48:19: 12
  • Learned part: Where to add a test, 10-30 12:57:09: 12
  • Learned part: Where to add a test, 10-30 16:58:15: 12
  • Learned part: Who owns new code, 10-30 12:53:27: 16
  • Learned part: Who owns new code, 10-30 13:04:00: 16
  • Learned part: Who owns new code, 10-30 20:52:37: 16
  • Learned part: Which steps carry a requirement, 10-30 11:53:16: 12
  • Learned part: Which steps carry a requirement, 10-30 12:57:49: 12
  • Learned part: Which steps carry a requirement, 10-30 17:05:52: 12
  • Learned part: Which tests catch a break, 10-30 12:10:27: 12
  • Learned part: Which tests catch a break, 10-30 12:59:27: 12
  • Learned part: Which tests catch a break, 10-30 17:39:17: 12

Stats recorded in a pass

The kit’s C# example6 readings, 10-29 15:26 to 10-29 21:42
020406015:2610-2915:2915:3115:3415:3421:4244
The kit’s Python example14 readings, 10-29 17:52 to 10-29 21:40
020406017:5210-2917:5321:3721:3821:4044
Learned part: Which breaks will get past the tests3 readings, 10-30 12:23 to 10-30 17:54
020406012:2310-3013:0017:5447
Learned part: Which breaks change nothing3 readings, 10-30 11:59 to 10-30 17:17
020406011:5910-3012:5817:1747
Learned part: Where to add a test3 readings, 10-30 11:48 to 10-30 16:58
020406011:4810-3012:5716:5847
Learned part: Who owns new code3 readings, 10-30 12:53 to 10-30 20:52
020406012:5310-3013:0420:5247
Learned part: Which steps carry a requirement3 readings, 10-30 11:53 to 10-30 17:05
020406011:5310-3012:5717:0547
Learned part: Which tests catch a break3 readings, 10-30 12:10 to 10-30 17:39
020406012:1010-3012:5917:3947
Projects with fewer than 3 readingsdots only; an open dot is an earlier reading, a filled one the latest
0204060The tool’s own code46 then 48 (2 readings)One game feature48 then 49 (2 readings)

What it is. How many fields a pass writes into its log row: the stats tracked. Rows rebuilt by hand from older reports are not counted.

What a rise does and does not prove. A rise means more is recorded. It does not mean more is used, and the count also moves with which optional parts a pass ran: a pass without the learned part writes none of its fields.

Every reading in this chart (42)
  • Kit C# example, 10-29 15:26:47: 37
  • Kit C# example, 10-29 15:29:20: 37
  • Kit C# example, 10-29 15:31:55: 37
  • Kit C# example, 10-29 15:34:30: 37
  • Kit C# example, 10-29 15:34:40: 37
  • Kit C# example, 10-29 21:42:35: 44
  • Kit Python example, 10-29 17:52:20: 45
  • Kit Python example, 10-29 17:52:46: 45
  • Kit Python example, 10-29 17:53:07: 45
  • Kit Python example, 10-29 17:53:09: 45
  • Kit Python example, 10-29 18:25:35: 44
  • Kit Python example, 10-29 18:25:51: 44
  • Kit Python example, 10-29 21:37:22: 44
  • Kit Python example, 10-29 21:37:41: 44
  • Kit Python example, 10-29 21:38:08: 44
  • Kit Python example, 10-29 21:38:44: 44
  • Kit Python example, 10-29 21:39:23: 44
  • Kit Python example, 10-29 21:40:00: 44
  • Kit Python example, 10-29 21:40:16: 44
  • Kit Python example, 10-29 21:40:28: 44
  • Tool’s own code, 10-29 17:34:54: 46
  • Tool’s own code, 10-30 16:28:41: 48
  • One game feature, 10-29 18:50:13: 48
  • One game feature, 10-30 16:28:22: 49
  • Learned part: Which breaks will get past the tests, 10-30 12:23:24: 47
  • Learned part: Which breaks will get past the tests, 10-30 13:00:07: 47
  • Learned part: Which breaks will get past the tests, 10-30 17:54:25: 47
  • Learned part: Which breaks change nothing, 10-30 11:59:24: 47
  • Learned part: Which breaks change nothing, 10-30 12:58:43: 47
  • Learned part: Which breaks change nothing, 10-30 17:17:48: 47
  • Learned part: Where to add a test, 10-30 11:48:19: 47
  • Learned part: Where to add a test, 10-30 12:57:09: 47
  • Learned part: Where to add a test, 10-30 16:58:15: 47
  • Learned part: Who owns new code, 10-30 12:53:27: 47
  • Learned part: Who owns new code, 10-30 13:04:00: 47
  • Learned part: Who owns new code, 10-30 20:52:37: 47
  • Learned part: Which steps carry a requirement, 10-30 11:53:16: 47
  • Learned part: Which steps carry a requirement, 10-30 12:57:49: 47
  • Learned part: Which steps carry a requirement, 10-30 17:05:52: 47
  • Learned part: Which tests catch a break, 10-30 12:10:27: 47
  • Learned part: Which tests catch a break, 10-30 12:59:27: 47
  • Learned part: Which tests catch a break, 10-30 17:39:17: 47

Size of the tool

Lines in the tool42 readings, one from each pass in every project, in time order
02,5005,0007,50010,00018:0010-2900:0010-3006:0012:0018:00not joined7,995

What it is. Lines in the tool’s own source files (8 of them now), counted by each pass and written in its log row. The list of files is checked against the stats recorder’s on every build, and the build stops if it changes without being recorded here, so the line breaks wherever the files counted changed.

What a rise does and does not prove. A rise proves the tool got bigger. It does not prove it got better: that needs a stat that moved with it. A tool that grows while nothing it measures moves is added complexity.

From 7,975 to 7,995 lines between 10-30 16:58 and 10-30 20:52: 20 more (0%).

The line is not joined between 10-30 16:28 and 10-30 16:58: files counted 7 to 8.

Every reading in this chart (42)
  • Lines in the tool, 10-29 15:26:47: 4,876
  • Lines in the tool, 10-29 15:29:20: 4,876
  • Lines in the tool, 10-29 15:31:55: 4,883
  • Lines in the tool, 10-29 15:34:30: 4,886
  • Lines in the tool, 10-29 15:34:40: 4,886
  • Lines in the tool, 10-29 17:34:54: 5,614
  • Lines in the tool, 10-29 17:52:20: 5,738
  • Lines in the tool, 10-29 17:52:46: 5,738
  • Lines in the tool, 10-29 17:53:07: 5,738
  • Lines in the tool, 10-29 17:53:09: 5,738
  • Lines in the tool, 10-29 18:25:35: 5,835
  • Lines in the tool, 10-29 18:25:51: 5,835
  • Lines in the tool, 10-29 18:50:13: 5,835
  • Lines in the tool, 10-29 21:37:22: 5,994
  • Lines in the tool, 10-29 21:37:41: 5,994
  • Lines in the tool, 10-29 21:38:08: 5,994
  • Lines in the tool, 10-29 21:38:44: 5,994
  • Lines in the tool, 10-29 21:39:23: 5,994
  • Lines in the tool, 10-29 21:40:00: 5,994
  • Lines in the tool, 10-29 21:40:16: 5,994
  • Lines in the tool, 10-29 21:40:28: 5,994
  • Lines in the tool, 10-29 21:42:35: 5,994
  • Lines in the tool, 10-30 11:48:19: 6,173
  • Lines in the tool, 10-30 11:53:16: 6,173
  • Lines in the tool, 10-30 11:59:24: 6,173
  • Lines in the tool, 10-30 12:10:27: 6,458
  • Lines in the tool, 10-30 12:23:24: 6,460
  • Lines in the tool, 10-30 12:53:27: 6,466
  • Lines in the tool, 10-30 12:57:09: 6,466
  • Lines in the tool, 10-30 12:57:49: 6,466
  • Lines in the tool, 10-30 12:58:43: 6,466
  • Lines in the tool, 10-30 12:59:27: 6,466
  • Lines in the tool, 10-30 13:00:07: 6,466
  • Lines in the tool, 10-30 13:04:00: 6,466
  • Lines in the tool, 10-30 16:28:22: 6,560
  • Lines in the tool, 10-30 16:28:41: 6,560
  • Lines in the tool, 10-30 16:58:15: 7,975
  • Lines in the tool, 10-30 17:05:52: 7,996
  • Lines in the tool, 10-30 17:17:48: 7,995
  • Lines in the tool, 10-30 17:39:17: 7,995
  • Lines in the tool, 10-30 17:54:25: 7,995
  • Lines in the tool, 10-30 20:52:37: 7,995

Improvement requirements

The improvement requirements4 readings, 10-29 20:41 to 10-30 09:24
01020304020:4110-2909:1810-3009:1909:243240
  • Improvement requirements
  • Triggers that fired
  • Measured outcomes

What it is. Each improvement requirement watches one stat, names the trigger that fires and the action to take. Triggers that fired is how many tripped at that reading. Measured outcomes is how many ticks aimed at a stat have a before and an after reading of it.

What a rise does and does not prove. More requirements is more watching, not more improving. Fewer triggers firing is good only when the cause was fixed in the world and not in the trigger; the fall here came from fixing triggers that fired on every pass, which says the watching got better.

Measured outcomes were 0 on 4 of 4 readings: no tick aimed at a stat has yet had its before and after measured.

Every reading in this chart (12)
  • Improvement requirements, 10-29 20:41:32: 29
  • Improvement requirements, 10-30 09:18:43: 32
  • Improvement requirements, 10-30 09:19:10: 32
  • Improvement requirements, 10-30 09:24:46: 32
  • Triggers that fired, 10-29 20:41:32: 12
  • Triggers that fired, 10-30 09:18:43: 12
  • Triggers that fired, 10-30 09:19:10: 12
  • Triggers that fired, 10-30 09:24:46: 4
  • Measured outcomes, 10-29 20:41:32: 0
  • Measured outcomes, 10-30 09:18:43: 0
  • Measured outcomes, 10-30 09:19:10: 0
  • Measured outcomes, 10-30 09:24:46: 0

Ticks: minutes per item, and was it worth it

Minutes per tick5 ticks, 10-30 08:43 to 10-30 15:24. Under each bar: its time, its date, and the tick’s own answer to was it worth it
02040608008:4310-30yes09:1410-30yes09:2710-30yes11:0410-30yes15:2410-30yes922117341
  • Light tick
  • Full tick

What it is. How long each improvement tick took, by mode, and the tick’s own answer to whether the item was worth doing, written when it ended.

What a rise does and does not prove. Minutes are cost, not growth, and bars are one per tick so no trend is drawn. Worth is the tick’s own judgement and not an outside check: 5 of 5 say yes, and a verdict that has not yet said no tells little.

Long passes launched by a tick: 3, collected: 3.

Every tick (5)
  • Tick 1, 10-30 08:43:44: 9 minutes, light, worth: yes
  • Tick 2, 10-30 09:14:32: 22 minutes, full, worth: yes
  • Tick 3, 10-30 09:27:59: 11 minutes, full, worth: yes
  • Tick 4, 10-30 11:04:35: 73 minutes, light, worth: yes
  • Tick 5, 10-30 15:24:00: 41 minutes, full, worth: yes

Goals: tried and reached

Goals with a trained network3 readings; a step holds the count until the next goal was trained
024616:0010-2918:0020:005

What it is. How many of the goals on the Goals List have a network trained for them, counted at the time each one’s code first appeared in git.

What a rise does and does not prove. More goals trained is more tried, not more reached. Trying is not reaching, and the reached count is a single reading, so it has no line.

Reached, one reading (10-30 12:56): 5 of 10 results reached their goal on paper, so 3 of 5 trained goals reached it on at least one project. A goal counts as reached only when it beats a plain rule at most seeds on code it has not seen. The log has 1 reading of this.

On the list: 5 trained, 4 planned, 1 uses another.

Every reading in this chart (3)
  • Goals with a trained network, 10-29 14:25:00: 1 (dated by the time its code first appeared)
  • Goals with a trained network, 10-29 20:13:00: 2 (dated by the time its code first appeared)
  • Goals with a trained network, 10-29 20:36:00: 5 (dated by the time its code first appeared)
  • Reached: Which breaks will get past the tests on the tool’s own code: won 9 of 10 seeds; Which breaks will get past the tests on one game feature: won 10 of 10 seeds; Which breaks change nothing on the tool’s own code: won 10 of 10 seeds; Which breaks change nothing on one game feature: won 7 of 10 seeds; Who owns new code on the tool’s own code: won 0 of 5 seeds; Who owns new code on one game feature: won 0 of 3 seeds; Which steps carry a requirement on the tool’s own code: won 0 of 1 seeds; Which steps carry a requirement on one game feature: won 0 of 1 seeds; Which tests catch a break on the tool’s own code: won 4 of 5 seeds; Which tests catch a break on one game feature: won 0 of 3 seeds

The plan’s burn-down

The plan’s burn-down170 readings, 10-30 16:07:15 to 10-31 22:07:04
020406016:0710-3018:0521:1900:4110-3108:3222:0727048
  • Items open
  • Items running
  • Items done

What it is. How many items of the plan are open, running and done, written each time one changes state.

What a rise does and does not prove. Open falling while done rises is the burn-down. A reading is written only when an item changes state, so the spacing says when work moved and not how fast it goes.

170 readings, 1800 minutes apart: read the line against the dates, not as a steady rate.

Every reading in this chart (170)
  • 10-30 16:07:15: 21 open, 2 running, 0 done, 0 blocked
  • 10-30 16:07:16: 20 open, 3 running, 0 done, 0 blocked
  • 10-30 16:22:20: 21 open, 3 running, 0 done, 0 blocked
  • 10-30 16:22:20: 21 open, 2 running, 1 done, 0 blocked
  • 10-30 16:22:21: 20 open, 3 running, 1 done, 0 blocked
  • 10-30 16:30:02: 21 open, 3 running, 1 done, 0 blocked
  • 10-30 16:45:53: 21 open, 2 running, 2 done, 0 blocked
  • 10-30 16:47:23: 21 open, 2 running, 2 done, 0 blocked
  • 10-30 16:47:24: 20 open, 3 running, 2 done, 0 blocked
  • 10-30 17:09:12: 20 open, 2 running, 3 done, 0 blocked
  • 10-30 17:09:13: 20 open, 2 running, 3 done, 0 blocked
  • 10-30 17:09:46: 19 open, 3 running, 3 done, 0 blocked
  • 10-30 17:19:37: 19 open, 2 running, 4 done, 0 blocked
  • 10-30 17:41:37: 20 open, 2 running, 4 done, 0 blocked
  • 10-30 17:41:37: 19 open, 3 running, 4 done, 0 blocked
  • 10-30 17:41:38: 20 open, 3 running, 4 done, 0 blocked
  • 10-30 17:41:38: 19 open, 3 running, 4 done, 1 blocked
  • 10-30 17:41:38: 20 open, 3 running, 4 done, 1 blocked
  • 10-30 17:41:39: 19 open, 3 running, 4 done, 2 blocked
  • 10-30 17:48:15: 20 open, 3 running, 4 done, 1 blocked
  • 10-30 17:48:15: 21 open, 3 running, 4 done, 1 blocked
  • 10-30 17:48:16: 20 open, 3 running, 4 done, 2 blocked
  • 10-30 17:48:16: 20 open, 3 running, 4 done, 2 blocked
  • 10-30 17:57:37: 21 open, 3 running, 4 done, 2 blocked
  • 10-30 17:57:37: 20 open, 4 running, 4 done, 2 blocked
  • 10-30 17:57:38: 21 open, 4 running, 4 done, 1 blocked
  • 10-30 17:58:17: 21 open, 4 running, 4 done, 1 blocked
  • 10-30 18:03:23: 22 open, 4 running, 4 done, 1 blocked
  • 10-30 18:03:23: 23 open, 4 running, 4 done, 1 blocked
  • 10-30 18:05:28: 24 open, 4 running, 4 done, 1 blocked
  • 10-30 18:05:28: 25 open, 4 running, 4 done, 1 blocked
  • 10-30 18:05:29: 26 open, 4 running, 4 done, 1 blocked
  • 10-30 18:05:29: 27 open, 4 running, 4 done, 1 blocked
  • 10-30 18:14:50: 28 open, 4 running, 4 done, 1 blocked
  • 10-30 18:14:50: 29 open, 4 running, 4 done, 1 blocked
  • 10-30 18:14:51: 30 open, 4 running, 4 done, 1 blocked
  • 10-30 18:14:51: 31 open, 4 running, 4 done, 1 blocked
  • 10-30 18:14:52: 32 open, 4 running, 4 done, 1 blocked
  • 10-30 18:14:52: 33 open, 4 running, 4 done, 1 blocked
  • 10-30 18:14:52: 34 open, 4 running, 4 done, 1 blocked
  • 10-30 18:14:53: 35 open, 4 running, 4 done, 1 blocked
  • 10-30 18:24:32: 35 open, 3 running, 5 done, 1 blocked
  • 10-30 18:25:06: 35 open, 3 running, 5 done, 1 blocked
  • 10-30 18:25:07: 35 open, 3 running, 5 done, 1 blocked
  • 10-30 18:26:49: 35 open, 3 running, 5 done, 1 blocked
  • 10-30 19:02:31: 36 open, 3 running, 5 done, 1 blocked
  • 10-30 19:02:31: 37 open, 3 running, 5 done, 1 blocked
  • 10-30 20:04:24: 38 open, 3 running, 5 done, 1 blocked
  • 10-30 20:06:21: 37 open, 4 running, 5 done, 1 blocked
  • 10-30 20:06:22: 36 open, 5 running, 5 done, 1 blocked
  • 10-30 20:53:33: 36 open, 4 running, 6 done, 1 blocked
  • 10-30 21:02:19: 36 open, 3 running, 7 done, 1 blocked
  • 10-30 21:02:20: 36 open, 3 running, 7 done, 1 blocked
  • 10-30 21:02:20: 35 open, 4 running, 7 done, 1 blocked
  • 10-30 21:02:21: 34 open, 5 running, 7 done, 1 blocked
  • 10-30 21:08:24: 33 open, 6 running, 7 done, 1 blocked
  • 10-30 21:19:12: 33 open, 5 running, 8 done, 1 blocked
  • 10-30 21:19:12: 33 open, 5 running, 8 done, 1 blocked
  • 10-30 21:19:13: 32 open, 6 running, 8 done, 1 blocked
  • 10-30 21:40:38: 32 open, 5 running, 9 done, 1 blocked
  • 10-30 21:40:39: 32 open, 4 running, 10 done, 1 blocked
  • 10-30 21:40:39: 32 open, 4 running, 10 done, 1 blocked
  • 10-30 22:05:43: 32 open, 3 running, 11 done, 1 blocked
  • 10-30 22:05:44: 32 open, 3 running, 11 done, 1 blocked
  • 10-30 22:05:45: 31 open, 4 running, 11 done, 1 blocked
  • 10-30 22:05:46: 30 open, 5 running, 11 done, 1 blocked
  • 10-30 22:28:09: 30 open, 4 running, 12 done, 1 blocked
  • 10-30 23:18:28: 30 open, 3 running, 13 done, 1 blocked
  • 10-30 23:18:29: 30 open, 2 running, 14 done, 1 blocked
  • 10-30 23:18:30: 30 open, 2 running, 14 done, 1 blocked
  • 10-30 23:18:30: 29 open, 3 running, 14 done, 1 blocked
  • 10-30 23:18:31: 28 open, 4 running, 14 done, 1 blocked
  • 10-30 23:34:22: 28 open, 3 running, 15 done, 1 blocked
  • 10-30 23:34:23: 28 open, 3 running, 15 done, 1 blocked
  • 10-30 23:34:24: 27 open, 4 running, 15 done, 1 blocked
  • 10-30 23:34:25: 28 open, 4 running, 15 done, 1 blocked
  • 10-31 00:22:23: 28 open, 2 running, 17 done, 1 blocked
  • 10-31 00:22:24: 28 open, 2 running, 17 done, 1 blocked
  • 10-31 00:22:24: 28 open, 2 running, 17 done, 1 blocked
  • 10-31 00:22:25: 27 open, 3 running, 17 done, 1 blocked
  • 10-31 00:34:34: 27 open, 2 running, 18 done, 1 blocked
  • 10-31 00:34:34: 27 open, 2 running, 18 done, 1 blocked
  • 10-31 00:34:35: 26 open, 3 running, 18 done, 1 blocked
  • 10-31 00:34:35: 25 open, 4 running, 18 done, 1 blocked
  • 10-31 00:34:35: 24 open, 5 running, 18 done, 1 blocked
  • 10-31 00:34:36: 23 open, 6 running, 18 done, 1 blocked
  • 10-31 00:41:21: 24 open, 6 running, 18 done, 1 blocked
  • 10-31 00:41:22: 25 open, 6 running, 18 done, 1 blocked
  • 10-31 00:41:23: 26 open, 6 running, 18 done, 1 blocked
  • 10-31 00:41:24: 27 open, 6 running, 18 done, 1 blocked
  • 10-31 00:41:26: 28 open, 6 running, 18 done, 1 blocked
  • 10-31 01:00:26: 28 open, 5 running, 19 done, 1 blocked
  • 10-31 01:00:27: 28 open, 5 running, 19 done, 1 blocked
  • 10-31 01:00:28: 29 open, 5 running, 19 done, 1 blocked
  • 10-31 01:00:29: 28 open, 6 running, 19 done, 1 blocked
  • 10-31 02:19:59: 28 open, 5 running, 20 done, 1 blocked
  • 10-31 02:20:00: 28 open, 5 running, 20 done, 1 blocked
  • 10-31 02:20:26: 27 open, 6 running, 20 done, 1 blocked
  • 10-31 02:20:26: 26 open, 7 running, 20 done, 1 blocked
  • 10-31 02:56:10: 26 open, 7 running, 21 done, 1 blocked
  • 10-31 04:05:55: 26 open, 6 running, 22 done, 1 blocked
  • 10-31 04:05:56: 26 open, 5 running, 23 done, 1 blocked
  • 10-31 04:05:56: 26 open, 5 running, 23 done, 1 blocked
  • 10-31 04:05:57: 25 open, 6 running, 23 done, 1 blocked
  • 10-31 04:05:58: 24 open, 7 running, 23 done, 1 blocked
  • 10-31 07:16:05: 25 open, 7 running, 23 done, 1 blocked
  • 10-31 07:16:06: 24 open, 8 running, 23 done, 1 blocked
  • 10-31 07:16:07: 24 open, 8 running, 23 done, 1 blocked
  • 10-31 07:21:06: 24 open, 7 running, 24 done, 1 blocked
  • 10-31 07:21:07: 24 open, 6 running, 25 done, 1 blocked
  • 10-31 07:21:08: 24 open, 6 running, 25 done, 1 blocked
  • 10-31 07:21:08: 23 open, 7 running, 25 done, 1 blocked
  • 10-31 08:27:08: 23 open, 6 running, 27 done, 1 blocked
  • 10-31 08:27:09: 23 open, 6 running, 27 done, 1 blocked
  • 10-31 08:27:09: 24 open, 6 running, 27 done, 1 blocked
  • 10-31 08:27:09: 23 open, 7 running, 27 done, 1 blocked
  • 10-31 08:32:14: 23 open, 6 running, 28 done, 1 blocked
  • 10-31 08:32:14: 23 open, 6 running, 28 done, 1 blocked
  • 10-31 08:34:12: 23 open, 5 running, 29 done, 1 blocked
  • 10-31 08:34:13: 23 open, 4 running, 30 done, 1 blocked
  • 10-31 08:34:13: 23 open, 3 running, 31 done, 1 blocked
  • 10-31 08:34:14: 23 open, 2 running, 32 done, 1 blocked
  • 10-31 08:34:14: 23 open, 2 running, 32 done, 1 blocked
  • 10-31 09:53:38: 23 open, 1 running, 33 done, 1 blocked
  • 10-31 09:53:39: 23 open, 0 running, 34 done, 1 blocked
  • 10-31 09:53:39: 23 open, 0 running, 34 done, 1 blocked
  • 10-31 09:53:40: 24 open, 0 running, 34 done, 1 blocked
  • 10-31 09:53:40: 23 open, 1 running, 34 done, 1 blocked
  • 10-31 09:53:41: 22 open, 2 running, 34 done, 1 blocked
  • 10-31 09:53:48: 22 open, 2 running, 33 done, 2 blocked
  • 10-31 10:16:17: 22 open, 0 running, 35 done, 2 blocked
  • 10-31 10:16:17: 22 open, 0 running, 35 done, 2 blocked
  • 10-31 10:16:18: 22 open, 0 running, 35 done, 2 blocked
  • 10-31 10:16:18: 21 open, 1 running, 35 done, 2 blocked
  • 10-31 10:16:18: 20 open, 2 running, 35 done, 2 blocked
  • 10-31 10:20:43: 21 open, 2 running, 35 done, 1 blocked
  • 10-31 10:33:46: 21 open, 1 running, 36 done, 1 blocked
  • 10-31 10:33:47: 21 open, 0 running, 37 done, 1 blocked
  • 10-31 10:33:48: 21 open, 0 running, 37 done, 1 blocked
  • 10-31 11:02:00: 21 open, 0 running, 37 done, 1 blocked
  • 10-31 12:05:35: 20 open, 0 running, 39 done, 1 blocked
  • 10-31 12:05:35: 20 open, 0 running, 39 done, 1 blocked
  • 10-31 12:14:02: 21 open, 0 running, 39 done, 1 blocked
  • 10-31 12:17:08: 22 open, 0 running, 39 done, 1 blocked
  • 10-31 12:17:08: 23 open, 0 running, 39 done, 1 blocked
  • 10-31 12:17:09: 24 open, 0 running, 39 done, 1 blocked
  • 10-31 12:42:07: 23 open, 0 running, 40 done, 1 blocked
  • 10-31 12:42:08: 23 open, 0 running, 40 done, 1 blocked
  • 10-31 12:42:08: 22 open, 1 running, 40 done, 1 blocked
  • 10-31 13:10:58: 22 open, 0 running, 41 done, 1 blocked
  • 10-31 13:10:59: 22 open, 0 running, 41 done, 1 blocked
  • 10-31 13:14:45: 23 open, 0 running, 41 done, 1 blocked
  • 10-31 13:14:48: 22 open, 0 running, 42 done, 1 blocked
  • 10-31 16:29:59: 23 open, 0 running, 42 done, 1 blocked
  • 10-31 16:29:59: 24 open, 0 running, 42 done, 1 blocked
  • 10-31 16:58:37: 25 open, 0 running, 42 done, 1 blocked
  • 10-31 18:29:18: 26 open, 0 running, 42 done, 1 blocked
  • 10-31 18:29:19: 25 open, 0 running, 43 done, 1 blocked
  • 10-31 18:29:19: 26 open, 0 running, 43 done, 1 blocked
  • 10-31 18:29:20: 25 open, 0 running, 44 done, 1 blocked
  • 10-31 19:14:25: 26 open, 0 running, 44 done, 1 blocked
  • 10-31 19:14:26: 25 open, 0 running, 45 done, 1 blocked
  • 10-31 19:52:45: 26 open, 0 running, 45 done, 1 blocked
  • 10-31 19:52:45: 25 open, 0 running, 46 done, 1 blocked
  • 10-31 20:28:28: 26 open, 0 running, 46 done, 1 blocked
  • 10-31 20:28:29: 25 open, 0 running, 47 done, 1 blocked
  • 10-31 21:58:21: 26 open, 0 running, 47 done, 1 blocked
  • 10-31 22:03:07: 27 open, 0 running, 47 done, 1 blocked
  • 10-31 22:07:04: 28 open, 0 running, 47 done, 1 blocked
  • 10-31 22:07:04: 27 open, 0 running, 48 done, 1 blocked

Today against the ceiling

What it is. For each stat that may be shown in public, the loop’s value today (filled dot) and the most it could ever be (the upright tick). Each row has its own scale; its two end labels are its own. A ceiling is drawn only where arithmetic or an oracle argues one.

What it does and does not prove. The gap to the tick is room left. Each value is read from a record.

kill rate
Kill rate, the kit's C# examplea reading of the work, not an outcome the tool changes
100.0%0.0%100.0%
Kill rate, the kit's Python examplea reading of the work, not an outcome the tool changes
100.0%0.0%100.0%
Kill rate, one game featurea reading of the work, not an outcome the tool changes
96.5%0.0%100.0%
Kill rate, the tool's own codea reading of the work, not an outcome the tool changes
60.4%0.0%100.0%
Kill rate, the owner network's codea reading of the work, not an outcome the tool changes
64.5%0.0%100.0%
Kill rate, the test-ranking network's codea reading of the work, not an outcome the tool changes
56.8%0.0%100.0%
Kill rate, the step-carrier network's codea reading of the work, not an outcome the tool changes
52.1%0.0%100.0%
Kill rate, the equivalent-break network's codea reading of the work, not an outcome the tool changes
80.8%0.0%100.0%
Kill rate, the test-hint network's codea reading of the work, not an outcome the tool changes
80.2%0.0%100.0%
Kill rate, the break-order network's codea reading of the work, not an outcome the tool changes
60.3%0.0%100.0%
Kill rate, a public version class (against line coverage)the loop reads 87.3% where the number a team has (line coverage of the same tests, a different measure of how much the tests hold) says 90.0%
87.3%0.0%100.0%
Kill rate, a large public class (against line coverage)the loop reads 69.7% where the number a team has (line coverage of the same tests, a different measure of how much the tests hold) says 77.0%
69.7%0.0%100.0%
Kill rate, the scoring rules of a shipped game (against line coverage)the loop reads 53.4% where the number a team has (line coverage of the same 151 tests, a different measure of how much the tests hold) says 72.0%
53.4%0.0%100.0%
holes of ours
Holes of ours, one game featurea reading of the work, not an outcome the tool changes
77077
Holes of ours, the tool's own codea reading of the work, not an outcome the tool changes
3870387
Holes of ours, the six learned parts togethera reading of the work, not an outcome the tool changes
4410441
Holes of ours, the kit's two examples togethera reading of the work, not an outcome the tool changes
000
first run
README steps that did not give what the README says, over the two full installs (machine with PyTorch and .NET, machine with Python only)the loop reads 0 steps where the number a team has (the same two installs before the fixes of 10-31: the install step ended with failures in both) says 2 steps
000
harmony
Links of the harmony chain that hold (a stat worsens, the loop follows it to the documents)the tool built this, so there is nothing to beat; what matters is what it caught
9010
documents
Document links that are right (60 sampled, read against the source line)the tool built this, so there is nothing to beat; what matters is what it caught
60060
Requirements of one game feature that a document names by its own wordsthe tool built this, so there is nothing to beat; what matters is what it caught
11027
Requirements of the kit's own model that a public document names by its own wordsthe tool built this, so there is nothing to beat; what matters is what it caught
0020
claims
Public numbers tied to a record (read from it, or written by a tool and found fresh)the tool built this, so there is nothing to beat; what matters is what it caught
65401,142
No ceiling argued, or no reading yet (10)
  • Tokens per hole an agent closed while working to the loop's list: no reading yet: not measured: no record joins the holes an agent closed to the tokens it spent. What would bound the ceiling: the cheapest arm that closes every hole it was shown
  • Requirements the walk added beyond the written ones, one game feature: a reading of the work, not an outcome the tool changes. What would bound the ceiling: an independent reading of the sheet that lists every rule it implies
  • Seconds for a pass, the kit's C# example (58 breaks): a reading of the work, not an outcome the tool changes. What would bound the ceiling: tests run once on each project, times the breaks tried, divided by the workers
  • Seconds for a pass, one game feature (1,104 breaks): a reading of the work, not an outcome the tool changes. What would bound the ceiling: tests run once on each project, times the breaks tried, divided by the workers
  • Seconds for a pass, the tool's own code (1,639 breaks): a reading of the work, not an outcome the tool changes. What would bound the ceiling: tests run once on each project, times the breaks tried, divided by the workers
  • Seconds for a pass, the owner network's code (1,077 breaks): a reading of the work, not an outcome the tool changes. What would bound the ceiling: tests run once on each project, times the breaks tried, divided by the workers
  • Minutes for a fresh agent to install from the zip (first-run baseline): a reading of the work, not an outcome the tool changes. What would bound the ceiling: V2's clean install: the time of the single command on its own examples
  • Minutes for a stranger's full install to prove itself, a clean home with PyTorch and .NET (cold package cache): the loop reads 7.4 minutes where the number a team has (the same install on the kit as it was before the first-run fixes of 10-31 (the proofs ran one after another and the install ended INSTALLED WITH FAILURES): the number to beat) says 14.6 minutes. What would bound the ceiling: time the C# example alone on a clean home with its packages cached
  • Minutes for a stranger's full install to prove itself, a clean home with Python and PyYAML only: the loop reads 0.8 minutes where the number a team has (the same install on the kit as it was before the first-run fixes of 10-31 (it ended INSTALLED WITH FAILURES): the number to beat) says 2.5 minutes. What would bound the ceiling: time each selftest alone on a clean home
  • Seconds for a stranger's quick install (the front door only) to prove itself, a clean home with PyTorch and .NET present: the loop reads 40 seconds where the number a team has (the same quick install before the fixes of 10-31, which imported PyTorch to see whether it was there: the number to beat) says 111 seconds. What would bound the ceiling: time the three alone on a clean home

Not drawn, and why

What was read

44 logs, every one present and every row readable

Made by RJ’s TNT Tools. TNT Tools’ own work, not a Heavy Iron product.