Ship the learned-key pipeline, without which six figures cannot be rebuilt

The package was missing every script behind the KM (lrn.) curves and
both learned table rows: exp_learned's driver, the merge that folds the
learned rows into sec_compare.csv and refresh_summary.csv, and the
report that reads the learned numbers back. It was also missing
sec_keylen_perm.csv, so Fig. 3 could not be regenerated at all, and the
two diagnostics that answer why a fixed key beats a learned one here and
where a learned mask would win instead.

The learned artifacts themselves are regenerated. They were trained on
the cross-entropy alone, which drifts to disjoint sparse supports: 99
percent of each key's energy on about six of the 64 entries, so a digit
is decided over a sixth of its period and the key set is a choice of
support rather than a dense direction in R^L. They are now the
regularized keys of Section V-C, and check_consistency asserts which of
the two families the figures draw.

Verified from inside this repository: replot_security.py rebuilds all
seven result figures, make_tables.py reproduces both result tables, and
check_consistency.py passes every check that does not need the
manuscript.

The README now lists what ships. Its run list, layout and figure map had
none of the learned pipeline, named two tables the manuscript renders as
prose, and gave Fig. 3 no data file for its permutation curve.

data/compare_learned.csv

@@ -1,2 +1,2 @@
 scheme,legit_ser,eve_out,eve_in,jam0_ser
-proposed_learned,0.06358416667,0.9998083333,0.9999833333,0.4046066667
+proposed_learned,0.06102,0.999,0.99998,0.3971366667

data/jscc.csv

@@ -0,0 +1,20 @@
+part,case,structured_ser,structured_sd,learned_ser,kappa_and_verdict
+A one user,Walsh row,0.01330,0.00007,,
+A one user,all ones (no mask),0.01325,0.00022,,
+A one user,"learned, free",0.01480,0.00045,,
+B key length,L=6,0.99610,0.00048,0.99722,0.11111/tie
+B key length,L=8,0.94056,0.00686,0.95188,0.00000/tie
+B key length,L=10,0.83318,0.03634,0.80853,0.06667/tie
+B key length,L=12,0.41391,0.00154,0.41663,0.00000/tie
+B key length,L=14,0.39963,0.00779,0.37367,0.04762/learned
+B key length,L=16,0.25810,0.00119,0.28998,0.00000/structured
+B key length,L=18,0.23796,0.00182,0.23525,0.03704/tie
+B key length,L=20,0.18811,0.00113,0.21565,0.00000/structured
+B key length,L=22,0.17300,0.00072,0.18993,0.03030/structured
+B key length,L=24,0.15290,0.00034,0.17582,0.00000/structured
+C load,U=4,0.25810,0.00119,0.28998,0.00000/structured
+C load,U=8,0.95022,0.01381,0.97322,0.00000/tie
+C load,U=12,0.99882,0.00005,0.99867,0.00000/tie
+C load,U=15,0.99976,0.00000,0.99976,0.00000/tie
+C load,U=16,0.99976,0.00001,0.99976,0.00000/tie
+C load,U=20,0.99982,0.00000,0.99981,0.02105/learned

data/kpa_learned.csv

@@ -1,43 +1,43 @@
 snr_db,n_frames,kappa,eve_ser
-0,1,0.2127109103,0.993336
-0,2,0.7559393242,0.666508
-0,3,0.8627683729,0.392272125
-0,4,0.9179756209,0.218212875
-0,5,0.945390512,0.14425925
-0,6,0.9590921029,0.107493
-0,8,0.9725584686,0.086112625
-0,10,0.9778045967,0.07933375
-0,12,0.9831736788,0.07420325
-0,16,0.9879393309,0.070712875
-0,24,0.9925390184,0.067800375
-0,32,0.9945808738,0.06649325
-0,48,0.9965663388,0.065265
-0,64,0.9975094497,0.065024875
-10,1,0.3165432975,0.948993875
-10,2,0.9290210679,0.19776825
-10,3,0.9781143948,0.095876875
-10,4,0.9896475986,0.070495125
-10,5,0.9934410676,0.067297125
-10,6,0.9953705788,0.06628575
-10,8,0.9971551418,0.06520175
-10,10,0.9979188025,0.064684875
-10,12,0.9982561454,0.06454375
-10,16,0.9987509355,0.06426625
-10,24,0.9992754847,0.064091125
-10,32,0.9994516179,0.06386675
-10,48,0.9996520028,0.063819875
-10,64,0.999746412,0.063847125
-20,1,0.742194891,0.513137625
-20,2,0.9947786465,0.06753025
-20,3,0.9986294076,0.06438075
-20,4,0.9992210969,0.064160375
-20,5,0.9994008377,0.064080125
-20,6,0.9995701849,0.063900375
-20,8,0.9997365534,0.063852875
-20,10,0.9998067141,0.063813
-20,12,0.9998438716,0.06370225
-20,16,0.9998808399,0.063670875
-20,24,0.9999239221,0.06380125
-20,32,0.9999452353,0.063820625
-20,48,0.9999649763,0.063811
-20,64,0.9999733046,0.06377525
+0,1,0.05034990469,0.997911625
+0,2,0.05894429332,0.662401
+0,3,0.08379636761,0.37693875
+0,4,0.107222241,0.22233425
+0,5,0.1292011227,0.152770125
+0,6,0.1429406652,0.131741625
+0,8,0.1788643047,0.086013875
+0,10,0.2027079877,0.077310875
+0,12,0.2198284302,0.07391475
+0,16,0.2579282403,0.069537375
+0,24,0.3094305053,0.066363625
+0,32,0.3459532984,0.06496575
+0,48,0.4114221439,0.063700375
+0,64,0.4645803586,0.06309375
+10,1,0.05442041112,0.950243625
+10,2,0.1407177548,0.181911125
+10,3,0.2204171771,0.080573875
+10,4,0.2837046772,0.072395125
+10,5,0.3326446027,0.06649075
+10,6,0.3704789877,0.0648705
+10,8,0.4370021872,0.063752375
+10,10,0.479882317,0.062964
+10,12,0.5199262805,0.062786125
+10,16,0.5935128644,0.062229125
+10,24,0.6796541184,0.061876875
+10,32,0.7379264548,0.0618305
+10,48,0.8053073436,0.061516125
+10,64,0.8465657607,0.061403
+20,1,0.06401134632,0.757281
+20,2,0.3653038088,0.06925225
+20,3,0.5262307867,0.06323275
+20,4,0.6446151808,0.06203725
+20,5,0.7232914954,0.06185625
+20,6,0.7802997425,0.06165275
+20,8,0.8357323289,0.061493375
+20,10,0.8705290645,0.061302375
+20,12,0.8942843586,0.061368125
+20,16,0.9240833685,0.06133
+20,24,0.9489109725,0.061323375
+20,32,0.9614683628,0.06120425
+20,48,0.97474062,0.061322875
+20,64,0.9812480465,0.06146825

data/real_sec_stats_learned.json

@@ -11,19 +11,19 @@
  "headline_runs": 4,
  "recovery": {
   "20": {
-   "legit": 0.7023870636550308,
+   "legit": 0.7183008213552361,
    "eve": 0.0,
    "insider": 0.0,
    "oma": 0.6463039014373717
   },
   "24": {
-   "legit": 0.8787217659137577,
+   "legit": 0.8805184804928131,
    "eve": 0.0,
    "insider": 0.0,
    "oma": 0.8390657084188912
   },
   "28": {
-   "legit": 0.9477669404517454,
+   "legit": 0.946611909650924,
    "eve": 0.0,
    "insider": 0.0,
    "oma": 0.9319815195071869

data/real_sec_ter_learned.csv

@@ -1,9 +1,9 @@
 snr_db,ter_legit,ter_eve,ter_insider,ter_oma
-0,0.4516948856,0.9998824906,0.9962722018,0.5232856128
-4,0.223975418,0.9998662393,0.9947945836,0.2740881771
-8,0.09790408233,0.9997899832,0.9940332727,0.123424874
-12,0.04099952996,0.9997912333,0.9936357409,0.05201666133
-16,0.01653382271,0.9997299784,0.9935107309,0.02118419474
-20,0.006725538043,0.999737479,0.9934594768,0.008630690455
-24,0.002707716617,0.9997112269,0.9934182235,0.003452776222
-28,0.001037583007,0.9997274782,0.9934144732,0.001385110809
+0,0.4424778982,0.9998887411,0.9961971958,0.5232856128
+4,0.2185937375,0.9998887411,0.9948020842,0.2740881771
+8,0.09510385831,0.9998837407,0.9940082707,0.123424874
+12,0.03968067445,0.9998599888,0.9936469918,0.05201666133
+16,0.0160362829,0.999899992,0.9935182315,0.02118419474
+20,0.006499269942,0.999887491,0.9934544764,0.008630690455
+24,0.002587707017,0.9998849908,0.9934232239,0.003452776222
+28,0.001065085207,0.999887491,0.9934157233,0.001385110809

data/refresh_learned.csv

@@ -1,9 +1,25 @@
 block,legit_ser,eve_ser
-0,0.06381666667,0.9999375
-1,0.06395583333,0.9981941667
-2,0.06397833333,0.999985
-3,0.06386833333,0.9983758333
-4,0.06338666667,0.9999908333
-5,0.06366416667,0.9999133333
-6,0.06398416667,0.9999433333
-7,0.06390583333,0.9809091667
+0,0.06113666667,0.9990225
+1,0.06151666667,0.99906
+2,0.06108666667,0.9999341667
+3,0.0615975,0.9995841667
+4,0.06189833333,0.9996233333
+5,0.0608925,0.9997391667
+6,0.06110333333,0.9999591667
+7,0.06100083333,0.994235
+8,0.06136166667,0.9995625
+9,0.06109166667,0.999875
+10,0.06130416667,0.9996441667
+11,0.06130916667,0.9975941667
+12,0.06105666667,0.9999325
+13,0.06113166667,0.9998916667
+14,0.06178,0.9994783333
+15,0.06124583333,0.9997775
+16,0.06141083333,0.9998683333
+17,0.0612175,0.99964
+18,0.06126833333,0.9997041667
+19,0.06113916667,0.9999766667
+20,0.06117583333,0.9994191667
+21,0.06127166667,0.9999558333
+22,0.06118416667,0.9995833333
+23,0.06149666667,0.9997783333

data/refresh_summary.csv

@@ -2,4 +2,4 @@ scheme,legit,eve,entropy_bits
 None (fixed key),0.05300666667,0.9997075,23.76910417
 Fresh orthogonal keys,0.1215548611,0.9996535069,23.76910417
 "Invariant, KM (str.)",0.05304121528,0.9995528819,364.5801064
-"Invariant, KM (lrn.)",0.063800,0.997200,364.5801064
+"Invariant, KM (lrn.)",0.061279,0.998895,364.5801064

data/sec_brute_learned.csv

@@ -1,15 +1,15 @@
 K,ser_mask
-1,0.9967928683
-3,0.9934662748
-10,0.9869523182
-30,0.9769415244
-100,0.961966217
-300,0.9457608404
-1000,0.9257560018
-3000,0.8881725568
-10000,0.8471349462
-30000,0.8132026814
-65536,0.790006818
-100000,0.7797171991
-300000,0.7435325695
-1000000,0.7188559867
+1,0.9967152519
+3,0.9933139604
+10,0.9866396597
+30,0.976371298
+100,0.9609886779
+300,0.9443205866
+1000,0.92371647
+3000,0.8849669152
+10000,0.8426466786
+30000,0.8076524286
+65536,0.7837337919
+100000,0.7731184695
+300000,0.7358238662
+1000000,0.7103847689

data/sec_compare.csv

@@ -4,4 +4,4 @@ public_mask,0.0529425,0.0529425,0.0529425,0.83882
 perm_key,0.0528525,0.9999925,0.0528525,0.36425
 index_cipher,0.0529425,0.9999847412,0.9999847412,0.83882
 oma_plain,0.08056383667,0.08056383667,0.08056383667,nan
-proposed_learned,0.06358416667,0.9998083333,0.9999833333,0.4046066667
+proposed_learned,0.06156083333,0.9996875,0.9999841667,0.3973533333

data/sec_jam_learned.csv

@@ -1,8 +1,17 @@
 jsr_db,blind,matched,nojam
--10,0.117582,0.40832,0.062852
--5,0.215158,0.657572,0.062852
-0,0.404244,0.851282,0.062852
-5,0.648786,0.94662,0.062852
-10,0.839218,0.982652,0.062852
-15,0.939304,0.994184,0.062852
-20,0.979206,0.99818,0.062852
+-10,0.114422,0.38569,0.060494
+-8,0.143006,0.483718,0.060494
+-6,0.183328,0.586128,0.060494
+-4,0.239698,0.685262,0.060494
+-2,0.310968,0.772466,0.060494
+0,0.398118,0.840574,0.060494
+2,0.493594,0.893422,0.060494
+4,0.594688,0.929018,0.060494
+6,0.686344,0.953816,0.060494
+8,0.768698,0.970236,0.060494
+10,0.835132,0.980904,0.060494
+12,0.886322,0.98835,0.060494
+14,0.923646,0.992284,0.060494
+16,0.949318,0.995096,0.060494
+18,0.967192,0.996858,0.060494
+20,0.978764,0.998038,0.060494

data/sec_keylen_learned.csv

@@ -1,9 +1,9 @@
 L,d,legit_ser,eve_ser,mask_xcorr,oma
-8,32,0.9297855,0.9998735,0.007307400461,0.6849191155
-12,48,0.416604,0.9997065,0.005153660662,nan
-16,64,0.2762895,0.999912,0.007116591092,0.2747696909
-20,80,0.2076175,0.999383,0.003162040841,0.2289444229
-24,96,0.1829615,0.999894,0.002973971656,0.1961714033
-32,128,0.131901,0.999616,0.005575809628,0.1524639978
-48,192,0.090206,0.9994575,0.005743456539,0.1054308944
-64,256,0.0635265,0.999637,0.006678360514,0.08056383667
+8,32,0.9594105,0.99997,0.003249221947,0.6849191155
+12,48,0.4157085,0.999767,0.003225991037,nan
+16,64,0.315952,0.9994685,0.01121100038,0.2747696909
+20,80,0.213059,0.9996415,0.00134725438,0.2289444229
+24,96,0.183031,0.9997345,0.001526024193,0.1961714033
+32,128,0.1231225,0.999805,0.002514706925,0.1524639978
+48,192,0.084432,0.999786,0.001822981867,0.1054308944
+64,256,0.061412,0.9993955,0.002381352475,0.08056383667

data/sec_keylen_perm.csv

@@ -0,0 +1,9 @@
+L,d,legit_ser,eve_ser
+8,32,0.948556,0.999984
+12,48,0.414228,0.99995
+16,64,0.258384,0.999992
+20,80,0.187222,0.999986
+24,96,0.152004,0.999972
+32,128,0.10816,0.999976
+48,192,0.071734,0.999966
+64,256,0.053092,0.999976

data/sec_sens_learned.csv

@@ -1,14 +1,14 @@
 frac,ser_mask
-0,0.9999754167
-0.2,0.9977270833
-0.4,0.9502445833
-0.6,0.6829479167
-0.75,0.31458375
-0.85,0.1250175
-0.9,0.09168625
-0.92,0.08508416667
-0.94,0.07654291667
-0.955,0.0716375
-0.97,0.06938458333
-0.985,0.06607833333
-1,0.0636775
+0,0.9999529167
+0.2,0.99770125
+0.4,0.9489816667
+0.6,0.6733195833
+0.75,0.29680625
+0.85,0.1140279167
+0.9,0.08714708333
+0.92,0.07980791667
+0.94,0.07278666667
+0.955,0.06856208333
+0.97,0.06626333333
+0.985,0.06358166667
+1,0.06154625

data/sec_snr_learned.csv

@@ -1,12 +1,12 @@
 snr_db,legit,eve_wrong
-0,0.45205625,0.999865625
-2,0.325615625,0.9998425
-4,0.224483125,0.9998196875
-6,0.150013125,0.999801875
-8,0.0982703125,0.9997996875
-10,0.06379375,0.9997821875
-12,0.0409540625,0.999758125
-14,0.0259790625,0.9997684375
-16,0.0165590625,0.999769375
-18,0.01049375,0.999760625
-20,0.006573125,0.9997428125
+0,0.439821875,0.999816875
+2,0.31516125,0.9997828125
+4,0.217330625,0.9997790625
+6,0.1449359375,0.9997196875
+8,0.0947978125,0.99971375
+10,0.0611025,0.999680625
+12,0.039268125,0.9996478125
+14,0.0251025,0.999646875
+16,0.0159153125,0.999625625
+18,0.0100140625,0.9996171875
+20,0.0063884375,0.999608125

data/verify_math.csv

@@ -15,4 +15,4 @@ V7 symbolic identities,exact,exact,0,0,PASS
 V8 cross-period remainder,0.0,0.000337,0.000337,0.0005,PASS
 V9 score-variance ratio,2.8,2.8252,0.0252,0.05,PASS
 V10 format-matched OMA at 10 dB,0.055,0.05520,0.00020,0.001,PASS
-V11 OMA closed form vs Monte Carlo,0.081118,0.080925,0.0024,0.01,PASS
+V11 OMA closed form vs Monte Carlo,0.081245,0.080925,0.0039,0.01,PASS

data/whygap.csv

@@ -0,0 +1,4 @@
+family,legit_ser,kappa_bar,entry_use
+structured (frozen),0.05259,0.00000,1.0000
+learned (free start),0.06363,0.00444,0.0916
+learned (Walsh start),0.06489,0.00478,0.2797