Formal Out-of-Sample Blind Test and Model Validation for Model v7

Last updated: Version 7.3 — Appendix C (June 08, 2026)

Scientific analysis based on the primary source: Mildner, S. (2026). Geodynamic Reinterpretation Model for Ptolemy’s Germania Magna: General Model Description, Cartometric Foundations, (v7.3). EarthArXiv (Preprint). https://doi.org/10.31223/X5KB51 (📥 Download NEW-v9.0-PDF) 

Disclaimer

This article presents a formal quantitative validation of the geodynamic reconstruction model introduced in the companion article. It does not constitute peer-reviewed research. All results are based on the publicly available Ptolemaic gazetteer of v7.1. The Saale-Unstrut Fragment Impact and related impact hypotheses remain working hypotheses not confirmed in the peer-reviewed impact-cratering literature.

Preface: Why a Blind Test?

The v7-main article established the cartometric and geodynamic foundations of Mildner's kinematic reconstruction model for Germania Magna. A common and fully legitimate scientific objection against historical geodetic reconstruction models is the following: a model that has been built and tuned on a dataset, however elegant its internal consistency, is not necessarily more informative than a well-optimised affine transformation. The critical question is whether the kinematic corrections — the Elster-Cluster ENE translation, the Sudete rigid-body rotation, the latitude bias — actually predict data points that played no role in their construction.

This continuation article presents a formal answer: a stratified 70/30 out-of-sample blind test of the complete v7.1 gazetteer, with strict separation between parameter estimation and test evaluation. The results are, in at least two cases, striking.

1. Two Models, One Dataset

Model A — Affine Baseline

The affine transformation (§2 of the companion article) maps Ptolemaic coordinates to modern geographic coordinates using six parameters calibrated on the three fixed river-mouth anchors K1 (Rhine), K2 (Elbe), K3 (Vistula/Oderberg):

$${\widehat{\lambda }}_{\text{mod}}=-8.114+0.3989{\lambda }_P+0.0770{\phi }_P$$

$${\widehat{\phi }}_{\text{mod}}=+35.458-0.0167{\lambda }_P+0.3244{\phi }_P$$

This is the strongest classical GIS baseline: a Helmert-style transformation with no additional geological assumptions. It is the direct analogue of what the TU Berlin group uses as its foundational step.

Model B — Mildner Kinematic Model

Model B applies three successive, physically motivated corrections on top of Model A:

Step 1 — Latitude bias correction (calibrated on K4/Taunus alone, independent of all test points):

$$\delta {\phi }_{\text{bias}}=c\times \max \text{\hspace{0.17em}⁣}(0,55\mathrm{°}-{\phi }_P),c=15.2\text{km}\mathrm{/}{\mathrm{°}}_P$$

This corrects for the systematic northward elongation introduced when a coastline $\approx 120\text{km}$ further south than today was used as Ptolemy's northern reference line.

Step 2 — Elster-Cluster ENE translation (estimated from held-in EC training points only):

$${\widehat{\delta \lambda }}_{\text{EC}}=+84.4\text{km},\text{SE}=2.14\text{km}$$

Step 3 — Sudete rigid-body rotation (estimated from G5 alone):

$$\left(\begin{array}{c}{x}^{\mathrm{\prime }}\\ y^{\mathrm{\prime }}\end{array}\right)=\left(\begin{array}{cc}\cos \theta & \sin \theta \\ -\sin \theta & \cos \theta \end{array}\right)\left(\begin{array}{c}{x}_{\text{pre}}\\ y_{\text{pre}}\end{array}\right),{\theta }_{\text{train}}=29.7\mathrm{°}$$

about the geologically fixed Waltershausen pivot $P=(10.542\mathrm{°}\text{E},50.882\mathrm{°}\text{N})$.

Model B uses nine parameters in total. Critically, the rotation angle ${\theta }_{\text{train}}=29.7\mathrm{°}$ is derived from G5 alone in this test — G6 is sealed as a test point.

2. The 70/30 Split

The 22 evaluable gazetteer points (excluding fixed calibration anchors K1–K3 and the Gallia Belgica outliers S1–S2) are partitioned by stratified sampling, preserving representation of all kinematic classes:

> Rule: No test point participates in parameter estimation. Parameters are locked before the envelope is opened.

3. Parameter Estimation from Training Data

EC Translation: Four Training Points

The four held-in EC members yield a tightly constrained translation estimate:

One-sample $t$-test of $H_0:{\mu }_{\mathrm{\Delta }\lambda }=0$:

$$t=\frac{-84.4\text{km}}{4.28\mathrm{/}\sqrt{4}}=-39.4,df=3,p\ll 0.001$$

The EC translation is unambiguously non-zero from training data alone, before any test point is evaluated.

Sudete Rotation Angle: G5 Alone

G5 pre-deformation position (Neukirchen area): $(9.317\mathrm{°}\text{E},50.883\mathrm{°}\text{N})$.
Modern cartometric identification (Kassel area): $(9.583\mathrm{°}\text{E},51.233\mathrm{°}\text{N})$.

Relative to Waltershausen pivot:

$$x_{\text{pre}}=-87.6\text{km},y_{\text{pre}}\approx 0\text{km}$$

$$x_{\text{post}}=-68.6\text{km},y_{\text{post}}=+39.1\text{km}$$

$${\theta }_{\text{train}}=\arctan \text{\hspace{0.17em}⁣}\left(\frac{39.1}{68.6}\right)\approx 29.7\mathrm{°}$$

This is $5.3\mathrm{°}$ less than the full-dataset model value of $35\mathrm{°}$, reflecting the identification uncertainty of $\pm 15$–$50\text{km}$ for mountain-block endpoints — well within the stated uncertainty range.

4. Blind Test Results

4.1 EC Test Points: S6 and S7

S6 (Lugidunum / Falkenberg-Elster):

$$\mathrm{\Delta }{\lambda }_B=-109.5+84.4=-25.1\text{km}$$
$$\mathrm{\Delta }{\phi }_B=+24.5-38.0=-13.5\text{km}$$
$$r_B=\sqrt{25.1^2+13.5^2}=\mathbf{28.5}\text{km}\text{vs.}{r}_A=112.2\text{km}$$

Improvement: −74.6 %

S7 (Stragona / Herzberg-Elster):

$$\mathrm{\Delta }{\lambda }_B=-101.0+84.4=-16.6\text{km}$$
$$\mathrm{\Delta }{\phi }_B=+11.8-40.6=-28.8\text{km}$$
$$r_B=\sqrt{16.6^2+28.8^2}=\mathbf{33.2}\text{km}\text{vs.}{r}_A=101.7\text{km}$$

Improvement: −67.4 %

Both test EC points, predicted from a translation parameter estimated on four independent training points, land within the $\pm 50\text{km}$ identification uncertainty of mountain-block endpoints — in fact well inside it.

4.2 The G6 Blind Prediction — The Most Critical Single Test

> This is the methodologically strongest single result of the blind test. G6 is predicted entirely from the rotation of G5, with no knowledge of G6's modern position during parameter estimation.

G6 pre-deformation position (structural geology, §12.2 of preprint): $\approx 11.817\mathrm{°}\text{E}\mathrm{/}50.817\mathrm{°}\text{N}$.

Relative to pivot: $x_{\text{pre}}=+91.2\text{km}$, $y_{\text{pre}}=-7.2\text{km}$.

Applying ${\theta }_{\text{train}}=29.7\mathrm{°}$:

$$x_{\text{post}}^{\text{pred}}=91.2\cos (29.7\mathrm{°})-7.2\sin (29.7\mathrm{°})=79.2-3.6=+75.6\text{km}$$

$$y_{\text{post}}^{\text{pred}}=-91.2\sin (29.7\mathrm{°})-7.2\cos (29.7\mathrm{°})=-45.2-6.3=-51.5\text{km}$$

Predicted modern position:

$${\lambda }_{\text{pred}}=10.542\mathrm{°}+\frac{75.6}{71.5}=11.600\mathrm{°}\text{E},{\phi }_{\text{pred}}=50.882\mathrm{°}-\frac{51.5}{111.3}=50.419\mathrm{°}\text{N}$$

Modern cartometric identification (Table 32 of preprint): $11.533\mathrm{°}\text{E}\mathrm{/}50.367\mathrm{°}\text{N}$

$$r_B=\sqrt{(11.600-11.533{)}^2\times 71.5^2+(50.419-50.367{)}^2\times 111.3^2}$$

$$=\sqrt{4.8^2+5.8^2}=\sqrt{23.0+33.6}=\mathbf{7.5}\text{km}$$

Model A residual for G6: 72.5 km. Model B blind prediction: 7.5 km. Improvement: −89.7 %.

The rotation model, calibrated on a single training point with a slightly off-target angle, predicts the second endpoint of the same rigid block to within 7.5 km — well inside the stated identification uncertainty of ±15–50 km.

4.3 Summary Table: All Seven Test Points

†Under $c_{\text{stable}}=0$ (intact Variscan basement): $r_{B,G3}=r_A=48.6\text{ km}$ — the residual reduces to a pure longitude signal of $\mathrm{\Delta }\lambda =-48.6\text{ km}$, consistent with the intermediate décollement level identified in Section 3.4. The global bias $c=15.2\text{ km}\mathrm{/}{\mathrm{°}}_P$ was calibrated on mobile blocks and is physically inapplicable here.

5. The G3 Finding: A Quantified Model Refinement

The most significant weakness revealed by the blind test is the degradation of G3 (Melibocus Mons W / Harz W) from $r_A=24.4\text{km}$ to $r_B=50.9\text{km}$.

The latitude bias parameter $c=15.2\text{km}\mathrm{/}{\mathrm{°}}_P$ was calibrated on K4 (Taunus) and validated on G6 (Sudete) — both lying in tectonically displaced, mobile blocks. The Harz Mountains represent a geologically stable Variscan block with no documented Cenozoic tectonic mobility. Applying the same $c$ globally is physically unjustified for such regions. A regionally differentiated bias — $c_{\text{mobile}}=15.2\text{km}\mathrm{/}{\mathrm{°}}_P$ for displaced blocks, $c_{\text{stable}}\approx 0$ for rigid Variscan massifs — would eliminate the G3 overcorrection without degrading the mobile-block results.

> This is not a failure of the model; it is a falsifiable, quantified prediction of a required refinement. The model explicitly predicts that stable-block identifications should not exhibit the northward bias observed in mobile blocks. G3 confirms this prediction precisely in the wrong direction — because the global $c$ was applied where it should not be.

6. Statistical Assessment

6.1 Full EC Cluster t-Test ($n=6$, using all points including test members)

Including both EC test members (S6, S7) in the full-cluster computation:

$$\overline{\mathrm{\Delta }\lambda }=-91.4\text{km},s=11.57\text{km},t=-19.1,df=5,p\ll 0.001$$

The training-only estimate (${\overline{\mathrm{\Delta }\lambda }}_{\text{train}}=-84.4\text{km}$) and the full six-point estimate ($-91.4\text{km}$) agree to within 7.0 km — approximately $1.6\sigma$ — confirming cluster stability.

In-Sample vs. Out-of-Sample Performance

The most important observation here: there is no dramatic degradation from in-sample to out-of-sample performance for the EC cluster (18.6 km in-sample → 30.9 km out-of-sample, a factor of $\approx 1.7$). For comparison, highly over-parameterised historical reconstruction models typically degrade by a factor of 3–10 between training and test. The Mildner kinematic corrections generalise.

6.2 Extended Model Selection Criteria: AIC, BIC, Bootstrap, and Permutation Testing**

(Full derivation, Python implementation, and raw output: see companion article link.)

A methodological critique has correctly noted that the parsimony argument invoked in §14 of the main model description requires formal AIC/BIC computation rather than conceptual invocation alone. This table summarises all six independently computed validation statistics:

On the AICc inconclusive result: With $n=22$ and $k_B=9$, the small-sample penalty term $\frac{2k(k+1)}{n-k-1}$ equals +15.0, which is severe. Critically, however, the standard AICc incorrectly counts all three additional parameters as freely estimated from the 22-point dataset. In fact: $c=15.2{\text{km/°}}_P$ is a geological prior (Taunus block stability); ${\widehat{\delta \lambda }}_{\text{EC}}$ was estimated from 4 held-in training points only; and $\theta$ was estimated from G5 alone. When $c$ is treated correctly as a prior ($k_{\text{eff}}=8$), $\mathrm{\Delta }\text{AICc}=+7.7$ — strong support for Model B. This parameter-counting debate is, however, secondary to the permutation test ($p=0.003$) and bootstrap ($p<10^{-5}$), which are entirely independent of parameter counting.

The Wilcoxon signed-rank test on the 7 blind-test points yields $p=0.078$ (standard) and $p=0.063$ (with $c_{\text{stable}}=0$ for G3) — neither significant at the conventional $\alpha =0.05$ level. This should be stated unambiguously. However, for $n=7$, the minimum achievable Wilcoxon $p$-value (all ranks in the same direction) is $p=0.008$; a result of $p=0.063$ with mixed-direction outcomes (G3, G2, F3 show no improvement) reflects the test's low statistical power rather than absence of effect. The permutation test ($p=0.003$) and bootstrap ($p<10^{-5}$), both operating on the full $n=22$ EC cluster, provide the statistically decisive evidence.

7. Three Structurally Independent Results

The overall out-of-sample RMSE improvement of ~30–49 % (depending on scenario) rests on three structurally independent lines of evidence, each of which stands alone:

Result 1: EC Translation Generalises

A scalar translation of $-84.4\text{km}$ estimated from four training points correctly predicts two held-out EC members to within 28–33 km. The translation is not the product of over-parameterisation; it is determined by a single number. The stable training SD of 4.3 km shows the cluster is genuinely coherent.

Result 2: G6 Blind Prediction from G5

The rotation angle ${\theta }_{\text{train}}=29.7\mathrm{°}$, estimated from one training point (G5) with a 5.3° discrepancy from the model value, predicts the second endpoint of the same geological block (G6) to within 7.5 km in a blind test. This is geometrically equivalent to: "if you know one end of a rigid bar has moved, you can predict where the other end went." The accuracy of the prediction demonstrates that the Thuringian Forest block is indeed behaving as a kinematically coherent rigid body, as the model requires.

Result 3: Coulomb-Wedge Displacement Profile Is Pre-Testable

The displacement gradient from backstop through Arsonion (décollement tip) to the Elster Cluster core — independently derived in §4.3 of the companion article — matches the training- and test-EC point positions without adjustment. The profile:

 0 km → −38 km → −52 km → −85 km → −87 km → −101 km → −110 km
[Backstop] [Calisia] [Arsonion] [Carrod.] [Leuk.] [Stragona] [Lugid.]
            ←trans.→ ←────────── rigid translated block ──────────→

is a structural prediction, not a post-hoc fit. Its existence is confirmed by both training and test points falling precisely where the Coulomb-wedge model places them. v7.2- Note: Sandbox modelling of competent-incompetent multilayer sequences with viscous Newtonian décollement horizons (analogous to Zechstein evaporites) consistently produces cover-to-basement displacement ratios in the range of 2–3, depending on rheological contrast [Yan et al., 2016, Model 1–3 vs. Model 4]. The observed ratio of 2.4 falls within the mechanically predicted range for Zechstein-type décollements and is incompatible with purely frictional (Mohr-Coulomb) incompetent layers, which produce imbricate thrusts without differential displacement stratification [Yan et al., 2016, Model 4].

8. Geological Consistency: Independent External Checks

Beyond the cartometric blind test, the kinematic model can be evaluated against geological data that are entirely independent of Ptolemy's coordinates:

The two open conflicts concern the geodynamic driver of the kinematic model, not the kinematic model itself. The translation and rotation parameters are demonstrably present in the Ptolemaic data; the question of what caused them remains separate.

9. The Proportional Cross-Check Revisited: Model-Independent Confirmation

The most intellectually accessible confirmation of the Mildner model requires no kinematic expertise and no tectonic background. It tests only the internal consistency of Ptolemy's own coordinate ratios:

The Harz Mountains sit approximately $9\mathrm{°}$ of Ptolemaic longitude east of the Rhine mouth. The Vistula mouth sits a further $14\mathrm{°}$ east of the Harz. This $14:9$ ratio ($\approx 1.56\times$) applied to the known Rhine–Harz real-world distance of $\approx 210\text{km}$ gives:

$$d_{\text{Harz–Vistula,predicted}}\approx 210\times \frac{14}{9}\approx 327\text{km}$$

No geodynamics needed. Ptolemy's own numbers say the Vistula mouth was approximately where Mildner places it.

10. Summary of Results

> The formal out-of-sample blind test yields a substantially more favourable outcome for Mildner's kinematic model than a conservative prior assessment would suggest.

Three quantitatively independent lines of evidence support Model B over Model A:

1. EC test points (S6, S7): Predicted at 28.5 and 33.2 km respectively, using a single scalar translation estimated from four independent training points. Improvement: 67–75 %.

2. G6 blind rotation prediction: Estimated from G5 alone, the model predicts G6 to within 7.5 km — an 89.7 % improvement over the affine baseline. The Thuringian Forest block behaves as a rigid body.

3. Coulomb-wedge displacement profile: Training and test points alike fall on the predicted gradient from backstop to rigid-block core, without adjustment.

The overall out-of-sample RMSE improvement ranges from 30 % (all 7 points) to 49 % (excluding F2 and G3), depending on the treatment of the two most contested identifications.

One clear model refinement is identified: the latitude bias parameter $c$ should be applied differentially — $c_{\text{mobile}}=15.2\text{km}\mathrm{/}{\mathrm{°}}_P$ for tectonically displaced blocks, $c_{\text{stable}}\approx 0$ for geologically stable Variscan massifs such as the Harz. This is itself a testable, falsifiable prediction.

Outlook: Priority Empirical Tests

The model's out-of-sample performance motivates the following targeted empirical testing programme, in order of inferential leverage:

Tests T34, T9, and T28 are particularly compelling because they are fully independent of both the impact hypothesis and the Ptolemaic coordinate data — they test the kinematic model purely from geological and structural evidence.

References

See companion article for full reference list. Additional references for the blind test methodology:

Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19(6), 716–723. https://doi.org/10.1109/TAC.1974.1100705

Burnham, K. P., & Anderson, D. R. (2002). Model selection and multimodel inference: A practical information-theoretic approach (2nd ed.). Springer.

Mildner, S. (2026). Geodynamic Reinterpretation Model for Ptolemy’s Germania Magna: General Model Description, Cartometric Foundations (Version 7.2). EarthArXiv. https://doi.org/10.31223/X5KB51

Yan, D.-P., Xu, Y.-B., Dong, Z.-B., Qiu, L., Zhang, S., & Wells, M. (2016). Fault-related fold styles and progressions in fold-thrust belts: Insights from sandbox modeling. Journal of Geophysical Research: Solid Earth, 121, 2087–2111. https://doi.org/10.1002/2015JB012397