Executive scientific verdict
GFIS remains a methane-yield prediction and decision-support system. Gate 3 adds a reproducible public-data experiment chain, but DS-03 evaluates total-biogas dynamics rather than measured methane. On that dataset, the current LSTM is not promoted: persistence won two of three rolling-origin folds. The LSTM remains a short-window challenger while methane-specific external validation continues separately.
Fold wins: persistence 2, LSTM 1. Selection was locked before the final test was opened.
This boundary applies only to the evaluator-suggested public dataset: methane fraction is 99.66% missing, so its total-biogas target is not relabelled as methane. GFIS methane prediction remains implemented and is demonstrated separately in the simulator evidence section.
Evaluator-suggested dataset: experimental impact
Gate 3 selected the catalogue's rank-1 public source—Mendeley DS-03, farm-scale digester process and weather data—in direct response to the evaluation requirement for real/open evidence and temperature sensitivity. The evaluator guidance motivated the source category and experiment; it does not imply that the evaluator authored the dataset.
Guidance
Use real/open anaerobic-digestion data, disclose limitations, test temperature sensitivity, compare LSTM with strong baselines, and keep claims traceable.
Gate 3 experiment
Immutable DS-03 ingestion → quality audit → rolling-origin persistence/XGBoost/LSTM comparison → natural drift study → locked test.
Changed understanding
Real farm-scale evidence did not justify promoting the LSTM. It exposed target and metadata limits, regime instability, and the need for a second licensed methane-rich dataset.
Before DS-03
- Synthetic evidence could demonstrate workflow, not field validity.
- “Biogas” risked being discussed too loosely as methane.
- A favourable aggregate score could hide unstable regimes.
- Temperature robustness and experiment replay were not evidenced end to end.
After DS-03
- Four reactor identities and 51,766 measured weather records are traceable.
- The evaluated target is explicitly total biogas; methane is 99.66% missing.
- Three chronological folds reveal persistence wins two folds and LSTM one.
- Natural drift, repeated seeds, checksums, predictions, epochs and checkpoints are retained.
Experiment-to-learning traceability
| Experiment or audit | Evidence produced | What it changed in GFIS understanding |
|---|---|---|
| Source and schema audit | Four farm-scale reactors; mixed Excel dates; duplicates, gaps and structural missingness documented. | Catalogue descriptions must be verified against files. GFIS now repairs dates transparently, flags exclusions and preserves reactor identity. |
| Target audit | Daily/interval total-biogas field usable; methane fraction 99.66% missing; VFA/ALK and required VS/BMP fields unavailable. | DS-03 cannot validate methane yield, VFA/ALK soft sensing or methane physics loss. Physics-violation rate is unavailable—not zero. |
| Leakage-safe rolling comparison | Persistence wins folds 1 and 3; LSTM wins fold 2; all models use identical chronological splits. | Temporal learning is regime-dependent. GFIS keeps persistence as the DS-03 champion and the compact LSTM only as a challenger. |
| Sequence windows 3/7/14/28 | Window differences are small relative to fold-to-fold dispersion. | No evidence supports a larger model zoo or Transformer. Short memory is the defensible current choice. |
| Natural temperature drift | RMSE degradation is −21.2%, +29.3% and −21.2% across folds. | Temperature association is inconsistent; Gate 3 makes no causal temperature claim and labels observed versus synthetic stress separately. |
| Locked test and experiment memory | Checksummed data, split version, seeds, epochs, predictions, environment, checkpoints and replay artifacts. | A single favourable final score cannot override rolling evidence; every dissertation claim can now be replayed and audited. |
Input provenance and integrity
The selected source is Mendeley Data V1, DOI 10.17632/gk3f363sfg.1, contributed by Daniel Mulat and licensed CC BY 4.0.
Immutable source package
Title: Process monitoring and weather datasets on farm-scale biogas digesters
Archive: 12,005,187 bytes
Archive SHA-256
{hash_box(run_manifest['dataset_archive_sha256'])}Canonical output
Schema: gfis-gate3-ds03-v1
Processed SHA-256
{hash_box(run_manifest['processed_data_sha256'])}Raw files are preserved; processing is additive.
Open input and processing artifacts
{linked_file_table(source_rows)}Figure 1. Four reactor histories and naturally observed air-temperature trajectories. Reactor identity is retained throughout.
Data quality and target audit
Figure 2. Predictor and target-field missingness. Methane coverage is scientifically unusable.
Acceptance rules
- One-day intervals only for comparable daily forecasting.
- Duplicate reactor timestamps excluded.
- Targets are never imputed.
- Predictor imputation and scaling are fitted within each training fold.
- R1–R4 identity is retained with group-aware sequence creation.
- Mixed Excel dates are repaired monotonically and flagged.
Integrated processing and evaluation flow
Chronological split
Split version: {split['version']}
Final test: {split['final_test_start']} to {split['final_test_end']}
The last 15% of dates stayed sealed during architecture selection.
{table(pd.DataFrame(split['rolling_folds']))}Past-only features
- Prior total-biogas value
- Past 3-observation mean and standard deviation
- Feed and water values
- Observed air temperature
- Calendar sine/cosine
- Explicit reactor identity
No future target or future-fitted preprocessing enters a fold.
Equations, derivations, and statistical definitions
These equations define exactly what the Gate 3 code computes. Symbols are indexed by reactor r, observation time t, rolling fold k, and random seed s.
1. Target construction and eligibility
B is the cumulative biogas-meter reading. The source workbook supplies this difference as “Daily Biogas (mL).” Gate 3 interprets it as interval total biogas and accepts it as daily only when Δt = 1 day.
Only observations with 𝓔 = 1 enter model evaluation. Missing targets are never imputed.
2. Past-only temporal features
The most recently observed total-biogas interval.
Three-observation past-only rolling mean.
Sample rolling standard deviation. The current or future target never enters these features.
Cyclic calendar encoding prevents an artificial discontinuity between 31 December and 1 January.
3. Fold-local preprocessing derivation
The imputation median m, mean μ, and standard deviation σ are estimated only from the current training fold. Validation and test observations are transformed but never used to fit preprocessing.
4. Persistence baseline
A temporal model must outperform this operationally meaningful baseline on leakage-safe validation before an improvement claim is accepted.
5. XGBoost objective
Each tree fits residual structure from the current ensemble; Ω regularizes tree complexity. All lag and rolling predictors remain past-only.
6. Current LSTM state derivation
ft = σ(Wxfxt + Whfht−1 + bf)
gt = tanh(Wxgxt + Whght−1 + bg)
ot = σ(Wxoxt + Whoht−1 + bo)
ct = ft ⊙ ct−1 + it ⊙ gt
ht = ot ⊙ tanh(ct)
ŷt = wyTht + by
σ is the logistic sigmoid and ⊙ is element-wise multiplication. Gate 3 uses one layer, hidden size 32, no random shuffling, and windows of 3, 7, 14, and 28 observations.
Targets are standardized from training data only. Adam minimizes scaled MSE; early stopping retains the state with lowest validation MSE.
7. Reported error metrics
sMAPE terms with a zero denominator are excluded. Negative R² means the model is worse than predicting the evaluation-period mean.
8. Repeated-seed aggregation and dispersion
S = 3 seeds (17, 29, 43). The report presents mean and sample standard deviation across fold/seed observations.
9. Natural temperature-drift definition
Positive degradation means worse error in naturally outlying temperature conditions. This is an association test, not a causal temperature derivation.
10. Robust architecture-selection rule
Ties are resolved by median fold RMSE and then dispersion. The rule was applied before opening the final test. Persistence won two folds; LSTM won one.
11. Future methane-physics formulation — not evaluated here
This residual/physics loss requires measured methane, substrate VS/BMP, and stability variables. DS-03 lacks them, so Gate 3 does not compute, tune, or claim this loss.
Rolling-origin model evidence
A single mean score would conceal the extreme second regime. Model selection therefore uses fold wins, then median fold RMSE and dispersion.
Figure 3. Fold-level RMSE on a logarithmic axis. Persistence is strongest in folds 1 and 3; LSTM is strongest only in fold 2.
Figure 4. Sequence-window changes are negligible relative to fold-to-fold dispersion.
Complete model-window rolling summary
{table(window_table)}Locked final-period comparison
Figure 5. The LSTM has slightly lower final-test RMSE, but much worse sMAPE and was not the rolling-validation champion.
Figure 6. Seed-averaged final-period predictions. Symlog scaling exposes both normal and extreme intervals.
Training sessions and epoch evidence
Every LSTM epoch is retained with run, fold, seed, window, training loss, and validation loss. Final-fit LSTM and XGBoost checkpoints are checksum-registered.
Figure 7. Window-3 rolling LSTM validation traces across all folds and seeds.
Experiment scale
{len(folds)} fold/seed/model/window metric records
{len(epochs)} epoch records
{len(pd.read_csv(run / 'predictions/rolling_predictions.csv')):,} rolling prediction rows
{len(final_predictions):,} final prediction rows
Model configuration
{code_block(config, "Click to inspect model and training configuration")}Open source code and configuration
{linked_file_table(code_rows)}Natural temperature-drift study
Drift observations fall outside each training fold's 10th–90th percentile temperature range. No artificial perturbation is presented as plant data.
Figure 8. Degradation is inconsistent across folds: −21.2%, +29.3%, and −21.2%. No causal temperature claim is supported.
Persistent experiment memory
The accepted run binds source identity, checksums, Git commit, split version, seeds, windows, environment, configurations, predictions, epoch histories, plots, checkpoints, and replay instructions.
Run identity
{run_id}
Git commit: {run_manifest['git_commit']}
Run manifest SHA-256: {sha256(run / 'run_manifest.json')}
Environment
{code_block(environment, "Click to inspect package and runtime versions")}Complete clickable artifact inventory and checksums
{linked_file_table(artifact_rows)}Exact replay
The accepted run was replayed. Fold metrics, summaries, final-test metrics, scientific decision, drift results, prediction files, and epoch history matched byte-for-byte.
{code_block("""PYTHONPATH=01_Product_Source/GFIS_Project python3 \\ 01_Product_Source/GFIS_Project/scripts/gate3_prepare_dataset.py \\ --workbook "03_Datasets/01_Raw_Public_Datasets/DS03_Mendeley_gk3f363sfg_v1/extracted/Farm-scale_Biodigester_Perfromance_dataset/Farm-scale Biodigester 2024_Performance.xlsx" \\ --weather-workbook "03_Datasets/01_Raw_Public_Datasets/DS03_Mendeley_gk3f363sfg_v1/extracted/Farm-scale_Biodigester_Perfromance_dataset/Mazingira main(z6-02152)-1729245757_weather_data.xlsx" \\ --output 03_Datasets/05_Processed_Final/DS03/gfis_ds03_canonical_v1.csv \\ --manifest 03_Datasets/05_Processed_Final/DS03/processing_manifest_v1.json PYTHONPATH=01_Product_Source/GFIS_Project python3 \\ 01_Product_Source/GFIS_Project/scripts/gate3_run_experiment.py \\ --run-id G3-E01-20260724T1530Z-ds03-v3-replay""", "Click to view exact replay commands")}GFIS capabilities working end to end
The simulator evidence below is not a conceptual mock-up. The trained GFIS service bundle was loaded and executed for {simulator_metadata['scenario_count']} controlled scenarios × {simulator_metadata['hours_per_scenario']} hours, producing {simulator_metadata['trace_row_count']} hour-indexed output rows. UI screenshots show the corresponding Control Room, scenario handoff, alarm state and experiment-memory surfaces.
Fresh Version 2 replay
Run: {simulator_run_id}
Trained model bundle loaded: {str(simulator_metadata['trained_bundle_loaded']).lower()}
Model SHA-256:
{hash_box(simulator_metadata['model_bundle_sha256'])}Evidence label: {simulator_metadata['evidence_label']}
What “working” means here
- Python service executed predictions and stateful 48-hour traces.
- VFA/ALK warning states changed under overload stress.
- A low-VS boundary challenge triggered and recorded the physics constraint.
- Every output row, configuration identity and checksum is retained.
- This is controlled synthetic replay evidence—not measured plant validation.
Figure 9. Fresh Version 2 backend replay. Methane trajectories, VFA/ALK responses and the VS-dependent physics ceiling are plotted directly from the preserved 432-row simulator output.
Executed scenario results
{table(simulator_display.round(3))}Figure 10. Localhost Control Room evidence captured 9 July 2026: process controls, methane/physics/VFA outputs, 48-hour handoff, and CSV/JSON/report experiment-memory export. The historical panel's model-status message is superseded by the fresh Version 2 replay above, which confirms the trained bundle loaded.
Figure 11. Localhost scenario-handoff evidence captured 9 July 2026: the industrial simulator imports OLR, pH, temperature, HRT, TS and VS, detects VFA/ALK stress, raises warning/critical UI states, and records the operator-applied process state.
Figure 12. Localhost industrial-simulator memory evidence captured 9 July 2026: scenario variables, action/effect records, VFA/ALK warning state, physics message and CSV/report export controls remain visible in one audit surface.
Open replayable simulator evidence
{linked_file_table(simulator_rows)} {code_block(simulator_metadata['replay_command_template'], "Click to view the simulator replay command template")}Capability conclusion
This evidence supports the statement that GFIS implements and executes continuous multi-hour methane simulation, VFA/ALK soft sensing, VS-based feasibility checking, scenario coordination and persistent experiment export. It does not convert controlled replay into an industrial-plant accuracy claim; that remains the next external-validation layer.
Implemented GFIS capability versus external validation
These are not missing product ideas. They already exist in the GFIS prototype and simulator. Gate 3's limitation is narrower: DS-03 cannot independently validate every capability because it provides total biogas with almost no measured methane and lacks the required VFA/ALK and VS/BMP targets.
| GFIS capability | Implemented evidence now | Validation boundary | Next evidence upgrade |
|---|---|---|---|
| Continuous methane forecasting | Trained methane predictor, temporal inputs and deterministic 48-hour plant trace with feedback history. | Demonstrated on synthetic/scenario data; DS-03 does not contain a usable continuous methane target. | Replay on a licensed timestamped methane dataset and compare chronologically against persistence and XGBoost. |
| VFA/ALK virtual soft sensor | Random-forest soft sensor, stability thresholds, overload/acidification scenarios, API output and simulator alarms. | Functional prototype evidence; no independent measured VFA/ALK labels in DS-03. | Validate classification and regression against laboratory VFA and alkalinity samples, including lead-time to warning. |
| VS-based physics guidance | VS-dependent methane ceiling, violation flag, post-prediction correction and scenario violation memory. | The current implementation is a feasibility constraint/post-processing layer—not yet a trained physics-loss experiment on measured methane/VS/BMP. | Add an ablation-controlled loss term and report accuracy, violation rate and yield-bound calibration. |
| Temporal memory | LSTM branch, lag/rolling history, sequence-window experiments and complete epoch logs/checkpoints. | On DS-03, longer windows do not improve rolling-origin robustness and persistence wins two of three folds. | Keep the compact LSTM challenger; retest only when a longer, methane-rich series provides adequate memory evidence. |
| Industrial digital-twin workflow | Control Room, 48-hour industrial simulator, scenario handoff, optimization, warnings and JSON/CSV/report memory export. | Product workflow is demonstrable; it is not yet a field-performance claim from a commissioned industrial plant. | Run shadow-mode pilot validation with sensor mapping, calibration records, uptime, error and operator-decision metrics. |
Defensible Version 2 statement
GFIS implements and demonstrates methane prediction, VFA/ALK soft sensing, VS-based feasibility checking, temporal modelling, and a memory-enabled industrial simulator. The validation level is explicitly labelled as synthetic, public-dataset, or future plant validation for each result.
Recommended temporal architecture
Keep persistence as the safe DS-03 champion. Retain the compact 3–7 observation LSTM as challenger. Advance the physics-guided methane architecture when a licensed continuous dataset supplies reactor identity, measured methane, temperature, feed/VS or BMP, and preferably VFA/ALK.
Open prediction and 48-hour simulation service Open LSTM and physics-constraint models Open controlled scenario evidence