Gate 3 Integrated Scientific Report

Executive scientific verdict

GFIS remains a methane-yield prediction and decision-support system. Gate 3 adds a reproducible public-data experiment chain, but DS-03 evaluates total-biogas dynamics rather than measured methane. On that dataset, the current LSTM is not promoted: persistence won two of three rolling-origin folds. The LSTM remains a short-window challenger while methane-specific external validation continues separately.

Evaluator-suggested dataset: experimental impact

Gate 3 selected the catalogue's rank-1 public source—Mendeley DS-03, farm-scale digester process and weather data—in direct response to the evaluation requirement for real/open evidence and temperature sensitivity. The evaluator guidance motivated the source category and experiment; it does not imply that the evaluator authored the dataset.

Experiment-to-learning traceability

Open evaluator action plan Open response report

Input provenance and integrity

The selected source is Mendeley Data V1, DOI 10.17632/gk3f363sfg.1, contributed by Daniel Mulat and licensed CC BY 4.0.

Open input and processing artifacts

Figure 1. Four reactor histories and naturally observed air-temperature trajectories. Reactor identity is retained throughout.

Data quality and target audit

Figure 2. Predictor and target-field missingness. Methane coverage is scientifically unusable.

Acceptance rules

• One-day intervals only for comparable daily forecasting.
• Duplicate reactor timestamps excluded.
• Targets are never imputed.
• Predictor imputation and scaling are fitted within each training fold.
• R1–R4 identity is retained with group-aware sequence creation.
• Mixed Excel dates are repaired monotonically and flagged.

{
  "rows": 1167,
  "reactors": [
    "R1",
    "R2",
    "R3",
    "R4"
  ],
  "timestamp_start": "2024-01-04 00:00:00",
  "timestamp_end": "2025-01-14 00:00:00",
  "target_definition": "Total biogas meter increment between recorded observations, source-labelled mL",
  "target_is_methane": false,
  "by_reactor": {
    "R1": {
      "rows": 306,
      "start": "2024-01-04",
      "end": "2025-01-14",
      "duplicate_timestamps": 0,
      "non_daily_intervals": 55,
      "median_interval_days": 1.0,
      "target_missing": 1,
      "temperature_missing": 0,
      "digester_temperature_missing": 3,
      "methane_fraction_observations": 1,
      "date_repairs": 0
    },
    "R2": {
      "rows": 306,
      "start": "2024-01-04",
      "end": "2025-01-14",
      "duplicate_timestamps": 0,
      "non_daily_intervals": 54,
      "median_interval_days": 1.0,
      "target_missing": 1,
      "temperature_missing": 0,
      "digester_temperature_missing": 7,
      "methane_fraction_observations": 1,
      "date_repairs": 0
    },
    "R3": {
      "rows": 279,
      "start": "2024-01-10",
      "end": "2025-01-14",
      "duplicate_timestamps": 6,
      "non_daily_intervals": 50,
      "median_interval_days": 1.0,
      "target_missing": 1,
      "temperature_missing": 10,
      "digester_temperature_missing": 279,
      "methane_fraction_observations": 1,
      "date_repairs": 23
    },
    "R4": {
      "rows": 276,
      "start": "2024-01-10",
      "end": "2025-01-14",
      "duplicate_timestamps": 5,
      "non_daily_intervals": 48,
      "median_interval_days": 1.0,
      "target_missing": 1,
      "temperature_missing": 8,
      "digester_temperature_missing": 276,
      "methane_fraction_observations": 1,
      "date_repairs": 32
    }
  },
  "missing_fraction": {
    "total_biogas_interval_ml": 0.003427592116538132,
    "air_temperature_c": 0.015424164524421594,
    "digester_temperature_c": 0.48414738646101113,
    "manure_fed_kg": 0.01456726649528706,
    "methane_fraction_percent": 0.9965724078834619
  }
}

Integrated processing and evaluation flow

Equations, derivations, and statistical definitions

These equations define exactly what the Gate 3 code computes. Symbols are indexed by reactor r, observation time t, rolling fold k, and random seed s.

1. Target construction and eligibility

B is the cumulative biogas-meter reading. The source workbook supplies this difference as “Daily Biogas (mL).” Gate 3 interprets it as interval total biogas and accepts it as daily only when Δt = 1 day.

Only observations with 𝓔 = 1 enter model evaluation. Missing targets are never imputed.

2. Past-only temporal features

Sample rolling standard deviation. The current or future target never enters these features.

Cyclic calendar encoding prevents an artificial discontinuity between 31 December and 1 January.

3. Fold-local preprocessing derivation

The imputation median m, mean μ, and standard deviation σ are estimated only from the current training fold. Validation and test observations are transformed but never used to fit preprocessing.

4. Persistence baseline

A temporal model must outperform this operationally meaningful baseline on leakage-safe validation before an improvement claim is accepted.

5. XGBoost objective

Each tree fits residual structure from the current ensemble; Ω regularizes tree complexity. All lag and rolling predictors remain past-only.

6. Current LSTM state derivation

σ is the logistic sigmoid and ⊙ is element-wise multiplication. Gate 3 uses one layer, hidden size 32, no random shuffling, and windows of 3, 7, 14, and 28 observations.

Targets are standardized from training data only. Adam minimizes scaled MSE; early stopping retains the state with lowest validation MSE.

7. Reported error metrics

sMAPE terms with a zero denominator are excluded. Negative R² means the model is worse than predicting the evaluation-period mean.

8. Repeated-seed aggregation and dispersion

S = 3 seeds (17, 29, 43). The report presents mean and sample standard deviation across fold/seed observations.

9. Natural temperature-drift definition

Positive degradation means worse error in naturally outlying temperature conditions. This is an association test, not a causal temperature derivation.

10. Robust architecture-selection rule

Ties are resolved by median fold RMSE and then dispersion. The rule was applied before opening the final test. Persistence won two folds; LSTM won one.

11. Future methane-physics formulation — not evaluated here

Rolling-origin model evidence

A single mean score would conceal the extreme second regime. Model selection therefore uses fold wins, then median fold RMSE and dispersion.

Figure 3. Fold-level RMSE on a logarithmic axis. Persistence is strongest in folds 1 and 3; LSTM is strongest only in fold 2.

Figure 4. Sequence-window changes are negligible relative to fold-to-fold dispersion.

Locked final-period comparison

Figure 5. The LSTM has slightly lower final-test RMSE, but much worse sMAPE and was not the rolling-validation champion.

Figure 6. Seed-averaged final-period predictions. Symlog scaling exposes both normal and extreme intervals.

Training sessions and epoch evidence

Every LSTM epoch is retained with run, fold, seed, window, training loss, and validation loss. Final-fit LSTM and XGBoost checkpoints are checksum-registered.

Figure 7. Window-3 rolling LSTM validation traces across all folds and seeds.

{
  "experiment_id": "G3-E01",
  "dataset_id": "DS-03",
  "processed_schema": "gfis-gate3-ds03-v1",
  "target": "total_biogas_interval_ml",
  "target_semantics": "total biogas meter increment between observations; not methane",
  "test_fraction_by_time": 0.15,
  "rolling_folds": 3,
  "sequence_windows": [
    3,
    7,
    14,
    28
  ],
  "seeds": [
    17,
    29,
    43
  ],
  "lstm": {
    "hidden_size": 32,
    "num_layers": 1,
    "epochs": 30,
    "patience": 6,
    "learning_rate": 0.003,
    "batch_size": 64
  },
  "xgboost": {
    "n_estimators": 250,
    "max_depth": 4,
    "learning_rate": 0.04,
    "subsample": 0.9,
    "colsample_bytree": 0.9
  },
  "temperature_drift": {
    "natural_only": true,
    "low_quantile": 0.1,
    "high_quantile": 0.9
  },
  "physics_violation": {
    "status": "unavailable",
    "reason": "methane fraction and substrate VS required for a methane-yield bound are absent"
  }
}

Open source code and configuration

Natural temperature-drift study

Drift observations fall outside each training fold's 10th–90th percentile temperature range. No artificial perturbation is presented as plant data.

Figure 8. Degradation is inconsistent across folds: −21.2%, +29.3%, and −21.2%. No causal temperature claim is supported.

Persistent experiment memory

The accepted run binds source identity, checksums, Git commit, split version, seeds, windows, environment, configurations, predictions, epoch histories, plots, checkpoints, and replay instructions.

{
  "python": "3.13.5 (v3.13.5:6cb20a219a8, Jun 11 2025, 12:23:45) [Clang 16.0.0 (clang-1600.0.26.6)]",
  "platform": "macOS-26.5.2-arm64-arm-64bit-Mach-O",
  "numpy": "2.2.6",
  "pandas": "2.2.2",
  "scikit_learn": "1.8.0",
  "xgboost": "2.0.1",
  "torch": "2.9.1"
}

Exact replay

The accepted run was replayed. Fold metrics, summaries, final-test metrics, scientific decision, drift results, prediction files, and epoch history matched byte-for-byte.

PYTHONPATH=01_Product_Source/GFIS_Project python3 \
  01_Product_Source/GFIS_Project/scripts/gate3_prepare_dataset.py \
  --workbook "03_Datasets/01_Raw_Public_Datasets/DS03_Mendeley_gk3f363sfg_v1/extracted/Farm-scale_Biodigester_Perfromance_dataset/Farm-scale Biodigester 2024_Performance.xlsx" \
  --weather-workbook "03_Datasets/01_Raw_Public_Datasets/DS03_Mendeley_gk3f363sfg_v1/extracted/Farm-scale_Biodigester_Perfromance_dataset/Mazingira main(z6-02152)-1729245757_weather_data.xlsx" \
  --output 03_Datasets/05_Processed_Final/DS03/gfis_ds03_canonical_v1.csv \
  --manifest 03_Datasets/05_Processed_Final/DS03/processing_manifest_v1.json

PYTHONPATH=01_Product_Source/GFIS_Project python3 \
  01_Product_Source/GFIS_Project/scripts/gate3_run_experiment.py \
  --run-id G3-E01-20260724T1530Z-ds03-v3-replay

GFIS capabilities working end to end

The simulator evidence below is not a conceptual mock-up. The trained GFIS service bundle was loaded and executed for 9 controlled scenarios × 48 hours, producing 432 hour-indexed output rows. UI screenshots show the corresponding Control Room, scenario handoff, alarm state and experiment-memory surfaces.

Figure 9. Fresh Version 2 backend replay. Methane trajectories, VFA/ALK responses and the VS-dependent physics ceiling are plotted directly from the preserved 432-row simulator output.

Executed scenario results

Figure 10. Localhost Control Room evidence captured 9 July 2026: process controls, methane/physics/VFA outputs, 48-hour handoff, and CSV/JSON/report experiment-memory export. The historical panel's model-status message is superseded by the fresh Version 2 replay above, which confirms the trained bundle loaded.

Figure 11. Localhost scenario-handoff evidence captured 9 July 2026: the industrial simulator imports OLR, pH, temperature, HRT, TS and VS, detects VFA/ALK stress, raises warning/critical UI states, and records the operator-applied process state.

Figure 12. Localhost industrial-simulator memory evidence captured 9 July 2026: scenario variables, action/effect records, VFA/ALK warning state, physics message and CSV/report export controls remain visible in one audit surface.

Open replayable simulator evidence

PYTHONPATH=01_Product_Source/GFIS_Project python3 01_Product_Source/GFIS_Project/scripts/generate_v2_simulator_evidence.py --run-id <NEW_UNIQUE_RUN_ID> --hours 48

Implemented GFIS capability versus external validation

These are not missing product ideas. They already exist in the GFIS prototype and simulator. Gate 3's limitation is narrower: DS-03 cannot independently validate every capability because it provides total biogas with almost no measured methane and lacks the required VFA/ALK and VS/BMP targets.

Open prediction and 48-hour simulation service Open LSTM and physics-constraint models Open controlled scenario evidence