Update general model

README.md

@@ -12,24 +12,25 @@ tags:
 - de-novo-peptide-sequencing
 ---
 
-## Winnow General Probability Calibrator
+# Winnow General Probability Calibrator
 
 [**Winnow**](https://github.com/instadeepai/winnow) recalibrates confidence scores and provides FDR control for *de novo* peptide sequencing (DNS) workflows.  
-This repository hosts a pretrained, general-purpose calibrator that maps raw [InstaNovo](https://github.com/instadeepai/instanovo) model confidences and complementary features (mass error, retention time, chimericity, beam features, Prosit features) to well-calibrated probabilities.
+This repository hosts a pretrained, general-purpose calibrator that maps raw [InstaNovo](https://github.com/instadeepai/instanovo) model confidences and complementary features (mass error, retention time, beam features, fragment matching features) to well-calibrated probabilities.
 
 - Intended inputs: spectrum input data and corresponding MS/MS PSM results produced by InstaNovo
-- Outputs: calibrated per-PSM probabilities in `calibrated_confidence`.
+- Outputs: rescored and calibrated per-PSM probabilities in `calibrated_confidence` with *de novo* FDR control.
 
-### What’s inside
-- `calibrator.pkl`: trained classifier
-- `scaler.pkl`: feature standardiser
-- `irt_predictor.pkl`: Prosit iRT regressor used by RT features
+## What’s inside
+
+- `model.safetensors`: trained classifier
+- `config.json`: classifier hyperparameter settings
 
 ---
 
 ## How to use
 
 ### Python
+
 ```python
 from pathlib import Path
 from huggingface_hub import snapshot_download
@@ -48,7 +49,7 @@ snapshot_download(
 )
 
 # 2) Load calibrator
-calibrator = ProbabilityCalibrator.load(general_model)
+calibrator = ProbabilityCalibrator.load(pretrained_model_name_or_path=general_model)
 
 # 3) Load your dataset (InstaNovo-style config)
 dataset = InstaNovoDatasetLoader().load(
@@ -68,45 +69,55 @@ dataset.metadata["keep@5%"] = dataset.metadata["calibrated_confidence"] >= cutof
 ```
 
 ### CLI
+
 ```bash
 # After `pip install winnow`
 winnow predict \
-  --data-source instanovo \
-  --dataset-config-path config_with_dataset_paths.yaml \
-  --model-folder general_model_folder \
-  --method winnow \
-  --fdr-threshold 0.05 \
-  --confidence-column calibrated_confidence \
-  --output-path outputs/winnow_predictions.csv
+  data_loader=instanovo \
+  dataset.spectrum_path_or_directory=my_data.parquet \
+  dataset.predictions_path=my_preds.csv \
+  calibrator.pretrained_model_name_or_path=config_with_dataset_paths.yaml \
+  fdr_control.fdr_threshold=0.05 \
+  output_folder=outputs
 ```
 
 ---
 
 ## Inputs and outputs
+
 **Required columns for calibration:**
-- Spectrum data (*.parquet)
+
+- Spectrum data (parquet\ipc\mgf)
   - `spectrum_id` (string): unique spectrum identifier
-  - `sequence` (string): ground truth peptide sequence from database search (optional)
+  - `experiment_name` (string): MS run identifier
   - `retention_time` (float): retention time (seconds)
-  - `precursor_mass` (float): mass of the precursor ion (from MS1)
+  - `precursor_charge` (float): charge of the precursor ion (from MS1)
+  - `precursor_mz` (float): mass-to-charge of the precursor ion (from MS1)
   - `mz_array` (list[float]): mass-to-charge values of the MS2 spectrum
   - `intensity_array` (list[float]): intensity values of the MS2 spectrum
-  - `precursor_charge` (int): charge of the precursor (from MS1)
 
-- Beam predictions (*_beams.csv)
+- Beam predictions (csv)
   - `spectrum_id` (string)
-  - `sequence` (string): ground truth peptide sequence from database search (optional)
-  - `preds` (string): top prediction, untokenised sequence
-  - `preds_tokenised` (string): comma‐separated tokens for the top prediction
-  - `log_probs` (float): top prediction log probability
-  - `preds_beam_k` (string): untokenised sequence for beam k (k≥0)
-  - `log_probs_beam_k` (float)
-  - `token_log_probs_k` (string/list-encoded): per-token log probabilities for beam k
-
-**Output columns (added by Winnow's calibrator on `predict`):**
+  - `predictions` (string): top prediction, untokenised sequence
+  - `predictions_tokenised` (string): comma‐separated tokens for the top prediction
+  - `log_probability` (float): top prediction log probability
+  - `token_log_probabilities` (list[float]): per-token log-probabilities for the top prediction
+  - `predictions_beam_k` (string): untokenised sequence for beam k (k≥0)
+  - `log_probability_beam_k` (float)
+  - `token_log_probabilities_k` (string/list-encoded)
+
+**Outputs:**
+
+- `metadata.csv`: spectrum metadata and computed features. Contains everything *except* the prediction and FDR columns, i.e.:
+  - `spectrum_id`, `experiment_name`, `precursor_mz`, `precursor_charge`, `retention_time`, etc. (all pass-through spectrum columns)
+  All computed feature columns, including intermediate results (`mass_error_da`, `irt_error`, `ion_matches`, `margin`, etc.)
+- `preds_and_fdr_metrics.csv`: predictions and FDR results. Always contains:
+  - `spectrum_id`
+  - `prediction`
   - `calibrated_confidence`: calibrated probability
-  - Optional (if requested): `psm_pep`, `psm_fdr`, `psm_qvalue`
-  - All input columns are retained in-place
+  - `psm_fdr`
+  - `psm_q_value`
+  - Optional: `psm_pep`
 
 ---
 
@@ -120,8 +131,9 @@ winnow predict \
   - Immunopeptidomics (PXD006939)
   - HeLa degradome (PXD044934)
   - Snake venoms (PXD036161)
-- All default features were enabled for the training of this model.
-- Predictions were obtained using InstaNovo v1.1.1 with knapsack beam search set to 50 beams.
+  - Therapeutic nanobodies (PXD044934)
+- This model uses fragment match features, iRT features, beam features, token score features and the mass error feature.
+- Predictions were obtained using InstaNovo v1.2.0 with beam search set to 5 beams.
 
 ---
 
@@ -180,4 +192,5 @@ If you use the `InstaNovo` model to generate predictions, please also cite: [Ins
 ```
 
 ## Contact
-For issues with this pretrained model or usage in Winnow, please open an issue on the Winnow GitHub: https://github.com/instadeepai/winnow
+
+For issues with this pretrained model or usage in Winnow, please open an issue on the Winnow GitHub: https://github.com/instadeepai/winnow

config.json

@@ -0,0 +1,236 @@
+{
+  "input_dim": 11,
+  "hidden_dims": [
+    50,
+    50
+  ],
+  "dropout": 0.3,
+  "learning_rate": 0.0001,
+  "weight_decay": 0.001,
+  "max_epochs": 1000,
+  "batch_size": 1024,
+  "n_iter_no_change": 10,
+  "tol": 0.0001,
+  "seed": 42,
+  "val_early_stopping_max_psms": null,
+  "val_subsample_seed": null,
+  "feature_columns": [
+    "ion_matches",
+    "ion_match_intensity",
+    "irt_error",
+    "margin",
+    "median_margin",
+    "entropy",
+    "z-score",
+    "min_token_probability",
+    "std_token_probability",
+    "mass_error_da"
+  ],
+  "feature_names": [
+    "Fragment Match Features",
+    "iRT Feature",
+    "Beam Features",
+    "Token Score Features",
+    "Mass Error (Da)"
+  ],
+  "features": {
+    "Fragment Match Features": {
+      "_target_": "winnow.calibration.features.fragment_match.FragmentMatchFeatures",
+      "mz_tolerance_ppm": 20,
+      "mz_tolerance_da": null,
+      "learn_from_missing": false,
+      "intensity_model_name": "Prosit_2025_intensity_22PTM",
+      "max_precursor_charge": 6,
+      "max_peptide_length": 30,
+      "unsupported_residues": [
+        "[UNIMOD:5]",
+        "[UNIMOD:385]",
+        "(+25.98)"
+      ],
+      "model_input_constants": null,
+      "model_input_columns": {
+        "collision_energies": "collision_energy",
+        "fragmentation_types": "frag_type"
+      },
+      "koina_server_url": "localhost:8500",
+      "koina_ssl": false,
+      "excluded_columns": [
+        "complementary_ion_count",
+        "max_ion_gap",
+        "spectral_angle",
+        "xcorr"
+      ]
+    },
+    "iRT Feature": {
+      "_target_": "winnow.calibration.features.retention_time.RetentionTimeFeature",
+      "train_fraction": 0.1,
+      "min_train_points": 5,
+      "learn_from_missing": false,
+      "seed": 42,
+      "irt_model_name": "Prosit_2025_irt_22PTM",
+      "max_peptide_length": 30,
+      "unsupported_residues": [
+        "[UNIMOD:5]",
+        "[UNIMOD:385]",
+        "(+25.98)"
+      ],
+      "koina_server_url": "localhost:8500",
+      "koina_ssl": false
+    },
+    "Beam Features": {
+      "_target_": "winnow.calibration.features.beam.BeamFeatures",
+      "excluded_columns": [
+        "edit_distance"
+      ]
+    },
+    "Token Score Features": {
+      "_target_": "winnow.calibration.features.token_score.TokenScoreFeatures"
+    },
+    "Mass Error (Da)": {
+      "_target_": "winnow.calibration.features.mass_error.MassErrorDaFeature",
+      "residue_masses": {
+        "G": 57.021464,
+        "A": 71.037114,
+        "S": 87.032028,
+        "P": 97.052764,
+        "V": 99.068414,
+        "T": 101.04767,
+        "C": 103.009185,
+        "L": 113.084064,
+        "I": 113.084064,
+        "N": 114.042927,
+        "D": 115.026943,
+        "Q": 128.058578,
+        "K": 128.094963,
+        "E": 129.042593,
+        "M": 131.040485,
+        "H": 137.058912,
+        "F": 147.068414,
+        "R": 156.101111,
+        "Y": 163.063329,
+        "W": 186.079313,
+        "U": 150.953633,
+        "O": 237.147727,
+        "K[UNIMOD:1]": 170.105528,
+        "K[UNIMOD:3]": 354.172561,
+        "C[UNIMOD:4]": 160.030649,
+        "Q[UNIMOD:7]": 129.042594,
+        "R[UNIMOD:7]": 157.085127,
+        "N[UNIMOD:7]": 115.026943,
+        "S[UNIMOD:21]": 166.998359,
+        "T[UNIMOD:21]": 181.014009,
+        "Y[UNIMOD:21]": 243.02966,
+        "E[UNIMOD:27]": 111.032028,
+        "Q[UNIMOD:28]": 111.032029,
+        "K[UNIMOD:34]": 142.110613,
+        "R[UNIMOD:34]": 170.116761,
+        "M[UNIMOD:35]": 147.0354,
+        "P[UNIMOD:35]": 113.047679,
+        "K[UNIMOD:36]": 156.126263,
+        "R[UNIMOD:36]": 184.132411,
+        "K[UNIMOD:37]": 170.141913,
+        "N[UNIMOD:43]": 317.1223,
+        "K[UNIMOD:58]": 184.121178,
+        "K[UNIMOD:64]": 228.111007,
+        "K[UNIMOD:121]": 242.13789,
+        "K[UNIMOD:122]": 156.089878,
+        "N[UNIMOD:137]": 1330.46579,
+        "N[UNIMOD:148]": 682.254496,
+        "N[UNIMOD:159]": 1006.360143,
+        "R[UNIMOD:188]": 162.12124,
+        "K[UNIMOD:214]": 272.197026,
+        "K[UNIMOD:259]": 136.109162,
+        "R[UNIMOD:267]": 166.10938,
+        "Y[UNIMOD:275]": 192.053493,
+        "N[UNIMOD:305]": 1558.576797,
+        "N[UNIMOD:307]": 1720.62962,
+        "N[UNIMOD:308]": 1882.682444,
+        "N[UNIMOD:309]": 1412.518888,
+        "N[UNIMOD:310]": 1574.571711,
+        "N[UNIMOD:311]": 1736.624535,
+        "C[UNIMOD:312]": 222.013284,
+        "K[UNIMOD:481]": 132.12007,
+        "K[UNIMOD:737]": 357.257895,
+        "S[UNIMOD:737]": 316.19496,
+        "K[UNIMOD:747]": 214.095357,
+        "K[UNIMOD:1289]": 198.136828,
+        "K[UNIMOD:1363]": 196.121178,
+        "N[UNIMOD:1408]": 2318.815368,
+        "N[UNIMOD:1409]": 2027.719952,
+        "N[UNIMOD:1410]": 2173.77786,
+        "N[UNIMOD:1452]": 1168.412966,
+        "N[UNIMOD:1462]": 1476.523699,
+        "N[UNIMOD:1465]": 1492.518613,
+        "N[UNIMOD:1467]": 1517.550248,
+        "N[UNIMOD:1468]": 1533.545162,
+        "N[UNIMOD:1477]": 1638.576522,
+        "N[UNIMOD:1480]": 1654.571437,
+        "N[UNIMOD:1481]": 1663.608157,
+        "N[UNIMOD:1484]": 1679.603071,
+        "N[UNIMOD:1487]": 1695.597986,
+        "N[UNIMOD:1488]": 1703.614305,
+        "N[UNIMOD:1496]": 1777.651084,
+        "N[UNIMOD:1500]": 1800.629346,
+        "N[UNIMOD:1504]": 1816.62426,
+        "N[UNIMOD:1506]": 1824.640579,
+        "N[UNIMOD:1509]": 1841.655895,
+        "N[UNIMOD:1510]": 1849.672213,
+        "N[UNIMOD:1511]": 1850.692615,
+        "N[UNIMOD:1519]": 1923.708993,
+        "N[UNIMOD:1529]": 1970.698488,
+        "N[UNIMOD:1531]": 1978.677084,
+        "N[UNIMOD:1532]": 1980.730457,
+        "N[UNIMOD:1534]": 1986.693402,
+        "N[UNIMOD:1537]": 2003.708718,
+        "N[UNIMOD:1540]": 2021.757006,
+        "N[UNIMOD:1541]": 2028.740353,
+        "N[UNIMOD:1547]": 2044.735267,
+        "N[UNIMOD:1549]": 2060.730182,
+        "N[UNIMOD:1551]": 2068.746501,
+        "N[UNIMOD:1555]": 2085.761816,
+        "N[UNIMOD:1559]": 2101.756731,
+        "N[UNIMOD:1562]": 2110.793451,
+        "N[UNIMOD:1722]": 1647.613242,
+        "N[UNIMOD:1746]": 1907.714078,
+        "N[UNIMOD:1761]": 1152.418052,
+        "N[UNIMOD:1763]": 1209.439515,
+        "N[UNIMOD:1766]": 1314.470875,
+        "N[UNIMOD:1768]": 1355.497424,
+        "N[UNIMOD:1769]": 1371.492339,
+        "N[UNIMOD:1771]": 1501.555333,
+        "N[UNIMOD:1772]": 1615.598261,
+        "N[UNIMOD:1773]": 1662.587755,
+        "N[UNIMOD:1775]": 1761.656169,
+        "N[UNIMOD:1776]": 1818.677633,
+        "N[UNIMOD:1777]": 1865.667128,
+        "N[UNIMOD:1778]": 1866.687529,
+        "N[UNIMOD:1779]": 1898.677358,
+        "N[UNIMOD:1780]": 1939.703908,
+        "N[UNIMOD:1781]": 1964.735542,
+        "N[UNIMOD:1782]": 2011.725037,
+        "N[UNIMOD:1783]": 2012.745438,
+        "N[UNIMOD:1784]": 2052.751586,
+        "N[UNIMOD:1785]": 2069.766902,
+        "N[UNIMOD:1840]": 2206.788091,
+        "K[UNIMOD:1848]": 242.126657,
+        "K[UNIMOD:1849]": 214.131742,
+        "N[UNIMOD:1963]": 1962.682169,
+        "N[UNIMOD:1964]": 1994.709721,
+        "K[UNIMOD:2016]": 432.302109,
+        "N[UNIMOD:2028]": 2975.042981,
+        "N[UNIMOD:2029]": 2466.888927,
+        "[UNIMOD:1]": 42.010565,
+        "[UNIMOD:5]": 43.005814,
+        "[UNIMOD:214]": 144.102063,
+        "[UNIMOD:385]": -17.026549,
+        "[UNIMOD:737]": 229.162932,
+        "[UNIMOD:2016]": 304.207146,
+        "(+25.98)": 25.980265
+      },
+      "isotope_error_range": [
+        0,
+        1
+      ]
+    }
+  }
+}

irt_predictor.pkl

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:6e60e92eb992edeec35a4ad575c692d52b9a62b0eb36e9159d72cf6bc95f1c91
-size 11548

calibrator.pkl → model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:a6bef25b6d56f887c29fa851670fe42a89c0a388838bec74ec42a2bbf53e0e52
-size 85140
+oid sha256:07dc59faaeed64a030044166b216948b54173e57338a0f2a939963f2e0bd4f3a
+size 13540

scaler.pkl

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:75bd5c573b75736ab293025a8a546c2a72d218d3c855655a77132a2d452dd704
-size 714