Anbinh93/MDGT-matching

MDGT Matching - FastVIT Cross-Sensor Checkpoint

This repository currently points to the FastVIT cross-sensor branch, not the older MDGT/NIST300A checkpoint.

Primary checkpoint:

checkpoints_fastvit_cross_sensor_sa12_round5_consistency_light/best_rank1.pt

Model

• Architecture: FastVITGraph
• Backbone: fastvit_sa12.apple_in1k
• Feature index: 1
• Token selection: TRAM
• Token count: 64
• GNN: 2 layers, dim 256, 4 heads, k=8
• Embedding dim: 256
• Input: preprocessed grayscale tensor (B, 1, 224, 224)
• Preprocess used for export/eval: squeeze resize, CLAHE on, Gabor off

Metrics

These metrics are from the checkpoint metadata for the full NIST302 cross-sensor 1:N protocol:

• Probe: NIST302A challengers, 13,630 probes
• Reference: NIST302B baseline, 8,000 gallery images
• Identities: 2,000

Metric	Value
Rank-1	0.8133528829
Rank-5	0.9174614549
Rank-10	0.9479089975
mAP	0.7901349983
AUC	0.9983431867
EER	0.0178200894

The earlier low Rank-1: 0.390625 result belonged to an MDGT/NIST300A QAI smoke benchmark on a 64-identity spread subset. It is not the metric for this FastVIT checkpoint.

Files

pytorch/fastvit_cross_sensor_sa12_round5_consistency_light_best_rank1.pt
onnx/fastvit_cross_sensor_embedding_fp32.onnx
onnx/fastvit_cross_sensor_embedding_fp16.onnx
onnx/fastvit_cross_sensor_embedding_int8_linear_qdq.onnx
onnx/export_summary.json
qai/fastvit_cross_sensor_qai_target_model.dlc
qai/qai_qnn_context_binary_summary.json
qai/qai_qnn_dlc_summary.json
qai/qai_qnn_lib_attempt_summary.json
results/checkpoint_metrics.json
training/history.jsonl
artifact_metadata.json

ONNX Export

The ONNX export uses an export-safe FastVITGraph wrapper with vectorized deterministic token deduplication.

Observed local drift on 16 samples:

Comparison	Cos mean	Cos min
Original PyTorch vs exportable PyTorch	1.0000000000	0.9999999404
Exportable PyTorch vs ONNX FP32	0.9999986887	0.9999958277
ONNX FP16 vs ONNX FP32	0.9990564585	0.9972373843
ONNX INT8 linear QDQ vs ONNX FP32	0.9941318631	0.9882222414

Qualcomm AI Hub

QAI test was rerun on the FastVIT ONNX, not on MDGT.

• qnn_context_binary: compile failed with exit code 14.
• qnn_lib_aarch64_android: unsupported by the installed QAI Hub client.
• qnn_dlc: compile succeeded, target DLC was produced.
• NPU profile/inference for the DLC failed with MODEL_GRAPH_ERROR from QnnModel_composeGraphsFromDlc.

Jobs:

qnn_context_binary compile: https://workbench.aihub.qualcomm.com/jobs/jgjwl2v75/
qnn_dlc compile:           https://workbench.aihub.qualcomm.com/jobs/jgdzvqxk5/
qnn_dlc profile:           https://workbench.aihub.qualcomm.com/jobs/jp847m2z5/
qnn_dlc inference:         https://workbench.aihub.qualcomm.com/jobs/jgz47j2zp/

This means the current full FastVITGraph ONNX can be exported and compiled to DLC, but the graph is not yet deployable as a working NPU profile/inference artifact on QAI Hub. The likely next step is to simplify or split the dynamic token/GNN section for NPU compatibility.