# Small-Model AI Qualification Summary

## Scope

- System under test: `HuggingFaceTB/SmolLM2-135M-Instruct`
- Revision: `12fd25f77366fa6b3b4b768ec3050bf629380bac`
- License: Apache-2.0
- Runtime: CPU, Python 3.10, PyTorch 2.10, Transformers 5.3
- Execution: September 24, 2026

The qualification exercises data contracts and lineage, authorization-aware
retrieval, grounded generation, abstention, citation validity, repeated sampled
generation, agent output contracts, tool allowlisting, and fail-closed policy.

## Results

| Layer | Result | Evidence |
| --- | --- | --- |
| Deterministic harness | Pass | 13/13 pytest checks |
| Data pipeline | Pass | 3/3 authorized sources reconciled; no contract errors |
| Retrieval | Pass | 4/4 expected retrieval outcomes |
| Deterministic RAG end to end | Fail | 0/4 complete answer, citation, and abstention contracts |
| Sampled RAG end to end | Fail | 5/10 trials; 10 unique outputs |
| Agent policy | Fail closed | 0/4 valid actions; 0 side effects executed |
| Release gate | Held | Model behavior remained below defined thresholds |

## Disposition

The source, index, authorization filter, and retrieval threshold behaved as
expected. Failures localized to generation: the model omitted or copied invalid
citations, did not reliably follow abstention instructions, and emitted prose or
code instead of the required agent JSON contract. Invalid actions were rejected
before execution.

The next controlled experiment should preserve the corpus and fixtures while
comparing grammar-constrained decoding and a slightly larger instruction model.
The current model should not be released for these RAG or agent contracts.