Nvidia Nemotron-3-Super-120B-FP8 Stained Glass Enabled

Protopia AI's Stained Glass Output Protection encrypts tokens as they are generated by an LLM. The generated response is never fully available in plaintext on the compute infrastructure. Raw data never leaves its trust zone-only these protected re-representations of the data reach the compute infrastructure.

## How It Works

Stained Glass Output Protection encrypts each token as it is generated by the LLM. Every request establishes a secure key exchange to individually encrypt each generated token. The plaintext response never exists in its entirety on the compute infrastructure. This prevents leakage via logs, dumps, state, etc. Only the authorized client holding its private key can decrypt the response.

Not included in this marketplace product, the companion Stained Glass Transform (SGT) transforms your input data into unintelligible prompt embeddings (prompt_embeds) before being sent to the compute infrastructure. The plaintext never leaves the data owner's trust zone. This [SGT-Enabled Nvidia Nemotron-3-Super-120B-FP8] supports SGT-transformed inputs, processing protected representations directly with no decoder on the endpoint.

This roundtrip protection pipeline means that neither the input nor the output is ever exposed on the inference host - a critical requirement for regulated industries handling patient records, financial

documents, or privileged legal communications.

## Use Case: Confidential Document Summarization

A financial services firm running retrieval-augmented generation (RAG) over proprietary research documents on shared cloud GPUs can deploy this listing to ensure that neither the retrieved context

(input) nor the generated summaries (output) are visible to the infrastructure operator or co-tenants.

The same pattern applies to healthcare organizations summarizing patient records or legal teams

analyzing privileged communications.

## About the Model

The underlying NVIDIA Nemotron-3-Super-120B-A12B model was pre-trained on over 25 trillion tokens

spanning code, math, science, and general knowledge across 20 languages, then instruction-tuned for

tool calling, structured outputs, and long-context retrieval. NVIDIA reports benchmark results

including 86.01 on MMLU and 90.67 on GSM8K (8-shot). This listing serves an FP8-quantized variant for

reduced memory footprint and inference latency, which may show minor accuracy variation from published

BF16 figures.

## Deployment and Scaling

Served on vLLM with tensor parallelism that auto-sizes to the deployed instance's GPU count.

Verified serving prompt_embeds correctly across all supported real-time instance sizes.

Note: despite the delivery configuration listing a recommended batch transform instance, batch (asynchronous) inference is not currently supported -- no AWS SageMaker batch transform instance type is capable of serving a model this large. Only real-time inference is supported today.

## Why Stained Glass Transform Over Alternatives

Unlike homomorphic encryption, SGT operates with low compute overhead on commodity hardware - no specialized TEE hardware required. Unlike differential privacy approaches that inject noise, SGT

preserves full data fidelity so model accuracy is maintained. This frees powerful GPUs for inference rather than encryption overhead, and integrates into existing AI pipelines with minimal latency impact.

## Getting Started

To evaluate this solution or schedule a technical demo with our solutions engineering team, contact us

at contact@protopia.ai. We support deployments across on-premises, multi-tenant cloud, and edge

environments.

## Input/Output Format

Inputs can be passed as prompt_embeds (transformed embeddings). Outputs are returned as tokenwise-encrypted

payloads using a secure key exchange. See the sample Jupyter notebook

(https://github.com/protopia-ai/sagemaker-marketplace/blob/main/invoke_sgt_nemotron_endpoint.ipynb)

for a full walkthrough of payload structure, decryption steps, and SDK usage.