Themis.Inference.QNN 4.0.0-preview

Themis.Inference.QNN

Themis.Inference.QNN adds an optional Qualcomm QNN NPU selection policy for Themis ONNX inference. It reuses the Windows ML execution-provider catalog and the existing ONNX model and typed-answer contracts; it does not include QNN native binaries or model weights. The package is distributed through the NuGet prerelease channel with a -preview version suffix.

Register the package with a Windows-targeted application:

services.AddThemis(options => options.Model = "laya-production");
services.AddThemisQNN();

The default configuration requires QNNExecutionProvider on an NPU and fails instead of falling back to CPU or selecting the QNN GPU device. Pass an InferenceRuntimeConfiguration with a different required provider only when that behavior is intentional. When a custom configuration omits RequiredExecutionProviderName, the extension preserves its other settings and still requires QNN.

The application must target net10.0-windows10.0.18362.0 or later and run on a compatible Windows 11 24H2+ device. Microsoft's current Windows ML requirements list Snapdragon X Elite and X Plus devices with a Qualcomm Hexagon NPU driver version 30.0.140.0 or later. Windows ML dynamically installs and registers certified providers; actual availability depends on the OS, device, driver, and provider catalog. Verify the current requirements before deployment.

The package contains Themis registration and device-selection code only. QNN provider licensing and model/operator compatibility remain separate requirements. This repository has not validated QNN inference or NPU profile placement on Qualcomm hardware; registration and package restore are not a support claim. Review the current Windows ML third-party notices and QNN license terms before deployment.

Initial hardware-validation target

The first validation target is Windows 11 version 24H2 or later (minimum build 26100), using net10.0-windows10.0.18362.0 and a native win-arm64 consumer. The Themis package uses Windows ML Microsoft.Windows.AI.MachineLearning 2.4.89 and ONNX Runtime 1.31.0. The expected device is a Snapdragon X Elite (X1E) or X Plus (X1P) with a Qualcomm Hexagon NPU and driver 30.0.140.0 or later. Windows ML downloads and registers the certified provider; these are test assumptions, not a device-compatibility guarantee.

The representative model is the pinned Mattepiu/laya-onnx test bundle at revision a7f385bd51d35f5cbeb88e8bc4f0c7e7c0f24c44. The request contains choice, score, and noul questions in one decision request; sequence length and question batching follow the bundle manifest. Neither this model/provider combination nor its NPU profile placement has been validated by Themis.

References:

• Windows ML supported execution providers
• Register Windows ML execution providers
• ONNX Runtime QNN execution provider