Kornia: Differentiable geometric vision and batched image operators for PyTorch
A differentiable geometry library for PyTorch vision teams, keeping batched augmentation, camera models, and feature matching on the GPU.
GitHub kornia/kornia Updated 2026-11-02 Branch main Stars 11.8K Forks 1.5K
Python PyTorch Geometric computer vision torch.compile CUDA/MPS ONNX

🧭 Decision Guide

Try it if you

  • 你在PyTorch中处理批量图像,并需要每个样本独立的sigma、角度或变换矩阵。
    README“Per-sample parameters in one batched call”说明gaussian_blur2d接收(B, 2) sigma,rotate接收(B,)角度。
  • 你需要可微的depth_to_3d_v2、unproject_points或project_points。
    README“Differentiable geometry”明确说明深度、位姿和相机参数保留梯度。
  • 你要在GPU上组合LoFTR、LightGlue、DISK或XFeat进行局部特征处理。
    README“Local features on the GPU”列出SIFT、HardNet、DISK、ALIKED、XFeat、DeDoDe、LoFTR和LightGlue。
  • 你需要把Kornia算子编译或导出到ONNX。
    README“Compiles and exports”称近乎全部探测配置支持torch.compile,超过1,000个算子可导出ONNX。

Skip it if you

  • 你的任务只是单张uint8图像上的CPU模糊、缩放或简单变换。
    README明确说单个固定参数操作可用PyTorch或torchvision,并指出OpenCV胜过Kornia的单图uint8 CPU工作。
  • 你的算法训练依赖RANSAC、DeDoDe或XFeat提取过程的梯度。
    README说明RANSAC及DeDoDe、XFeat等学习特征提取方法无梯度。
  • 你要提交没有维护者赞助的新VLM或VLA模型集成。
    README“AI Models”说明模型库目前冻结扩展,新集成需要明确的维护者赞助。

Requirements

  • 项目构建在PyTorch之上,安装命令为pip install kornia。
  • CUDA调用或PyTorch ≥ 2.13上的MPS调用可避免相关参数检查的主机同步。
  • RANSAC使用compile=True需要PyTorch ≥ 2.14,支持CPU和CUDA。
  • ONNX和Stable Diffusion dissolving需要额外依赖,详见Optional extras。
  • 0.9.0旗舰基准使用RTX 4090和batch 32。

First step (verbatim from README)

pip install kornia

Watch out

  • 不要把torchvision的单次参数语义直接套到Kornia批量参数上。
    README指出Kornia的rotate使用(B,)角度,而torchvision functional ops每次调用只应用一个参数。
  • RANSAC默认会把精修和最终选择放到主机float64。
    README“Batched minimal solvers and robust estimation”。
  • 3D增强不提供annotations或inverse能力。
    README“Annotated augmentation”明确区分2D增强与3D增强。
  • StableDiffusionDissolving和ONNX功能可能需要额外安装包。
    README“Installation”说明ONNX和Stable Diffusion dissolving需要extra packages。
  • 0.9.0基准中的速度数字依赖RTX 4090、batch 32及具体工作负载。
    README“Fast where it is designed to be fast”说明结果随硬件和工作负载变化。

Alternatives

  • torchvision:只需要resize、crop、flip、color jitter或blur等固定参数操作时更直接。
    README“When to use Kornia”
  • OpenCV:单张uint8图像的CPU处理场景中,README报告OpenCV更快。
    README“When to use Kornia”
  • scikit-image:需要与README中0.9.0基准相同的单图CPU图像处理比较对象时可选。
    README“When to use Kornia”

Not stated in the README

  • 材料没有给出Kornia支持的最低PyTorch版本。
  • 材料没有给出不同GPU、CPU型号和batch size下的完整性能数据。
  • 材料没有说明1,200+ public operators的完整版本范围与逐算子兼容矩阵。
  • 材料没有提供生产环境的API稳定性或向后兼容承诺。
  • 材料没有说明Apache License 2.0下各内置模型权重的具体许可细节。
  • 材料没有给出PyPI安装包的体积、运行时依赖版本和显存占用。

💡 Deep Analysis

6
No I want to integrate a new VLM/VLA model into Kornia and can take long-term maintenance responsibility. Is submitting an implementation and tests enough to add it to the current model zoo?
For: An open-source contributor who wants to add a VLM/VLA model to the existing Kornia model zoo and maintain the integration long term

No, not by code submission alone: the README explicitly says that model-zoo expansion is currently frozen and new integrations require a named maintenance sponsor.

  • The AI Models section states that the model zoo is “currently frozen for expansion” while maintainer capacity is focused on the geometry core.
  • Existing models such as LoFTR, LightGlue, DISK, DeDoDe, and SAM remain available and maintained, but that does not imply that new model categories are open for addition.
  • The README specifically requires every new integration, including VLM/VLA models, to have a named maintainer sponsor who accepts ongoing ownership; “a contributor implementation alone cannot reopen the surface.” Your implementation and tests are therefore not sufficient by themselves.
  • README / AI Models: “The model zoo is currently frozen for expansion”
  • README / AI Models: “New integrations — including VLM/VLA models — require a named maintainer sponsor”
  • README / AI Models: “a contributor implementation alone cannot reopen the surface”
Not stated in the README:The README excerpt does not explain how to obtain a named maintainer sponsor, who approves it, or which model-interface and licensing requirements apply.;It does not specify a date, exact condition, or roadmap version for lifting the freeze.
No My service processes one uint8 image at a time on CPU, mainly performing resize, flip, and blur, with no autograd requirement. Should I migrate these operations to Kornia?
For: An engineer maintaining a single-image uint8 CPU service that mainly uses OpenCV and torchvision without a GPU training-graph requirement

No, not as a direct migration: your workload does not use Kornia’s strongest advantages—batched tensors, GPU execution, and autograd—and may instead add PyTorch tensor-conversion overhead.

  • The project insights explicitly state that Kornia’s main advantages appear in batched Tensor, GPU, and PyTorch workflows; for single images, simple fixed-parameter operations, or CPU uint8 processing, OpenCV, Pillow, scikit-image, or torchvision may be simpler or faster.
  • The README positions Kornia as a differentiable vision library built on PyTorch, indicating that its design target is not the shortest path for conventional single-image CPU APIs.
  • Reconsidering it would make more sense if the service later needs augmentation inside a training graph, batched GPU inference, or geometric vision operations.
  • Project insights / usage_limitations: “for single images, simple fixed-parameter operations, or CPU uint8 processing, OpenCV, Pillow, scikit-image, or torchvision may be simpler or faster”
  • README / Quick Start: “Built on top of PyTorch”
  • Project insights / market_gap: OpenCV and scikit-image are mature for single-image CPU processing, while Kornia focuses on PyTorch autograd and batched GPU workflows
Not stated in the README:The README excerpt does not provide latency, memory, or conversion-cost measurements for your specific OpenCV, torchvision, and Kornia versions.;It does not state whether the service already includes a PyTorch runtime or will later require batched GPU execution or training-graph integration.
Yes I am training a camera-pose model in PyTorch and want homographies, camera projection, and image augmentation inside the training graph with autograd support. Is Kornia suitable instead of NumPy/OpenCV preprocessing?
For: A computer-vision researcher training a PyTorch camera-pose model who needs homography and projection operations inside backpropagation

Yes, because Kornia is built around differentiable PyTorch tensor operators rather than CPU-only preprocessing outside the training graph.

  • The README describes it as a “differentiable computer vision library” and explicitly highlights batch transformations, auto-differentiation, and GPU acceleration.
  • Its stated geometry core includes warping, homographies, cameras, epipolar geometry, rotations, Lie groups, and geometry-consistent augmentation, which align well with camera-pose training.
  • However, the whole library should not be assumed differentiable: the project insights state that RANSAC and some learned feature-extraction paths do not guarantee gradients, so the exact operators must be checked individually.
  • README / Quick Start: “Kornia is a differentiable computer vision library”
  • README / Quick Start: “batch transformations”, “auto-differentiation”, and “GPU acceleration”
  • README / Strengthen the Core: “warping and sampling, homographies, cameras, epipolar geometry, rotations and Lie groups”
pip install kornia
Not stated in the README:The provided README excerpt does not enumerate gradient coverage, degenerate-input behavior, or numerical error for every camera-projection and homography operator.;The target model's exact tensor layout, coordinate convention, and training throughput are not specified.
Yes I am training detection and keypoint models in PyTorch and need one augmentation pipeline to transform images, masks, bounding boxes, and keypoints together, including inverse 2D transforms. Can Kornia reduce annotation misalignment?
For: An algorithm engineer training detection and keypoint models in PyTorch who needs synchronized augmentation of images, masks, boxes, and keypoints

Yes, because Kornia explicitly provides augmentation pipelines intended to transform images and geometric annotations together, matching the alignment problem in detection and keypoint training.

  • The project insights describe a unified augmentation pipeline for synchronized images, masks, bounding boxes, and keypoints, with inverse 2D transforms.
  • The README Quick Start imports AugmentationSequential, RandomAffine, and RandomBrightness, then applies augmentation_pipeline(img) to a batch, showing that this is a public usage path.
  • Interpolation, cropping, keypoint visibility, and coordinate conventions still affect correctness. The project insights specifically call for validating mask interpolation, box cropping, keypoint visibility, and inverse-transform behavior.
  • Project insights / key_features: “a unified augmentation pipeline can synchronously transform images, masks, bounding boxes, and keypoints”
  • README / Quick Start: “from kornia.augmentation import AugmentationSequential, RandomAffine, RandomBrightness”
  • README / Quick Start: “img = augmentation_pipeline(img)”
pip install kornia
Not stated in the README:The README excerpt does not provide complete current-version examples for every annotation type, interpolation defaults, or the exact return semantics for out-of-bounds keypoints.;It does not state whether the existing dataset coordinate format matches Kornia's required format.
It depends I need to chain Kornia geometric transforms with my own ONNX model into one production inference graph. Is the README's ONNXSequential sufficient for this workflow?
For: A deployment engineer exporting a PyTorch vision pipeline to ONNX and composing it with a proprietary model in production

It depends: the README demonstrates model chaining and export, but production suitability depends on the exact operators, input shapes, and target runtime.

  • The README’s ONNXSequential example chains a Kornia operator, an RT-DETR model from Hugging Face, and “YOUR_OWN_MODEL.onnx”, then exports chained_model.onnx. This directly supports the intended composition workflow.
  • The README states that ONNX features require extra packages, so installing only the base package does not guarantee a complete export environment.
  • The project insights note that export support varies by operator and configuration; complex control flow, dynamic shapes, and particular models still require individual validation. ONNXSequential is therefore not a guarantee of universal operator compatibility.
  • README / Quick Start: “Find out Kornia ONNX models with ONNXSequential!”
  • README / Quick Start: “Chain ONNX models from HuggingFace repo and your own local model together”
  • README / Installation: “Some features (ONNX, Stable Diffusion dissolving) need extra packages”
pip install kornia
Not stated in the README:The README excerpt does not provide the exact ONNX extra-install command, supported-operator list, dynamic-shape limitations, or target ONNX Runtime version.;It does not state whether the proprietary model's input/output signatures require adapters around Kornia operators.
It depends I need to process many sets of correspondences in batches on CUDA, using RANSAC, PnP, and fundamental-matrix estimation, while enabling half precision. Is Kornia suitable as the geometry layer for my localization pipeline?
For: A robotics or SLAM engineer running batched visual localization with RANSAC and half precision on CUDA GPUs

It depends: Kornia matches the batched GPU geometry workflow, but numerical risks in half-precision robust estimation are not eliminated by the README.

  • The project insights list batched minimal solvers, RANSAC, PnP, fundamental/essential-matrix estimation, and CUDA/half-precision support, fitting parallel processing of many geometric hypotheses on tensors.
  • The README emphasizes conformance tests, explicit conventions, and honest benchmarks, showing that geometric correctness and device performance are both project concerns.
  • RANSAC generally does not guarantee gradients, and the project insights warn that half precision can reduce stability for degenerate cases, small angles, or large-scale coordinates. Half precision therefore should not be assumed safe for every solver stage.
  • Project insights / key_features: “batched minimal solvers and RANSAC”
  • Project insights / key_features: CUDA and half-precision support, plus PnP and fundamental/essential-matrix solvers
  • README / Direction: “explicit conventions, conformance tests, and honest benchmarks”
pip install kornia
Not stated in the README:The README excerpt does not specify the target CUDA GPU, PyTorch/CUDA versions, batch size, or half-precision error thresholds for RANSAC/PnP.;It does not state whether each solver supports torch.compile or how each one handles degenerate inputs.

✨ Highlights

  • 1,200+ operators cover camera models, RANSAC, and morphology
  • gaussian_blur2d supports batched (B, 2) parameters
  • LoFTR and LightGlue feature modules run on the GPU
  • Nearly all probed configurations run with torch.compile
  • The model zoo is frozen for expansion; new integrations need a maintainer sponsor

🔧 Engineering

  • Built on PyTorch for differentiable image processing, geometric vision, and GPU batching.
  • AugmentationSequential transforms images, masks, boxes, and keypoints together.
  • find_fundamental supports batched 7-point solving and returns multiple solutions.
  • ONNXSequential chains HuggingFace and local ONNX models.

⚠️ Risks

  • RANSAC refinement and final selection run on the host in float64 by default.
  • Model-zoo expansion is frozen; new VLM/VLA integrations need a maintainer sponsor.
  • OpenCV outperforms Kornia for single-image uint8 CPU work.
  • RANSAC and DeDoDe/XFeat extraction run without gradients.

👥 For who?

  • Vision engineers processing batched images with PyTorch and CUDA.
  • Researchers needing differentiable depth, pose, and camera-parameter geometry.
  • Teams using LoFTR, LightGlue, or DISK for GPU feature matching.
  • Deployment teams needing torch.compile or ONNX export for 1,000+ operators.