AI Optimization platforms leverage artificial intelligence to enhance decision-making and resource allocation. These tools provide advanced capabilities for process optimization, resource management, and performance improvement across various business operations.
27 verified AI-first sites in Optimization.
Cost-quality optimization for enterprise AI agents. Intelligent Gateway is a drop-in endpoint that routes traffic between cheap open models and frontier models to cut token spend; Harness Optimizer searches agent configurations against private evals; Post-Training tunes small open models to frontier quality on your workloads. Ended standalone Jamba model sales in 2026.
Pricing: Enterprise contracts; contact AI21 or try the gateway.
Microsoft's cross-platform inference engine for models exported to ONNX. Graph optimizations, quantization, and hardware execution providers for CUDA, TensorRT, DirectML, OpenVINO, CoreML, and more, from cloud servers to Windows, mobile, and the browser. Offers Python, C++, C#, Java, and JavaScript APIs for embedding models in apps.
Pricing: Free and open source.
AI software platform behind the Mojo language, the MAX inference engine, and Modular Cloud for compiling and serving models across CPUs, GPUs, NPUs, and custom silicon. Acquired by Qualcomm in July 2026; continues as a hardware-agnostic stack, with Mojo and MAX slated to be open-sourced. Mojo is a Python-syntax language for writing high-performance CPU and GPU kernels.
Pricing: Contact for pricing.
Model lab and inference provider shipping its own performance models as serverless endpoints, led by real-time image and video generation and editing tuned for the speed, cost, and quality Pareto front. Also maintains the open-source pruna framework for pruning, quantization, and distillation, and hosts optimized models with partner inference platforms. The framework applies compression methods such as pruning, quantization, caching, and distillation through a Python API.
Pricing: Per-second endpoint pricing by model; open-source framework free.
Inference cloud for running frontier open-weight and custom models in production, combining custom GPU kernels, caching, continuous batching, and speculative decoding with multi-cloud scaling. Serves models through serverless APIs, dedicated endpoints, and containers, drawing on a large Hugging Face catalog and published uptime SLAs for agent workloads. The company was founded by the team behind Orca, the iteration-level scheduling research that underpins continuous batching in modern LLM serving.
Pricing: Serverless per-token and dedicated endpoint pricing; enterprise plans on request.
Model compression company behind CompactifAI, which shrinks LLMs with quantum-inspired tensor network methods so they run cheaper across cloud, data center, and edge. Also ships its own compressed models for on-premise and on-device deployment. Raised $570M in 2026 at a $1.7B valuation; 100+ enterprise customers.
Pricing: Contact for pricing.
Intel's open-source toolkit for optimizing and deploying deep learning and generative AI models on CPUs, integrated and discrete GPUs, and NPUs. Converts models from PyTorch, TensorFlow, and ONNX, with post-training quantization and compression via NNCF. Includes an inference runtime for edge and cloud, plus OpenVINO GenAI and Model Server for LLM serving.
Pricing: Free and open source. Enterprise support: Contact Intel.
LLM compression as a service. Describe the task in plain English and it assembles a distillation, quantization, and pruning pipeline that trains a small specialized model on your own data or a matched public dataset. Aimed at replacing general-purpose API calls with self-hosted task-specific models.
Pricing: Token-based compression compute; see site for plans.
High-throughput LLM inference and serving engine. Features PagedAttention, continuous batching, CUDA/HIP graphs, quantization (GPTQ, AWQ, INT4/INT8, FP8), speculative decoding, prefix caching, and multi-LoRA. Provides OpenAI-compatible API server and broad hardware support for deploying open models efficiently.
Pricing: Free open source; see site and sponsors for support options.
High-performance serving framework for LLMs and multimodal models. Features disaggregated prefill and decode, speculative decoding, optimized schedulers and GPU kernels, and deployment from single GPU to distributed clusters. OpenAI-compatible endpoints with broad model and hardware coverage.
Pricing: Free open source; see site for support and hosting partners.
Machine learning compiler and deployment engine for large language models via MLCEngine. Compiles and runs models across GPU, CPU, and mobile with unified performance tuning. Includes OpenAI-compatible REST server and Python, JavaScript, iOS, and Android interfaces backed by the same stack.
Pricing: Free open source; see site for details.
NVIDIA inference optimization ecosystem: TensorRT compiler and runtime, TensorRT-LLM for large language models, TensorRT Model Optimizer for quantization and compression, and related tools for fusion, kernel tuning, and deployment. Integrates with PyTorch, ONNX, Hugging Face, and serving stacks such as Triton. Supports low-precision FP8, FP4, and INT8 inference on recent NVIDIA GPUs.
Pricing: SDK and open-source components; enterprise offerings via NVIDIA; see site.
Microsoft library for efficient training and inference of large models. Features ZeRO memory optimization, custom CUDA kernels, quantization, and inference-oriented runtimes including DeepSpeed-FastGen for LLM serving. Integrates with PyTorch, Hugging Face, and popular serving stacks.
Pricing: Free open source.
GPU kernel library for LLM inference serving. Customizable attention, sampling, and cascade-decoding kernels used inside vLLM, SGLang, and related servers. Targets memory-bandwidth limits on shared-prefix and batched decode workloads.
Pricing: Free open source.
Open machine learning compiler framework for graph-level and kernel-level optimization across CPUs, GPUs, and accelerators. Auto-tuning, operator fusion, and deployment to mobile, edge, and cloud targets from PyTorch, TensorFlow, and ONNX frontends. Originated at the University of Washington and is developed as an Apache Software Foundation project.
Pricing: Free open source.
NVIDIA open-source library for optimizing and serving large language models on GPUs. TensorRT kernels, in-flight batching, quantization (FP8, INT4), and multi-GPU/multi-node inference. Standard production path for high-throughput LLM serving on NVIDIA hardware alongside vLLM and Triton.
Pricing: Free and open source; requires NVIDIA GPUs.
Hugging Face production server for deploying LLMs with continuous batching, tensor parallelism, and optimized kernels. Powers Hugging Face Inference Endpoints and self-hosted open-model serving. Strong default for teams already in the Transformers ecosystem who need Rust-based serving.
Pricing: Free and open source (Apache 2.0); cloud via Hugging Face Endpoints.
Hugging Face library to accelerate Transformers, Diffusers, TIMM, and Sentence Transformers with hardware-specific optimization. Quantization, graph capture, and export paths toward ONNX Runtime, OpenVINO, TensorRT, and other backends from one toolkit. Provides ONNX Runtime and OpenVINO model classes so optimized models load with familiar Transformers-style APIs.
Pricing: Free and open source.
ML compiler and runtime that lowers models from PyTorch, JAX, and other frontends to CPU, GPU, and NPU backends. Graph and kernel compilation for edge and server deployment; sibling path to TVM in the open compiler stack. Built on the MLIR compiler infrastructure and produces compact deployable artifacts for mobile and embedded targets.
Pricing: Free and open source.
KV-cache infrastructure layer that reuses prefixes across LLM serving engines. Plugs into vLLM and similar servers to cut prefill cost on RAG, multi-turn, and shared-prompt traffic. Backed by Tensormesh, which launched its enterprise Tensormesh Platform for KV-cache management in October 2026.
Pricing: Free and open source; see LMCache.
NVIDIA open inference server for deploying ML and LLM models at scale. Dynamic batching, multi-framework backends, and production serving controls for optimized GPU inference. Now branded Dynamo-Triton, it serves models from TensorRT, PyTorch, ONNX, and Python backends behind one endpoint.
Pricing: Open source; NVIDIA enterprise support options available.
Open ML compiler stack for accelerating models across CPUs, GPUs, and accelerators. Lowers frameworks into optimized executables with shared infrastructure used across the XLA ecosystem. XLA is the compiler behind JAX and also powers TensorFlow and PyTorch/XLA workloads.
Pricing: Open source; see OpenXLA project resources.
AWS SDK and compiler for training and inference on Inferentia and Trainium. Optimizes deep learning and generative AI models for high-throughput, cost-efficient AWS accelerators. Includes the Neuron compiler, runtime, and profiling tools, with PyTorch and JAX integration.
Pricing: Included with AWS Inferentia and Trainium usage.
Hugging Face production server for embedding models with optimized kernels, dynamic batching, and high-throughput serving. Companion to Text Generation Inference for retrieval and RAG stacks. Also serves reranker and sequence-classification models alongside embedding models.
Pricing: Open source; free to self-host. See Hugging Face docs.
Toolkit for compressing, deploying, and serving LLMs and vision-language models. TurboMind and PyTorch backends with quantization, persistent batching, and multi-GPU serving including Ascend targets. Developed by the InternLM team at Shanghai AI Laboratory.
Pricing: Open source Apache-2.0; see LMDeploy docs.
Caltech-founded model lab building ultra-dense, low-bit language models. Its Ternary Bonsai 2 family spans 2B to 27B parameters with memory footprints from 0.43GB to 5.9GB, aimed at local AI on PCs, smartphones, and wearables. The company also publishes 1-bit Bonsai models and reports 9x less memory, 8x faster throughput, and 5x less energy than standard models in its own benchmarks.
Pricing: Open-source model releases; see site for access.
Open-source, Kubernetes-native stack for distributed LLM inference that runs vLLM, SGLang, and other engines across a cluster. It ships tested deployment recipes called well-lit paths, including precise prefix-cache-aware routing. Published results cite up to 3x higher output throughput and 2x faster time to first token, with portability across GPUs, TPUs, CPUs, and NPUs.
Pricing: Free and open source.