Vlad Mihalcea

More in Technology

• Scaling NVIDIA TensorRT Performance Optimization Beyond a Single Build Route with the Global Performance Tuner, Part 1

  ### Briefing #### Scaling NVIDIA TensorRT Performance Optimization Beyond a Single Build Route with the Global Performance Tuner, Part 1 Deploying a trained neural network involves more than translating each operator into a GPU kernel. For example, when NVIDIA TensorRT builds an engine, it makes interdependent decisions about graph-IR transformations, fusion, implementation selection, memory placement, and quantization. Different scheduling choices can preserve model semantics yet produce very different inferenc…

  12 min read
• Scaling NVIDIA TensorRT Performance Optimization Beyond a Single Build Route with the Global Performance Tuner, Part 2

  NVIDIA TensorRT's Global Performance Tuner optimizes performance by tuning compiler knobs for different models, allowing for alternative scheduling strategies. This feature exposes meaningful scheduling alternatives and measures their end-to-end performance, searching for the best solution. It requires building both candidate engines and comparing results, with tunable knobs controlling aspects of the compilation pipeline. The compact API surface in TensorRT’s trtexec provides access to apply and inspect build rou…

  10 min read
• Multi-Turn Reinforcement Learning on Amazon SageMaker AI: A hands-on guide

  Multi-turn reinforcement learning (MTRL) on Amazon SageMaker AI simplifies the process of training and deploying agents for tasks that require multiple steps, such as file expense reports or support ticket triage. Amazon SageMaker AI provides a platform for training and deploying agents, making it accessible to businesses of all sizes. This tutorial demonstrates how to use MTRL on SageMaker AI, including deploying the agent and reward function, preparing the dataset, training, monitoring, and evaluating the model.…

  17 min read