Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Duration 21 hours
Course Outline
Performance Fundamentals and Key Metrics
- Analysis of latency, throughput, energy consumption, and resource utilization
- Distinguishing between system-wide and model-specific bottlenecks
- Profiling strategies for inference versus training phases
Profiling on Huawei Ascend
- Utilizing CANN Profiler and MindInsight
- Diagnosing kernel and operator performance
- Examining offload patterns and memory mapping
Profiling on Biren GPU
- Leveraging performance monitoring features within the Biren SDK
- Addressing kernel fusion, memory alignment, and execution queues
- Conducting power and temperature-aware profiling
Profiling on Cambricon MLU
- Using BANGPy and Neuware performance tools
- Gaining kernel-level visibility and interpreting logs
- Integrating the MLU profiler with deployment frameworks
Graph and Model-Level Optimizations
- Strategies for graph pruning and quantization
- Operator fusion and restructuring computational graphs
- Standardizing input sizes and tuning batch parameters
Memory and Kernel Tuning
- Optimizing memory layout and data reuse
- Managing buffers efficiently across different chipsets
- Applying platform-specific kernel tuning techniques
Cross-Platform Best Practices
- Achieving performance portability through abstraction strategies
- Creating shared tuning pipelines for multi-chip environments
- Case study: tuning an object detection model across Ascend, Biren, and MLU
Recap and Future Directions
Requirements
- Practical experience with AI model training or deployment pipelines
- Fundamental understanding of GPU/MLU compute principles and model optimization
- Basic knowledge of performance profiling tools and key metrics
Target Audience
- Performance engineers
- Machine learning infrastructure teams
- AI system architects
