The session presents two open-source frameworks, SGLN and Miles, designed to build scalable, high-performance AI infrastructure optimized for AMD hardware, supporting diverse AI models and complex agentic workloads while enabling efficient reinforcement learning fine-tuning. Developed by Redisarc, these tools aim to democratize frontier AI technology by fostering collaboration and continuous improvement within the community.
The session introduces two open-source frameworks, SGLN and Miles, developed to build frontier AI infrastructure. SGLN is a high-performance inference engine supporting multiple modalities including language models, visual language models, diffusion models for images and videos, and audio models. It is optimized for AMD hardware and widely adopted in production across various industries. Miles complements SGLN by providing an enterprise-grade reinforcement learning (RL) framework that enables users to fine-tune models through a loop of inference rollouts and training updates, enhancing model performance for specific use cases.
The speaker highlights the rapid evolution of AI models and infrastructure, using DeepSigleaf 4 as an example. On day zero, the model was runnable on AMD GPUs, and within weeks, performance improved eightfold, achieving throughput comparable or superior to competing hardware at a lower total cost of ownership. SGLN supports a broad range of mainstream models across different AI categories and is used by numerous companies spanning hardware manufacturers, hyperscalers, enterprises, and startups, demonstrating its versatility and scalability.
Agentic workloads, characterized by complex multi-turn interactions and tool calls, present unique challenges for AI infrastructure. These include the need for high cache reuse due to repetitive patterns, handling extremely large models with trillions of parameters and mixture-of-experts layers, and managing very long context lengths that can reach millions of tokens. SGLN addresses these challenges through innovations such as a unified hybrid registry for cache management, leveraging host and external storage to increase cache hit rates, and advanced parallelism strategies like chunked pipeline and context parallelism to optimize latency and throughput.
On the reinforcement learning side, Miles tackles challenges related to diverse environments, training-inference consistency, and maximizing throughput. It integrates with various external tools and environments, ensures deterministic kernel execution to avoid training-inference mismatches, and employs techniques like token-in token-out alignment and rollout replay to maintain policy correctness. To improve system utilization, Miles supports asynchronous training and inference pipelines and leverages low-precision training methods such as 8-bit and 4-bit quantization to accelerate performance.
The talk concludes by emphasizing the ongoing collaboration with AMD and the broader ecosystem to optimize these frameworks for AMD hardware, achieving significant performance gains. Redisarc, the startup behind SGLN and Miles, aims to democratize frontier AI infrastructure by making it open, accessible, and scalable for everyone. Their mission is to enable researchers and developers worldwide to experiment and innovate at the cutting edge of AI, supported by continuous improvements in stability, usability, observability, and performance. Attendees are invited to join the SGLN and Miles community to contribute and benefit from these advancements.