2025 Program at a Glance - FMS: the Future of Memory and Storage

01:00 PM to 02:45 PM

PRO PDSA1: Memory-Centric Computing (Part 1): Fundamental Techniques

Room 209, Meeting Rooms SCCC Floor 2

Track: Professional Development Series (Pre-Conference)

Speakers:

Onur Mutlu, Professor of Computer Science, ETH Zurich

Pre-Con Seminar Description:

This short course (consisting of two sessions, which can be taken independently of each other) covers some major ideas and techniques in modern computing platforms and applications, with a special focus on the design of the memory (and storage) system (using a cross-layer approach that spans systems, applications, software, and hardware). The first session of the two-part course enables the attendees to develop a rigorous approach in memory systems and memory-centric computing systems and gets them ready to develop methods to solve data movement and memory bottleneck problems. The major focus is on memory-centric computing, including processing near memory (PnM), processing in memory (PiM), and processing using memory (PuM) systems and techniques, to enable fundamentally higher performance, energy-efficient, and scalable systems. Both main memory and storage will be examined, with the goal of greatly improving the performance and efficiency of major workloads, such as ML/AI workloads, large language models, (LLMs), graph analytics, databases, video analytics, data analytics.

PRO PDSB1: DRAM for AI – HBM and What’s Next

Room 203, Meeting Rooms SCCC Floor 2

Track: Professional Development Series (Pre-Conference)

Speakers:

Marc Greenberg, Principal/CEO, Marc Greenberg Consulting

Pre-Con Seminar Description:

High Bandwidth Memory (HBM) is a critical technology in many processors running the latest LLMs and Generative Artificial Intelligence applications. In this PDS tutorial we will cover the case for both existing and novel techniques for interfacing DRAM such as HBM to the predominantly non-Von-Neumann compute architectures found in GPU/NPU/TPU (collectively, xPU). Attendees will learn the key aspects of HBM, including its history, architecture, and market trends, as well as a brief comparison to other popular DRAM memory types such as DDR, LPDDR, and GDDR. We'll use this foundation to discuss AI processor architectures and how they use DRAM including the impact of arithmetic intensity, quantization and sparsity on DRAM access. Finally we'll cover architectural techniques for improving memory access in AI applications ("breaking the memory wall"), including improving bandwidth, moving compute closer to RAM and moving RAM closer to compute. Takeaways from this session: - Understand the internal structure of HBM and how it evolved to be the memory of choice for datacenter AI applications - Understand the complex relationships between memory and compute in AI - Learn how the architecture of xPUs drive DRAM selection - Understand the techniques used to optimize the interaction between compute and memory in AI - See how evolving memory architectures will shape AI processors in the future

PRO PDSC1: DRAM Part 1: Fundamentals, From Cells to Modules

Room 204, Meeting Rooms SCCC Floor 2

Track: Professional Development Series (Pre-Conference)

Speakers:

Bill Gervasi, Principal Memory Solutions Architect, Monolithic Power Systems

Pre-Con Seminar Description:

Understanding computer memory architectures and tiers starts with analyzing the internal workings of the memory devices to understand how they store information, the challenges with maintaining that information, and how to get that data in and out of the attached system. As data rates have increased, new techniques have been developed to ensure data reliability and keep power under control. The majority of memory is assembled onto carriers called DIMMs which come in a variety of configurations based on the application. This training will compare and contrast the families of DIMMs. Takeaways from this session: * Understand the internal structure of DRAM, how DDR5 evolved from past implementations – and how things stalled to create the Memory Wall * See how module requirements drive DRAM architecture, and where the RAS-CAS protocol came from * Understand the many configurations of memory modules * Understand the module support subsystems and how they work together * See how system software optimizes performance

PRO PDSD1: AiSAQ™ OSS: Scaling RAG Beyond DRAM Limits with SSD

Room 210, Meeting Rooms SCCC Floor 2

Track: Professional Development Series (Pre-Conference)

Speakers:

Rory Bolt, Sr. Fellow, KIOXIA America, Inc

Pre-Con Seminar Description:

Open source software (OSS) project AiSAQ provides a new approach to scaling AI, especially for Retrieval-Augmented Generation (RAG). RAG can improve accuracy for AI models that use Approximate Nearest Neighbor Search (ANNS)) of vector databases. DiskANN was developed to store some elements of the vector database on SSDs, which enables a larger database. AiSAQ™ open-source technology allows all of the vector database elements to be stored in SSDs. This enables a number of capabilities, including a limitless vector database size, faster time-to-ready, and can connect to multiple AI host systems simultaneously. The end result is a better RAG implementation for the AI system utilizing AiSAQ technology. Learn about graph-based ANNS vs. cluster-based ANNS, how to deploy and utilize AiSAQ, the advantages for RAG/vector administrators and service providers, and an introduction to the SSD-based ANN algorithm.

03:15 PM to 05:00 PM

PRO PDSA2: Memory-Centric Computing (Part 2): Advanced Applications

Room 209, Meeting Rooms SCCC Floor 2

Track: Professional Development Series (Pre-Conference)

Speakers:

Onur Mutlu, Professor of Computer Science, ETH Zurich

Pre-Con Seminar Description:

This short course (consisting of two sessions, which can be taken independently of each other) covers some major ideas and techniques in modern computing platforms and applications, with a special focus on the design of the memory (and storage) system (using a cross-layer approach that spans systems, applications, SW and HW. The second session of this two-part course studies memory-centric computing in more depth with a focus on two other major topics, examining on both problems and effective solution techniques, covering both software and hardware levels, using a system-level cross-layer approach: 1) acceleration techniques for major data-intensive workloads, including especially ML/AI workloads, large language models, (LLMs), graph analytics, databases, video analytics, data analytics, genome analysis, mobile workloads, via memory-centric methods (across the computing stack) 2) machine learning and artificial intelligence assisted system design for better decision making, including ML/AI-driven (e.g., reinforcement learning based, DNN-based, perceptron-based) intelligent memory systems (e.g., prefetchers, storage management systems, memory controllers, data location predictors)

PRO PDSB2: Emerging Technologies for Future Memory Subsystems

Room 203, Meeting Rooms SCCC Floor 2

Track: Professional Development Series (Pre-Conference)

Speakers:

Shimeng Yu, Professor, Georgia Institute of Technology

Pre-Con Seminar Description:

We present emerging memory device technologies to fulfill the ever-increasing demands for data-intensive AI applications. We will first survey the recent industry’s research and development progresses in chip macros and prototypes including resistive random access memory (RRAM), phase change memory (PCM), magnetic random access memory (MRAM) and ferroelectric memories (FeRAM or FeFET). While many of the emerging memories are positioned to serve embedded non-volatile memories (NVMs) for automotive microcontrollers, we envision several disruptive technology breakthroughs that may revolutionize the mainstream memory hierarchy in the CPU/GPU architectures from the on-chip caches towards main memories and storage. Here are the bets: 1) for last-level-cache towards GB, we present innovations in back-end-of-line (BEOL) compatible oxide semiconductor based gain cell memories, that is monolithically stackable on top of CMOS thus overcomes the SRAM scaling limits for sub-2nm nodes; 2) for sub-10nm DRAM generations and future high-bandwidth-memory (HBM), we present a bit-cost-scalable 3D DRAM architecture of horizontal 1T1C structures; 3) for 3D NAND that scales towards 1000+ layers.

PRO PDSC2: DRAM Part 2: Systems Implications, Low Power, and HBM Alternatives

Room 204, Meeting Rooms SCCC Floor 2

Track: Professional Development Series (Pre-Conference)

Speakers:

Bill Gervasi, Principal Memory Solutions Architect, Monolithic Power Systems

Pre-Con Seminar Description:

Long gone are the days where DRAM and storage were the only two options for computer architectures. Chiplets, fabrics, and switching tiers have all entered the equation, and hybrids of memory and storage contribute to a top to bottom rethinking of data flow. Artificial intelligence as a dominant emerging technology is also affecting the equation where memory capacity requirements force a blending of multiple memory approaches to feed the beast. These new requirements are also driving an energy crisis. The combination of AI and cryptocurrency already consumes 1.5% of the energy resources of planet Earth, with an unsustainable future if design for efficiency is ignored. To address efficiency without incurring unsustainable cost requires effective total cost of ownership analyses. Takeaways from this session: * Understand systems integration of the variety of memory solutions * Understand the new memory tiers including CXL and the impact of UALINK/NVLINK * Understand how AI is driving new demands for system memory, including both DDR and LPDDR memories * See how AI and cryptocurrency are creating an energy crisis and understand the factors to address wasted power

PRO PDSD2: Rearchitecting Storage for GenAI

Room 210, Meeting Rooms SCCC Floor 2

Track: Professional Development Series (Pre-Conference)

Speakers:

Chris Newburn, Distinguished Engineer, NVIDIA

Vikram Sharma Mailthody, Sr Research Scientist, NVIDIA

Pre-Con Seminar Description:

Applications are changing faster in the GenAI space than we've ever seen. It's very difficult to keep up with shifting requirements that these applications impose on data transmission within computing elements and in storage systems. In this tutorial, the we'll share data-driven insights into storage criticality, access patterns, bandwidth, latency, and granularity. We cover a range of applications, including LLM training and inference, retrieval augmented generation, vector search and vector databases, and graph neural networks including those integrated into LLMs. From these data-driven insights, the audience will see the motivation for the Storage-Next effort that drives toward creating a new storage SKU and storage reference architecture that's focused on IOPs/TCO rather than just TB/$. Expect to leave with greater clarity on what kind of storage is needed where in the data center (global, cluster-local, compute-local) and how this relates to each kind of application.