A Multi-Agent Reinforcement Learning-Assisted Space Management Scheme for Hybrid SSDs

Abstract

Abstract

Hybrid SSDs, which allow flash cells to convert among different types of flash cells (e.g., SLC/MLC/QLC), are designed for achieving both high performance and high density. However, previous designs with two types of flash cells encounter a performance cliff degradation once the flash cells of single bit mode (SLC) are consumed. In this work, we propose a novel level-based hybrid SSD (e.g., including SLC-MLC-QLC), named RL-hybridSSD, that adopts an intermediate layer (e.g., MLC) as a performance cushion. We design a workload-aware dynamic placement mechanism that adaptively routes hot and cold writes among SLC, MLC, and QLC regions based on workload dynamics and device space pressure. Further, a multi-agent reinforcement learning-assisted space management scheme is designed to coordinate the garbage collection and mode conversion processes considering both the SSD internal status and workload patterns. We evaluated RL-hybridSSD with various real-world workloads based on simulation. The experimental results show that the proposed RL-hybridSSD provides 2.03 × higher performance on average compared with state-of-the-art schemes.