Revisiting Data Updates in Erasure-Coded Storage Clusters

Full abstract

Read the full abstract

Erasure coding is widely adopted to maintain data reliability, yet it introduces a significant update penalty. We analyze real-world traces and observe several challenges that are not addressed by existing studies, which thereby restricts the performance gains. We propose FastUpdate , an efficient multi-stripe updates framework that assists existing update schemes for fast updates. FastUpdate comprises three key designs: (i) it perceives the update locality and carefully merges multiple update requests accessing the same stripe to reduce the incurred network traffic; (ii) it abstracts the existing update schemes into collector selection and tree construction, greedily generates the update solution for each stripe to balance the transmission load across nodes; (iii) it dynamically schedules appropriate stripes to update in heterogeneous and dynamic networks to fully utilize the bandwidth resources. Comprehensive evaluations verify the effectiveness of FastUpdate on Alibaba ECS. It can increase the update throughput by 16.15%-88.71% for various update schemes.