2025 Theses Doctoral

Optimizing Distributed Transactions via Modern AI, Storage and Networking Technologies

Eldeeb, Tamer

Distributed ACID transactions, once declared as hopelessly unscalable and unnecessary, are back by popular developer demand. Unfortunately, designing on-disk database systems that support distributed transactions remains quite challenging. Designers of such systems typically face a difficult choice. They can either use an expensive commit protocol like two-phase commit (2PC) to guarantee atomicity, and suffer from slow distributed transactions, or forgo 2PC, which leads to weaker semantics, limitations to the programming model, or constrained scalability, making the system less general. Therefore, there is a trade-off between speed and generality in distributed transactions database systems.

This thesis posits that it is time to revisit that trade-off. We argue that modern developments in AI, storage, and networking unlock the potential of database system designs that offer fast and general distributed transactions. We tackle the problem from different angles. First, we leverage low-latency networking, storage hardware and system software to directly reduce the cost of the commit protocol. Second, when such a low latency stack is unavailable, we design algorithms to mask latency and maintain high throughput in the face of slow 2PC. Finally, we explore applying Reinforcement Learning to the sharding problem so that we reduce the number of distributed transactions the system has to execute without sacrificing other important objectives.