2025 Theses Doctoral

Scalable Silicon Systems for the Accelerator Era

Zuckerman, Joseph

System-on-chip (SoC) architectures have evolved rapidly in the past two decades. The early 2000s saw a shift from single-core to multi-core architectures due to technology scaling trends. In the past decade, the limits of multi-core scaling have instead pushed SoCs into an era of specialization. The semiconductor industry has turned to heterogeneous SoCs, which combine general-purpose processors with various specialized hardware units called accelerators, to continue to push the bounds of performance and energy efficiency. As this trend has become pervasive across various domains, we are surely in the Accelerator Era of computing.

Accelerator-rich SoCs, however, are much more difficult to implement than their homogeneous counterparts, as they require the design, integration, and verification of many different components. One solution to lower the growing design costs is to reuse components developed by other teams. However, small academic and industrial teams do not have the budgets to acquire expensive intellectual property. The growing open-source hardware (OSH) movement presents an opportunity to foster truly collaborative engineering. Recently, there has been notable progress towards an OSH ecosystem, thanks to an increased availability of OSH components and tools.

Despite this progress, several key challenges remain. First, integrating many components, designed by different teams, with different languages and tools is no easy task; arriving at silicon implementations, or even just FPGA prototypes, of complete SoCs can be daunting for small teams. Second, the usage of many heterogeneous components can cause contention on key system resources, such as memory, interconnect, and power. Because these components are designed independently, they have no notion of the status of the surrounding system.

Methods to optimize system-level performance must therefore be independent of the design of components themselves. My thesis is that decoupling the design of components, in particular accelerators, from the surrounding SoC architecture is the key to supporting flexible integration and efficient resource management, thereby enabling the scalable design of complex silicon prototypes and the optimization of system-level performance.

Throughout this dissertation, I identify the shortcomings of existing solutions to various computer architecture and integrated-circuit design problems when they are applied to heterogeneous SoCs; the characteristics of these systems necessitate a rethinking of traditional approaches. I propose new solutions that are scalable to large designs and are not only tailored to the diverse characteristics of accelerators but are also decoupled from the design of any particular accelerator. Specifically, these solutions innovate on the management of critical shared resources – such as the memory hierarchy, on-chip interconnect, and power – in many-accelerator SoCs. The proposed solutions are built on top of ESP, an open-source platform for heterogeneous SoC design, to allow for their rigorous evaluation in complete SoC prototypes. On top of FPGA-based prototyping, these solutions are also implemented in fabricated ESP-based chips, thanks to a new agile design methodology for silicon prototyping. I show how this methodology enabled small teams to implement several complex silicon prototypes, such as the EPOCHS-1 SoC. With 14 different types of accelerators, multiple RISC-V cores that coherently boot a Linux operating system, and novel strategies for data orchestration and power management, EPOCHS-1 is arguably the most complex SoC demonstrated in the literature by an academic team.

The contributions of my dissertation also have broader impact on both research and teaching. I have contributed all of my work back into the open-source release of ESP, thereby strengthening its offering to the OSH community. In fact, in just the past few years, ESP has been used for research at over 20 institutions. I have also helped bring ESP into the classroom at Columbia University to teach collaborative SoC design to the next generation of engineers.