Engineering Case Study

Database Engine

This project began as an attempt to build a relational database in C. As the research progressed, the focus shifted from implementation to understanding how production database systems are engineered through architecture, subsystem decomposition and system design.

System Design Documentation

This page provides a concise overview of the project. The complete engineering documentation—including High-Level Design (HLD), Low-Level Design (LLD), algorithms, SQL grammar, storage engine, networking and developer documentation—is available in the project repository.

View Complete System Design Documentation →

Engineering Question

How would I design a modern relational database from first principles while understanding every major subsystem before writing production code?

Research Journey

Rather than immediately implementing storage structures or SQL parsing, I began by studying production database systems such as SQLite and MySQL. The research gradually evolved into a complete architectural study covering networking, query processing, transactions, storage engines, indexing and system design. Throughout the project, every subsystem was documented using High-Level Design (HLD), Low-Level Design (LLD), algorithms and engineering documentation before implementation.

Architecture Highlights

Client–Server ArchitectureNetwork Protocol DesignGateway & Thread PoolTransaction ManagementSQL InterpreterStorage EngineBuffer Pool (LRU-K)Write Ahead Log (WAL)Disk Page LayoutB+ Tree Indexing

Key Engineering Decisions

  • Layered Client → Gateway → Thread Pool → Interpreter → Storage Engine architecture.
  • Event-driven networking separated from query execution through a dedicated gateway and worker thread model.
  • Page-oriented storage using a Buffer Pool, Write Ahead Log and B+ Tree indexing.
  • Modular subsystem decomposition to keep responsibilities isolated and independently evolvable.
  • Research-first Software Development Life Cycle before any production implementation.

What This Project Taught Me

• Research is an engineering activity, not just a preparation step.
• Architecture should precede implementation.
• Large systems become manageable through subsystem decomposition.
• A database is a collection of independent systems working together.

Outcome

Although implementation was intentionally deferred, the project resulted in a comprehensive engineering specification describing the architecture of a relational database. The research produced detailed documentation covering networking, storage, transaction management, SQL processing, subsystem interactions and software architecture, forming a strong foundation for future implementation.

Current Status

Research Completed
Architecture Documented
Implementation Terminated