Github Designing Data-intensive Applications Info

The new data-intensive architecture brought significant improvements to GitHub's platform:

This is where gh-ost (GitHub Online Schema Tool) shines. Traditional ALTER TABLE locks the table, blocking writes for minutes or hours. gh-ost instead creates a shadow table with the new schema, copies data in small chunks, and replays the binary log of writes from the original table onto the shadow table—all while the application continues running. At the final moment, it performs a near-instantaneous atomic swap of table names. This is a direct implementation of Kleppmann’s discussion of and eventual consistency . The system is in a temporary, inconsistent state (rows exist in both tables), but the application logic hides this complexity. The maintainability payoff is immense: GitHub can deploy schema changes hundreds of times per day, a velocity unthinkable in a system that required scheduled maintenance windows. github designing data-intensive applications

To overcome these challenges, the GitHub team adopted a data-intensive architecture, centered around the following key components: At the final moment, it performs a near-instantaneous

Repositories that let you spin up a distributed cluster (Kafka, ZooKeeper, Postgres) with a single command. The maintainability payoff is immense: GitHub can deploy