2022-12-23 15:52:09 -05:00
# LSM in a Week
2022-12-23 21:14:11 -05:00
[](https://github.com/skyzh/mini-lsm/actions/workflows/main.yml)
2022-12-23 15:52:09 -05:00
Build a simple key-value storage engine in a week!
2022-12-23 21:14:11 -05:00
## Tutorial
The tutorial is available at [https://skyzh.github.io/mini-lsm ](https://skyzh.github.io/mini-lsm ). You can use the provided starter
code to kick off your project, and follow the tutorial to implement the LSM tree.
2024-01-18 15:15:51 +08:00
## Community
You may join skyzh's Discord server and study with the mini-lsm community.
[](https://skyzh.dev/join/discord)
2024-01-18 15:11:14 +08:00
2022-12-23 15:52:09 -05:00
## Development
```
cargo x install-tools
cargo x check
cargo x book
```
2022-12-23 22:35:38 -05:00
If you changed public API in the reference solution, you might also need to synchronize it to the starter crate.
To do this, use `cargo x sync` .
2022-12-24 00:19:29 -05:00
2024-01-21 17:45:20 +08:00
## Structure
* mini-lsm: the final solution code
* mini-lsm-starter: the starter code
* mini-lsm-book: the tutorial
We have another repo mini-lsm-solution-checkpoint at [https://github.com/skyzh/mini-lsm-solution-checkpoint ](https://github.com/skyzh/mini-lsm-solution-checkpoint ). In this repo, each commit corresponds to a chapter in the tutorial. We will not update the solution checkpoint very often.
2024-01-24 17:48:18 +08:00
## Demo
You can run the reference solution by yourself to gain an overview of the system before you start.
```
cargo run --bin mini-lsm-cli-ref
```
And we have a compaction simulator to experiment with your compaction algorithm implementation,
```
cargo run --bin compaction-simulator-ref
```
2024-01-24 17:53:00 +08:00
## Progress
We are working on chapter 3 and more test cases for all existing contents.
* Week 1: Storage Format + Engine Skeleton
* Week 2: Compaction and Persistence
* Week 3: Multi-Version Concurrency Control
* The Extra Week / Rest of Your Life: Optimizations (unlikely to be available in 2024...)
2024-01-25 23:45:00 +08:00
✅: Finished \
2024-01-24 17:53:00 +08:00
🚧: WIP and will likely be available soon
| Week + Chapter | Topic | Solution | Starter Code | Writeup |
| -------------- | ----------------------------------------------- | -------- | ------------ | ------- |
2024-01-25 23:45:00 +08:00
| 2.7 | Batch Write + Checksum | ✅ | ✅ | |
| 3.1 | Timestamp Key Encoding | ✅ | 🚧 | |
| 3.2 | Snapshot Read - Blocks, Memtables, and SSTs | ✅ | 🚧 | |
2024-01-26 16:59:03 +08:00
| 3.3 | Snapshot Read - Engine Read Path | ✅ | 🚧 | |
| 3.4 | Watermark and Garbage Collection | ✅ | 🚧 | |
| 3.5 | Transactions and Optimistic Concurrency Control | 🚧 | | |
| 3.6 | Serializable Snapshot Isolation | 🚧 | | |
| 3.7 | Compaction Filter | 🚧 | | |
2024-01-24 17:53:00 +08:00
2024-01-20 12:18:47 +08:00
## License
The Mini-LSM starter code and solution are under Apache 2.0 license. The author reserves the full copyright of the tutorial materials (markdown files and figures).
