2022-12-23 15:52:09 -05:00
|
|
|
# LSM in a Week
|
|
|
|
|
|
2022-12-23 21:14:11 -05:00
|
|
|
[](https://github.com/skyzh/mini-lsm/actions/workflows/main.yml)
|
|
|
|
|
|
2022-12-23 15:52:09 -05:00
|
|
|
Build a simple key-value storage engine in a week!
|
|
|
|
|
|
2022-12-23 21:14:11 -05:00
|
|
|
## Tutorial
|
|
|
|
|
|
|
|
|
|
The tutorial is available at [https://skyzh.github.io/mini-lsm](https://skyzh.github.io/mini-lsm). You can use the provided starter
|
|
|
|
|
code to kick off your project, and follow the tutorial to implement the LSM tree.
|
|
|
|
|
|
2022-12-23 15:52:09 -05:00
|
|
|
## Development
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
cargo x install-tools
|
|
|
|
|
cargo x check
|
|
|
|
|
cargo x book
|
|
|
|
|
```
|
2022-12-23 22:35:38 -05:00
|
|
|
|
|
|
|
|
If you changed public API in the reference solution, you might also need to synchronize it to the starter crate.
|
|
|
|
|
To do this, use `cargo x sync`.
|
2022-12-24 00:19:29 -05:00
|
|
|
|
|
|
|
|
## Progress
|
|
|
|
|
|
2024-01-10 14:25:23 +08:00
|
|
|
We are working on a new version of the mini-lsm tutorial that is split into 3 weeks.
|
|
|
|
|
|
|
|
|
|
* Week 1: Storage Format + Engine Skeleton
|
|
|
|
|
* Week 2: Compaction and Persistence
|
2024-01-10 14:26:06 +08:00
|
|
|
* Week 3: Multi-Version Concurrency Control
|
2024-01-10 14:25:23 +08:00
|
|
|
|
|
|
|
|
| Week + Chapter | Topic | Solution | Starter Code | Writeup |
|
|
|
|
|
| ---- | ------------------ | --------------- | ----------------- | --------- |
|
|
|
|
|
| 1.1 | Block Format | ✅ | ✅ | ✅ |
|
|
|
|
|
| 1.2 | Table Format | ✅ | ✅ | ✅ | |
|
|
|
|
|
| 1.3 | Memtables | ✅ | ✅ | ✅ | |
|
|
|
|
|
| 1.4 | Merge Iterators | ✅ | ✅ | ✅ |
|
|
|
|
|
| 1.5 | Storage Engine - Read Path | ✅ | ✅ | ✅ |
|
|
|
|
|
| 1.6 | Storage Engine - Write Path | ✅ | ✅ | ✅ |
|
|
|
|
|
| 2.1 | Compaction Framework | ✅ | 🚧 | 🚧 |
|
|
|
|
|
| 2.2 | Compaction Strategy | 🚧 | | |
|
2024-01-10 14:38:29 +08:00
|
|
|
| 2.3 | Manifest | | | |
|
|
|
|
|
| 2.4 | Write-Ahead Log | | | |
|
|
|
|
|
| 2.5 | Bloom Filter and Key Compression | | | |
|
|
|
|
|
| 2.6 | Benchmarking | | | |
|
|
|
|
|
| 3.1 | Timestamp Encoding + Prefix Bloom Filter | | | |
|
|
|
|
|
| 3.2 | Snapshot Read | | | |
|
|
|
|
|
| 3.3 | Watermark and Garbage Collection | | | |
|
|
|
|
|
| 3.4 | Transactions | | | |
|
|
|
|
|
| 3.5 | Serializable Snapshot Isolation | | | |
|
|
|
|
|
| 3.6 | What's Next... | | | |
|