maokaihe/mini_lsm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

migrate to v2 tutorial

Browse Source 
Signed-off-by: Alex Chi Z <iskyzh@gmail.com>
This commit is contained in:
Alex Chi Z
2024-01-19 12:00:36 +08:00
parent 6f262d6a8f
commit 75d85f4b45
73 changed files with 2149 additions and 191 deletions
Download Patch File Download Diff File Expand all files Collapse all files

71
mini-lsm-book/src/00-overview.md
View File

@@ -67,7 +67,7 @@ systems.
## Write Flow
![Write Flow](figures/lsm-tutorial/00-lsm-write-flow.svg)
![Write Flow](lsm-tutorial/00-lsm-write-flow.svg)
The write flow of LSM contains 4 steps:
@@ -80,38 +80,13 @@ The write flow of LSM contains 4 steps:
## Read Flow
![Read Flow](figures/lsm-tutorial/00-lsm-read-flow.svg)
![Read Flow](lsm-tutorial/00-lsm-read-flow.svg)
When we want to read a key,
1. We will first probe all the memtables from latest to oldest.
2. If the key is not found, we will then search the entire LSM tree containing SSTs to find the data.
## Tutorial Overview
![Tutorial Overview](figures/lsm-tutorial/00-lsm-tutorial-overview.svg)
In this tutorial, we will build the LSM tree structure in 7 days:
* Day 1: Block encoding. SSTs are composed of multiple data blocks. We will implement the block encoding.
* Day 2: SST encoding.
* Day 3: MemTable and Merge Iterators.
* Day 4: Block cache and Engine. To reduce disk I/O and maximize performance, we will use moka-rs to build a block cache
for the LSM tree. In this day we will get a functional (but not persistent) key-value engine with `get`, `put`, `scan`,
`delete` API.
* Day 5: Compaction. Now it's time to maintain a leveled structure for SSTs.
* Day 6: Recovery. We will implement WAL and manifest so that the engine can recover after restart.
* Day 7: Bloom filter and key compression. They are widely-used optimizations in LSM tree structures.
## Development Guide
We provide you starter code (see `mini-lsm-starter` crate), where we simply replace all function body with
`unimplemented!()`. You can start your project based on this starter code. We provide test cases, but they are very
simple. We recommend you to think carefully about your implementation and write test cases by yourself.
* You can use `cargo x scheck` to run all test cases and do style check in your codebase.
* You can use `cargo x copy-test dayX` to copy test cases to the starter code.
## Community
You may join skyzh's Discord server and study with the mini-lsm community.
@@ -138,3 +113,45 @@ vectorized expression framework if you are also interested in that topic.
[tweet]: https://twitter.com/andy_pavlo/status/1598137241016360961
[type-exercise]: https://github.com/skyzh/type-exercise-in-rust
[bustub]: https://github.com/cmu-db/bustub
<!--
## Structure
chapters + snacks, clear goal
implement, think, try by yourself
required tasks, check your understanding questions, bonus tasks
## Testing
exploring and understanding is more important than passing all the test cases
testing basic requirements, not the internal structure or something
## Solution
### Checkpoints
the final version, but many things can be simplified, read the docs
comments / tests / not up-to-date with the starter code
### How to use the solutions
## Feedbacks
join the Discord server, your feedback is important, thank GitHub users
## License
### Free forever?
### Video lectures + Review Service + Office Hour?
should have a separate preface (before you start) chapter? and what's new with v2?
## Target audience?
## What will you get after taking this course...
-->