@@ -31,3 +31,5 @@ cargo x install-tools
|
|||||||
cargo x copy-test day1
|
cargo x copy-test day1
|
||||||
cargo x scheck
|
cargo x scheck
|
||||||
```
|
```
|
||||||
|
|
||||||
|
{{#include copyright.md}}
|
||||||
|
|||||||
@@ -155,3 +155,5 @@ should have a separate preface (before you start) chapter? and what's new with v
|
|||||||
|
|
||||||
## What will you get after taking this course...
|
## What will you get after taking this course...
|
||||||
-->
|
-->
|
||||||
|
|
||||||
|
{{#include copyright.md}}
|
||||||
|
|||||||
@@ -26,3 +26,5 @@ simple. We recommend you to think carefully about your implementation and write
|
|||||||
|
|
||||||
* You can use `cargo x scheck` to run all test cases and do style check in your codebase.
|
* You can use `cargo x scheck` to run all test cases and do style check in your codebase.
|
||||||
* You can use `cargo x copy-test dayX` to copy test cases to the starter code.
|
* You can use `cargo x copy-test dayX` to copy test cases to the starter code.
|
||||||
|
|
||||||
|
{{#include copyright.md}}
|
||||||
|
|||||||
@@ -112,3 +112,5 @@ Here is a list of extra tasks you can do to make the block encoding more robust
|
|||||||
|
|
||||||
* Implement block checksum. Verify checksum when decoding the block.
|
* Implement block checksum. Verify checksum when decoding the block.
|
||||||
* Compress / Decompress block. Compress on `build` and decompress on decoding.
|
* Compress / Decompress block. Compress on `build` and decompress on decoding.
|
||||||
|
|
||||||
|
{{#include copyright.md}}
|
||||||
|
|||||||
@@ -97,3 +97,5 @@ Here is a list of extra tasks you can do to make the block encoding more robust
|
|||||||
* Implement index checksum. Verify checksum when decoding.
|
* Implement index checksum. Verify checksum when decoding.
|
||||||
* Explore different SST encoding and layout. For example, in the [Lethe](https://disc-projects.bu.edu/lethe/) paper,
|
* Explore different SST encoding and layout. For example, in the [Lethe](https://disc-projects.bu.edu/lethe/) paper,
|
||||||
the author adds secondary key support to SST.
|
the author adds secondary key support to SST.
|
||||||
|
|
||||||
|
{{#include copyright.md}}
|
||||||
|
|||||||
@@ -151,3 +151,5 @@ common optimization in LSM storage engines.
|
|||||||
will stay in the memory even if it has been flushed to disk. To solve this, we can provide a `ForegroundIterator` /
|
will stay in the memory even if it has been flushed to disk. To solve this, we can provide a `ForegroundIterator` /
|
||||||
`LongIterator` to our user. The iterator will periodically create new underlying storage iterator so as to allow
|
`LongIterator` to our user. The iterator will periodically create new underlying storage iterator so as to allow
|
||||||
garbage collection of the resources.
|
garbage collection of the resources.
|
||||||
|
|
||||||
|
{{#include copyright.md}}
|
||||||
|
|||||||
@@ -89,3 +89,5 @@ Remember to change `SsTableIterator` to use the block cache.
|
|||||||
do some work. One possible optimization is to implement `WriteBatch`. We don't need to immediately write users'
|
do some work. One possible optimization is to implement `WriteBatch`. We don't need to immediately write users'
|
||||||
requests into mem-table + WAL. We can allow users to do a batch of writes.
|
requests into mem-table + WAL. We can allow users to do a batch of writes.
|
||||||
* Align blocks to 4K and use direct I/O.
|
* Align blocks to 4K and use direct I/O.
|
||||||
|
|
||||||
|
{{#include copyright.md}}
|
||||||
|
|||||||
@@ -14,3 +14,5 @@ and this chapter is now part of:
|
|||||||
</div>
|
</div>
|
||||||
|
|
||||||
We did not finish this chapter as part of Mini-LSM v1.
|
We did not finish this chapter as part of Mini-LSM v1.
|
||||||
|
|
||||||
|
{{#include copyright.md}}
|
||||||
|
|||||||
@@ -11,3 +11,5 @@ and this chapter is now part of:
|
|||||||
</div>
|
</div>
|
||||||
|
|
||||||
We did not finish this chapter as part of Mini-LSM v1.
|
We did not finish this chapter as part of Mini-LSM v1.
|
||||||
|
|
||||||
|
{{#include copyright.md}}
|
||||||
|
|||||||
@@ -9,3 +9,5 @@ and this chapter is now part of [Mini LSM Week 1 Day 7: SST Optimizations](./wee
|
|||||||
</div>
|
</div>
|
||||||
|
|
||||||
We did not finish this chapter as part of Mini-LSM v1.
|
We did not finish this chapter as part of Mini-LSM v1.
|
||||||
|
|
||||||
|
{{#include copyright.md}}
|
||||||
|
|||||||
@@ -8,3 +8,5 @@ and this chapter is now part of [Mini LSM Week 1 Day 7: SST Optimizations](./wee
|
|||||||
</div>
|
</div>
|
||||||
|
|
||||||
We did not finish this chapter as part of Mini-LSM v1.
|
We did not finish this chapter as part of Mini-LSM v1.
|
||||||
|
|
||||||
|
{{#include copyright.md}}
|
||||||
|
|||||||
@@ -1,3 +1,5 @@
|
|||||||
# What's Next
|
# What's Next
|
||||||
|
|
||||||
We did not finish this chapter as part of Mini-LSM v1.
|
We did not finish this chapter as part of Mini-LSM v1.
|
||||||
|
|
||||||
|
{{#include copyright.md}}
|
||||||
|
|||||||
1
mini-lsm-book/src/copyright.md
Normal file
1
mini-lsm-book/src/copyright.md
Normal file
@@ -0,0 +1 @@
|
|||||||
|
<p style="text-color: grey; text-align: center; margin-top: 3em"><small>Copyright © 2022 - 2024 Alex Chi Z. All Rights Reserved.</small></p>
|
||||||
@@ -13,3 +13,5 @@ In this chapter, you will:
|
|||||||
## Task 2: Write Path - Freezing a Memtable
|
## Task 2: Write Path - Freezing a Memtable
|
||||||
|
|
||||||
## Task 3: Read Path - Get
|
## Task 3: Read Path - Get
|
||||||
|
|
||||||
|
{{#include copyright.md}}
|
||||||
|
|||||||
@@ -13,3 +13,5 @@ In this chapter, you will:
|
|||||||
## Task 2: Merge Iterator
|
## Task 2: Merge Iterator
|
||||||
|
|
||||||
## Task 3: Read Path - Scan
|
## Task 3: Read Path - Scan
|
||||||
|
|
||||||
|
{{#include copyright.md}}
|
||||||
|
|||||||
@@ -10,3 +10,5 @@ In this chapter, you will:
|
|||||||
## Task 1: Block Builder
|
## Task 1: Block Builder
|
||||||
|
|
||||||
## Task 2: Block Iterator
|
## Task 2: Block Iterator
|
||||||
|
|
||||||
|
{{#include copyright.md}}
|
||||||
|
|||||||
@@ -10,3 +10,5 @@ In this chapter, you will:
|
|||||||
## Task 1: SST Builder
|
## Task 1: SST Builder
|
||||||
|
|
||||||
## Task 2: SST Iterator
|
## Task 2: SST Iterator
|
||||||
|
|
||||||
|
{{#include copyright.md}}
|
||||||
|
|||||||
@@ -13,3 +13,5 @@ In this chapter, you will:
|
|||||||
## Task 2: Read Path - Get
|
## Task 2: Read Path - Get
|
||||||
|
|
||||||
## Task 3: Read Path - Scan
|
## Task 3: Read Path - Scan
|
||||||
|
|
||||||
|
{{#include copyright.md}}
|
||||||
|
|||||||
@@ -10,3 +10,5 @@ In this chapter, you will:
|
|||||||
## Task 1: Flush Memtable to SST
|
## Task 1: Flush Memtable to SST
|
||||||
|
|
||||||
## Task 2: Update the LSM State
|
## Task 2: Update the LSM State
|
||||||
|
|
||||||
|
{{#include copyright.md}}
|
||||||
|
|||||||
@@ -8,3 +8,5 @@ In this chapter, you will:
|
|||||||
|
|
||||||
* Implement bloom filter on SSTs and integrate into the LSM read path `get`.
|
* Implement bloom filter on SSTs and integrate into the LSM read path `get`.
|
||||||
* Implement key compression in SST block format.
|
* Implement key compression in SST block format.
|
||||||
|
|
||||||
|
{{#include copyright.md}}
|
||||||
|
|||||||
@@ -4,3 +4,4 @@
|
|||||||
|
|
||||||
key functionalities index
|
key functionalities index
|
||||||
|
|
||||||
|
{{#include copyright.md}}
|
||||||
|
|||||||
@@ -7,3 +7,5 @@ In this chapter, you will:
|
|||||||
* Implement the compaction logic that combines some files and produces new files.
|
* Implement the compaction logic that combines some files and produces new files.
|
||||||
* Implement the logic to update the LSM states and manage SST files on the filesystem.
|
* Implement the logic to update the LSM states and manage SST files on the filesystem.
|
||||||
* Update LSM read path to incorporate the LSM levels.
|
* Update LSM read path to incorporate the LSM levels.
|
||||||
|
|
||||||
|
{{#include copyright.md}}
|
||||||
|
|||||||
@@ -6,3 +6,5 @@ In this chapter, you will:
|
|||||||
|
|
||||||
* Implement a simple leveled compaction strategy and simulate it on the compaction simulator.
|
* Implement a simple leveled compaction strategy and simulate it on the compaction simulator.
|
||||||
* Start compaction as a background task and implement a compaction trigger in the system.
|
* Start compaction as a background task and implement a compaction trigger in the system.
|
||||||
|
|
||||||
|
{{#include copyright.md}}
|
||||||
|
|||||||
@@ -6,3 +6,5 @@ In this chapter, you will:
|
|||||||
|
|
||||||
* Implement a tiered compaction strategy and simulate it on the compaction simulator.
|
* Implement a tiered compaction strategy and simulate it on the compaction simulator.
|
||||||
* Incorporate tiered compaction strategy into the system.
|
* Incorporate tiered compaction strategy into the system.
|
||||||
|
|
||||||
|
{{#include copyright.md}}
|
||||||
|
|||||||
@@ -6,3 +6,5 @@ In this chapter, you will:
|
|||||||
|
|
||||||
* Implement a leveled compaction strategy and simulate it on the compaction simulator.
|
* Implement a leveled compaction strategy and simulate it on the compaction simulator.
|
||||||
* Incorporate leveled compaction strategy into the system.
|
* Incorporate leveled compaction strategy into the system.
|
||||||
|
|
||||||
|
{{#include copyright.md}}
|
||||||
|
|||||||
@@ -6,3 +6,5 @@ In this chapter, you will:
|
|||||||
|
|
||||||
* Implement encoding and decoding of the manifest file.
|
* Implement encoding and decoding of the manifest file.
|
||||||
* Recover from the manifest when the system restarts.
|
* Recover from the manifest when the system restarts.
|
||||||
|
|
||||||
|
{{#include copyright.md}}
|
||||||
|
|||||||
@@ -6,3 +6,5 @@ In this chapter, you will:
|
|||||||
|
|
||||||
* Implement encoding and decoding of the write-ahead log file.
|
* Implement encoding and decoding of the write-ahead log file.
|
||||||
* Recover memtables from the WALs when the system restarts.
|
* Recover memtables from the WALs when the system restarts.
|
||||||
|
|
||||||
|
{{#include copyright.md}}
|
||||||
|
|||||||
@@ -8,3 +8,5 @@ In this chapter, you will:
|
|||||||
|
|
||||||
* Implement the batch write interface.
|
* Implement the batch write interface.
|
||||||
* Add checksums to the blocks, SST metadata, manifest, and WALs.
|
* Add checksums to the blocks, SST metadata, manifest, and WALs.
|
||||||
|
|
||||||
|
{{#include copyright.md}}
|
||||||
|
|||||||
@@ -9,3 +9,5 @@ ask 2 what if questions
|
|||||||
LSM is about tradeoffs
|
LSM is about tradeoffs
|
||||||
|
|
||||||
learning path: feel free to skip compactions?
|
learning path: feel free to skip compactions?
|
||||||
|
|
||||||
|
{{#include copyright.md}}
|
||||||
|
|||||||
@@ -1 +1,3 @@
|
|||||||
# Overview
|
# Overview
|
||||||
|
|
||||||
|
{{#include copyright.md}}
|
||||||
|
|||||||
Reference in New Issue
Block a user