@@ -11,12 +11,16 @@ The tiered compaction we talk about in this chapter is the same as RocksDB's uni
|
||||
|
||||
## Task 1: Universal Compaction
|
||||
|
||||
In this chapter, you will implement RocksDB's universal compaction, which is of the tiered compaction family compaction strategies. Similar to the simple leveled compaction strategy, we only use number of files as the indicator in this compaction strategy. And when we trigger the compaction jobs, we always include a full sorted run (tier) in the compaction job.
|
||||
|
||||
### Task 1.1: Triggered by Space Amplification Ratio
|
||||
|
||||
### Task 1.2: Triggered by Size Ratio
|
||||
|
||||
### Task 1.3: Reduce Sorted Runs
|
||||
|
||||
**Note: we do not provide fine-grained unit tests for this part. You can run the compaction simulator and compare with the output of the reference solution to see if your implementation is correct.**
|
||||
|
||||
## Task 2: Integrate with the Read Path
|
||||
|
||||
As tiered compaction does not use the L0 level of the LSM state, you should directly flush your memtables to a new tier instead of as an L0 SST. You can use `self.compaction_controller.flush_to_l0()` to know whether to flush to L0. You may use the first output SST id as the level/tier id for your new sorted run.
|
||||
|
@@ -9,15 +9,22 @@ In this chapter, you will:
|
||||
|
||||
## Task 1: Leveled Compaction
|
||||
|
||||
In chapter 2 day 2, you have implemented the simple leveled compaction strategies. However, the implementation has a few problems:
|
||||
|
||||
* Compaction always include a full level. Note that you cannot remove the old files until you finish the compaction, and therefore, your storage engine might use 2x storage space while the compaction is going on (if it is a full compaction). Tiered compaction has the same problem. In this chapter, we will implement partial compaction that we select one SST from the upper level for compaction, instead of the full level.
|
||||
* SSTs may be compacted across empty levels. As you have seen in the compaction simulator, when the LSM state is empty, and the engine flushes some L0 SSTs, these SSTs will be first compacted to L1, then from L1 to L2, etc. An optimal strategy is to directly place the SST from L0 to the lowest level possible, so as to avoid unnecessary write amplification.
|
||||
|
||||
In this chapter, you will implement a production-ready leveled compaction strategy. The strategy is the same as RocksDB's leveled compaction.
|
||||
|
||||
### Task 1.1: Compute Target Sizes
|
||||
|
||||
### Task 1.2: Decide Base Level
|
||||
|
||||
### Task 1.3: Decide Level Priorities
|
||||
|
||||
## Task 2: Compaction Simulation
|
||||
**Note: we do not provide fine-grained unit tests for this part. You can run the compaction simulator and compare with the output of the reference solution to see if your implementation is correct.**
|
||||
|
||||
## Task 3: Integrate with the Read Path
|
||||
## Task 2: Integrate with the Read Path
|
||||
|
||||
## Test Your Understanding
|
||||
|
||||
|
@@ -11,6 +11,8 @@ In this chapter, you will:
|
||||
|
||||
## Task 2: Write Manifests
|
||||
|
||||
## Task 3: Recover from the State
|
||||
## Task 3: Flush on Close
|
||||
|
||||
## Task 4: Recover from the State
|
||||
|
||||
{{#include copyright.md}}
|
||||
|
Reference in New Issue
Block a user