add compaction tests and fix bugs in compaction

Signed-off-by: Alex Chi <iskyzh@gmail.com>
This commit is contained in:
Alex Chi
2024-01-25 15:25:23 +08:00
parent 971d0b1c81
commit 8dbaf54e38
23 changed files with 379 additions and 42 deletions

View File

@@ -8,6 +8,13 @@ In this chapter, you will:
* Implement the logic to update the LSM states and manage SST files on the filesystem.
* Update LSM read path to incorporate the LSM levels.
To copy the test cases into the starter code and run them,
```
cargo x copy-test --week 2 --day 1
cargo x scheck
```
## Task 1: Compaction Implementation
In this task, you will implement the core logic of doing a compaction -- merge sort a set of SST files into a sorted run. You will need to modify:

View File

@@ -7,6 +7,13 @@ In this chapter, you will:
* Implement a simple leveled compaction strategy and simulate it on the compaction simulator.
* Start compaction as a background task and implement a compaction trigger in the system.
To copy the test cases into the starter code and run them,
```
cargo x copy-test --week 2 --day 2
cargo x scheck
```
## Task 1: Simple Leveled Compaction
In this chapter, we are going to implement our first compaction strategy -- simple leveled compaction. In this task, you will need to modify:

View File

@@ -9,6 +9,13 @@ In this chapter, you will:
The tiered compaction we talk about in this chapter is the same as RocksDB's universal compaction. We will use these two terminologies interchangeably.
To copy the test cases into the starter code and run them,
```
cargo x copy-test --week 2 --day 3
cargo x scheck
```
## Task 1: Universal Compaction
In this chapter, you will implement RocksDB's universal compaction, which is of the tiered compaction family compaction strategies. Similar to the simple leveled compaction strategy, we only use number of files as the indicator in this compaction strategy. And when we trigger the compaction jobs, we always include a full sorted run (tier) in the compaction job.

View File

@@ -7,6 +7,13 @@ In this chapter, you will:
* Implement a leveled compaction strategy and simulate it on the compaction simulator.
* Incorporate leveled compaction strategy into the system.
To copy the test cases into the starter code and run them,
```
cargo x copy-test --week 2 --day 4
cargo x scheck
```
## Task 1: Leveled Compaction
In chapter 2 day 2, you have implemented the simple leveled compaction strategies. However, the implementation has a few problems: