maokaihe/mini_lsm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

ensure compaction can finish in mvcc

Browse Source 
Signed-off-by: Alex Chi <iskyzh@gmail.com>
This commit is contained in:
Alex Chi
2024-01-25 23:45:00 +08:00
parent 63429b50d5
commit 595016f2b6
3 changed files with 8 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
README.md
View File

@@ -58,16 +58,16 @@ We are working on chapter 3 and more test cases for all existing contents.
* Week 3: Multi-Version Concurrency Control * Week 3: Multi-Version Concurrency Control
* The Extra Week / Rest of Your Life: Optimizations (unlikely to be available in 2024...) * The Extra Week / Rest of Your Life: Optimizations (unlikely to be available in 2024...)
✅: finished \ ✅: Finished \
🚧: WIP and will likely be available soon 🚧: WIP and will likely be available soon
| Week + Chapter | Topic | Solution | Starter Code | Writeup | | Week + Chapter | Topic | Solution | Starter Code | Writeup |
| -------------- | ----------------------------------------------- | -------- | ------------ | ------- | | -------------- | ----------------------------------------------- | -------- | ------------ | ------- |
| 2.7 | Batch Write + Checksum | 🚧 | 🚧 | | | 2.7 | Batch Write + Checksum | ✅ | ✅ | |
| 3.1 | Timestamp Key Encoding | 🚧 | | | | 3.1 | Timestamp Key Encoding | ✅ | 🚧 | |
| 3.2 | Snapshot Read - Blocks, Memtables, and SSTs | | | | | 3.2 | Snapshot Read - Blocks, Memtables, and SSTs | ✅ | 🚧 | |
| 3.3 | Snapshot Read - Engine Read Path | | | | | 3.3 | Snapshot Read - Engine Read Path | 🚧 | 🚧 | |
| 3.4 | Watermark and Garbage Collection | | | | | 3.4 | Watermark and Garbage Collection | 🚧 | 🚧 | |
| 3.5 | Transactions and Optimistic Concurrency Control | | | | | 3.5 | Transactions and Optimistic Concurrency Control | | | |
| 3.6 | Serializable Snapshot Isolation | | | | | 3.6 | Serializable Snapshot Isolation | | | |
| 3.7 | TTL (Time-to-Live) Entries | | | | | 3.7 | TTL (Time-to-Live) Entries | | | |

2
mini-lsm-mvcc/src/compact.rs
View File

@@ -307,7 +307,6 @@ impl LsmStorageInner {
} }
fn trigger_compaction(&self) -> Result<()> { fn trigger_compaction(&self) -> Result<()> {
self.dump_structure();
let snapshot = { let snapshot = {
let state = self.state.read(); let state = self.state.read();
state.clone() state.clone()
@@ -318,6 +317,7 @@ impl LsmStorageInner {
let Some(task) = task else { let Some(task) = task else {
return Ok(()); return Ok(());
}; };
self.dump_structure();
println!("running compaction task: {:?}", task); println!("running compaction task: {:?}", task);
let sstables = self.compact(&task)?; let sstables = self.compact(&task)?;
let output = sstables.iter().map(|x| x.sst_id()).collect::<Vec<_>>(); let output = sstables.iter().map(|x| x.sst_id()).collect::<Vec<_>>();

2
mini-lsm/src/tests/harness.rs
View File

@@ -173,7 +173,7 @@ pub fn compaction_bench(storage: Arc<MiniLsm>) {
let mut max_key = 0; let mut max_key = 0;
for iter in 0..10 { for iter in 0..10 {
let range_begin = iter * 5000; let range_begin = iter * 5000;
for i in range_begin..(range_begin + 40000) { for i in range_begin..(range_begin + 10000) {
// 120B per key, 4MB data populated // 120B per key, 4MB data populated
let key = gen_key(i); let key = gen_key(i);
let version = key_map.get(&i).copied().unwrap_or_default() + 1; let version = key_map.get(&i).copied().unwrap_or_default() + 1;