fix: ensure WAL is atomic for each write batch (#84)

Signed-off-by: Alex Chi <iskyzh@gmail.com>
This commit is contained in:
Alex Chi Z
2024-07-02 20:23:33 -04:00
committed by GitHub
parent 6d16ae2d01
commit 2b527fd6dc
9 changed files with 124 additions and 38 deletions

View File

@@ -41,11 +41,40 @@ We assume that a transaction will only be used on a single thread. Once your tra
Your commit implementation should simply collect all key-value pairs from the local storage and submit a write batch to the storage engine.
## Task 4: Atomic WAL
In this task, you will need to modify:
```
src/wal.rs
src/mem_table.rs
```
Note that `commit` involves producing a write batch, and for now, the write batch does not guarantee atomicity. You will need to change the WAL implementation to produce a header and a footer for the write batch.
The new WAL encoding is as follows:
```
| HEADER | BODY | FOOTER |
| u32 | u16 | var | u64 | u16 | var | ... | u32 |
| batch_size | key_len | key | ts | value_len | value | more key-value pairs ... | checksum |
```
`batch_size` is the size of the `BODY` section. `checksum` is the checksum for the `BODY` section.
There are no test cases to verify your implementation. As long as you pass all existing test cases and implement the above WAL format, everything should be fine.
You should implement `Wal::put_batch` and `MemTable::put_batch`. The original `put` function should treat the
single key-value pair as a batch. That is to say, at this point, your `put` function should call `put_batch`.
A batch should be handled in the same mem table and the same WAL, even if it exceeds the mem table size limit.
## Test Your Understanding
* With all the things we have implemented up to this point, does the system satisfy snapshot isolation? If not, what else do we need to do to support snapshot isolation? (Note: snapshot isolation is different from serializable snapshot isolation we will talk about in the next chapter)
* What if the user wants to batch import data (i.e., 1TB?) If they use the transaction API to do that, will you give them some advice? Is there any opportunity to optimize for this case?
* What is optimistic concurrency control? What would the system be like if we implement pessimistic concurrency control instead in Mini-LSM?
* What happens if your system crashes and leave a corrupted WAL on the disk? How do you handle this situation?
## Bonus Tasks