@@ -1,3 +1,5 @@
|
|||||||
|

|
||||||
|
|
||||||
# LSM in a Week
|
# LSM in a Week
|
||||||
|
|
||||||
[](https://github.com/skyzh/mini-lsm/actions/workflows/main.yml)
|
[](https://github.com/skyzh/mini-lsm/actions/workflows/main.yml)
|
||||||
|
|||||||
@@ -1,5 +1,17 @@
|
|||||||
# Mini-LSM Course Overview
|
# Mini-LSM Course Overview
|
||||||
|
|
||||||
|
## Tutorial Structure
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|
We have 3 parts (weeks) for this tutorial. In the first week, we will focus on the storage structure and the storage format of an LSM storage engine. In the second week, we will dive into compactions in depth and implement persistence support for the storage engine. In the third week, we will implement multi-version concurrency control.
|
||||||
|
|
||||||
|
* [The First Week: Mini-LSM](./week1-overview.md)
|
||||||
|
* [The Second Week: Compaction and Persistence](./week2-overview.md)
|
||||||
|
* [The Third Week: Multi-Version Concurrency Control](./week3-overview.md)
|
||||||
|
|
||||||
|
To set up the environment, please take a look at [Environment Setup](./00-get-started.md).
|
||||||
|
|
||||||
## Overview of LSM
|
## Overview of LSM
|
||||||
|
|
||||||
An LSM storage engine generally contains 3 parts:
|
An LSM storage engine generally contains 3 parts:
|
||||||
@@ -49,16 +61,4 @@ When we want to read a key,
|
|||||||
|
|
||||||
There are two types of read: lookup and scan. Lookup finds one key in the LSM tree, while scan iterates all keys within a range in the storage engine. We will cover both of them throughout the tutorial.
|
There are two types of read: lookup and scan. Lookup finds one key in the LSM tree, while scan iterates all keys within a range in the storage engine. We will cover both of them throughout the tutorial.
|
||||||
|
|
||||||
## Tutorial Structure
|
|
||||||
|
|
||||||

|
|
||||||
|
|
||||||
We have 3 parts (weeks) for this tutorial. In the first week, we will focus on the storage structure and the storage format of an LSM storage engine. In the second week, we will dive into compactions in depth and implement persistence support for the storage engine. In the third week, we will implement multi-version concurrency control.
|
|
||||||
|
|
||||||
* [The First Week: Mini-LSM](./week1-overview.md)
|
|
||||||
* [The Second Week: Compaction and Persistence](./week2-overview.md)
|
|
||||||
* [The Third Week: Multi-Version Concurrency Control](./week3-overview.md)
|
|
||||||
|
|
||||||
To set up the environment, please take a look at [Environment Setup](./00-get-started.md).
|
|
||||||
|
|
||||||
{{#include copyright.md}}
|
{{#include copyright.md}}
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# Preface
|
# Preface
|
||||||
|
|
||||||

|

|
||||||
|
|
||||||
In this tutorial, you will learn how to build a simple LSM-Tree storage engine in the Rust programming language.
|
In this tutorial, you will learn how to build a simple LSM-Tree storage engine in the Rust programming language.
|
||||||
|
|
||||||
|
|||||||
BIN
mini-lsm-book/src/mini-lsm-logo.png
Normal file
BIN
mini-lsm-book/src/mini-lsm-logo.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 65 KiB |
@@ -28,5 +28,12 @@ https://skyzh.github.io/mini-lsm/week2-05-manifest
|
|||||||
https://skyzh.github.io/mini-lsm/week2-06-wal
|
https://skyzh.github.io/mini-lsm/week2-06-wal
|
||||||
https://skyzh.github.io/mini-lsm/week2-07-snacks
|
https://skyzh.github.io/mini-lsm/week2-07-snacks
|
||||||
https://skyzh.github.io/mini-lsm/week2-overview
|
https://skyzh.github.io/mini-lsm/week2-overview
|
||||||
|
https://skyzh.github.io/mini-lsm/week3-01-ts-key-refactor
|
||||||
|
https://skyzh.github.io/mini-lsm/week3-02-snapshot-read-part-1
|
||||||
|
https://skyzh.github.io/mini-lsm/week3-03-snapshot-read-part-2
|
||||||
|
https://skyzh.github.io/mini-lsm/week3-04-watermark
|
||||||
|
https://skyzh.github.io/mini-lsm/week3-05-txn-occ
|
||||||
|
https://skyzh.github.io/mini-lsm/week3-06-serializable
|
||||||
|
https://skyzh.github.io/mini-lsm/week3-07-compaction-filter
|
||||||
https://skyzh.github.io/mini-lsm/week3-overview
|
https://skyzh.github.io/mini-lsm/week3-overview
|
||||||
https://skyzh.github.io/mini-lsm/week4-overview
|
https://skyzh.github.io/mini-lsm/week4-overview
|
||||||
|
|||||||
@@ -2,130 +2,158 @@
|
|||||||
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
|
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
|
||||||
<url>
|
<url>
|
||||||
<loc>https://skyzh.github.io/mini-lsm</loc>
|
<loc>https://skyzh.github.io/mini-lsm</loc>
|
||||||
<lastmod>2024-01-25T02:56:28.231Z</lastmod>
|
<lastmod>2024-01-30T08:17:56.731Z</lastmod>
|
||||||
</url>
|
</url>
|
||||||
<url>
|
<url>
|
||||||
<loc>https://skyzh.github.io/mini-lsm/00-get-started</loc>
|
<loc>https://skyzh.github.io/mini-lsm/00-get-started</loc>
|
||||||
<lastmod>2024-01-25T02:56:28.234Z</lastmod>
|
<lastmod>2024-01-30T08:17:56.734Z</lastmod>
|
||||||
</url>
|
</url>
|
||||||
<url>
|
<url>
|
||||||
<loc>https://skyzh.github.io/mini-lsm/00-overview</loc>
|
<loc>https://skyzh.github.io/mini-lsm/00-overview</loc>
|
||||||
<lastmod>2024-01-25T02:56:28.232Z</lastmod>
|
<lastmod>2024-01-30T08:17:56.732Z</lastmod>
|
||||||
</url>
|
</url>
|
||||||
<url>
|
<url>
|
||||||
<loc>https://skyzh.github.io/mini-lsm/00-preface</loc>
|
<loc>https://skyzh.github.io/mini-lsm/00-preface</loc>
|
||||||
<lastmod>2024-01-25T02:56:28.230Z</lastmod>
|
<lastmod>2024-01-30T08:17:56.730Z</lastmod>
|
||||||
</url>
|
</url>
|
||||||
<url>
|
<url>
|
||||||
<loc>https://skyzh.github.io/mini-lsm/00-v1</loc>
|
<loc>https://skyzh.github.io/mini-lsm/00-v1</loc>
|
||||||
<lastmod>2024-01-25T02:56:28.256Z</lastmod>
|
<lastmod>2024-01-30T08:17:56.762Z</lastmod>
|
||||||
</url>
|
</url>
|
||||||
<url>
|
<url>
|
||||||
<loc>https://skyzh.github.io/mini-lsm/01-block</loc>
|
<loc>https://skyzh.github.io/mini-lsm/01-block</loc>
|
||||||
<lastmod>2024-01-25T02:56:28.257Z</lastmod>
|
<lastmod>2024-01-30T08:17:56.763Z</lastmod>
|
||||||
</url>
|
</url>
|
||||||
<url>
|
<url>
|
||||||
<loc>https://skyzh.github.io/mini-lsm/02-sst</loc>
|
<loc>https://skyzh.github.io/mini-lsm/02-sst</loc>
|
||||||
<lastmod>2024-01-25T02:56:28.258Z</lastmod>
|
<lastmod>2024-01-30T08:17:56.764Z</lastmod>
|
||||||
</url>
|
</url>
|
||||||
<url>
|
<url>
|
||||||
<loc>https://skyzh.github.io/mini-lsm/03-memtable</loc>
|
<loc>https://skyzh.github.io/mini-lsm/03-memtable</loc>
|
||||||
<lastmod>2024-01-25T02:56:28.259Z</lastmod>
|
<lastmod>2024-01-30T08:17:56.765Z</lastmod>
|
||||||
</url>
|
</url>
|
||||||
<url>
|
<url>
|
||||||
<loc>https://skyzh.github.io/mini-lsm/04-engine</loc>
|
<loc>https://skyzh.github.io/mini-lsm/04-engine</loc>
|
||||||
<lastmod>2024-01-25T02:56:28.260Z</lastmod>
|
<lastmod>2024-01-30T08:17:56.766Z</lastmod>
|
||||||
</url>
|
</url>
|
||||||
<url>
|
<url>
|
||||||
<loc>https://skyzh.github.io/mini-lsm/05-compaction</loc>
|
<loc>https://skyzh.github.io/mini-lsm/05-compaction</loc>
|
||||||
<lastmod>2024-01-25T02:56:28.261Z</lastmod>
|
<lastmod>2024-01-30T08:17:56.767Z</lastmod>
|
||||||
</url>
|
</url>
|
||||||
<url>
|
<url>
|
||||||
<loc>https://skyzh.github.io/mini-lsm/06-recovery</loc>
|
<loc>https://skyzh.github.io/mini-lsm/06-recovery</loc>
|
||||||
<lastmod>2024-01-25T02:56:28.262Z</lastmod>
|
<lastmod>2024-01-30T08:17:56.768Z</lastmod>
|
||||||
</url>
|
</url>
|
||||||
<url>
|
<url>
|
||||||
<loc>https://skyzh.github.io/mini-lsm/07-bloom-filter</loc>
|
<loc>https://skyzh.github.io/mini-lsm/07-bloom-filter</loc>
|
||||||
<lastmod>2024-01-25T02:56:28.263Z</lastmod>
|
<lastmod>2024-01-30T08:17:56.768Z</lastmod>
|
||||||
</url>
|
</url>
|
||||||
<url>
|
<url>
|
||||||
<loc>https://skyzh.github.io/mini-lsm/08-key-compression</loc>
|
<loc>https://skyzh.github.io/mini-lsm/08-key-compression</loc>
|
||||||
<lastmod>2024-01-25T02:56:28.264Z</lastmod>
|
<lastmod>2024-01-30T08:17:56.769Z</lastmod>
|
||||||
</url>
|
</url>
|
||||||
<url>
|
<url>
|
||||||
<loc>https://skyzh.github.io/mini-lsm/09-whats-next</loc>
|
<loc>https://skyzh.github.io/mini-lsm/09-whats-next</loc>
|
||||||
<lastmod>2024-01-25T02:56:28.265Z</lastmod>
|
<lastmod>2024-01-30T08:17:56.770Z</lastmod>
|
||||||
</url>
|
</url>
|
||||||
<url>
|
<url>
|
||||||
<loc>https://skyzh.github.io/mini-lsm/week1-01-memtable</loc>
|
<loc>https://skyzh.github.io/mini-lsm/week1-01-memtable</loc>
|
||||||
<lastmod>2024-01-25T02:56:28.237Z</lastmod>
|
<lastmod>2024-01-30T08:17:56.736Z</lastmod>
|
||||||
</url>
|
</url>
|
||||||
<url>
|
<url>
|
||||||
<loc>https://skyzh.github.io/mini-lsm/week1-02-merge-iterator</loc>
|
<loc>https://skyzh.github.io/mini-lsm/week1-02-merge-iterator</loc>
|
||||||
<lastmod>2024-01-25T02:56:28.238Z</lastmod>
|
<lastmod>2024-01-30T08:17:56.738Z</lastmod>
|
||||||
</url>
|
</url>
|
||||||
<url>
|
<url>
|
||||||
<loc>https://skyzh.github.io/mini-lsm/week1-03-block</loc>
|
<loc>https://skyzh.github.io/mini-lsm/week1-03-block</loc>
|
||||||
<lastmod>2024-01-25T02:56:28.239Z</lastmod>
|
<lastmod>2024-01-30T08:17:56.739Z</lastmod>
|
||||||
</url>
|
</url>
|
||||||
<url>
|
<url>
|
||||||
<loc>https://skyzh.github.io/mini-lsm/week1-04-sst</loc>
|
<loc>https://skyzh.github.io/mini-lsm/week1-04-sst</loc>
|
||||||
<lastmod>2024-01-25T02:56:28.240Z</lastmod>
|
<lastmod>2024-01-30T08:17:56.740Z</lastmod>
|
||||||
</url>
|
</url>
|
||||||
<url>
|
<url>
|
||||||
<loc>https://skyzh.github.io/mini-lsm/week1-05-read-path</loc>
|
<loc>https://skyzh.github.io/mini-lsm/week1-05-read-path</loc>
|
||||||
<lastmod>2024-01-25T02:56:28.242Z</lastmod>
|
<lastmod>2024-01-30T08:17:56.741Z</lastmod>
|
||||||
</url>
|
</url>
|
||||||
<url>
|
<url>
|
||||||
<loc>https://skyzh.github.io/mini-lsm/week1-06-write-path</loc>
|
<loc>https://skyzh.github.io/mini-lsm/week1-06-write-path</loc>
|
||||||
<lastmod>2024-01-25T02:56:28.243Z</lastmod>
|
<lastmod>2024-01-30T08:17:56.743Z</lastmod>
|
||||||
</url>
|
</url>
|
||||||
<url>
|
<url>
|
||||||
<loc>https://skyzh.github.io/mini-lsm/week1-07-sst-optimizations</loc>
|
<loc>https://skyzh.github.io/mini-lsm/week1-07-sst-optimizations</loc>
|
||||||
<lastmod>2024-01-25T02:56:28.244Z</lastmod>
|
<lastmod>2024-01-30T08:17:56.744Z</lastmod>
|
||||||
</url>
|
</url>
|
||||||
<url>
|
<url>
|
||||||
<loc>https://skyzh.github.io/mini-lsm/week1-overview</loc>
|
<loc>https://skyzh.github.io/mini-lsm/week1-overview</loc>
|
||||||
<lastmod>2024-01-25T02:56:28.235Z</lastmod>
|
<lastmod>2024-01-30T08:17:56.735Z</lastmod>
|
||||||
</url>
|
</url>
|
||||||
<url>
|
<url>
|
||||||
<loc>https://skyzh.github.io/mini-lsm/week2-01-compaction</loc>
|
<loc>https://skyzh.github.io/mini-lsm/week2-01-compaction</loc>
|
||||||
<lastmod>2024-01-25T02:56:28.246Z</lastmod>
|
<lastmod>2024-01-30T08:17:56.746Z</lastmod>
|
||||||
</url>
|
</url>
|
||||||
<url>
|
<url>
|
||||||
<loc>https://skyzh.github.io/mini-lsm/week2-02-simple</loc>
|
<loc>https://skyzh.github.io/mini-lsm/week2-02-simple</loc>
|
||||||
<lastmod>2024-01-25T02:56:28.248Z</lastmod>
|
<lastmod>2024-01-30T08:17:56.747Z</lastmod>
|
||||||
</url>
|
</url>
|
||||||
<url>
|
<url>
|
||||||
<loc>https://skyzh.github.io/mini-lsm/week2-03-tiered</loc>
|
<loc>https://skyzh.github.io/mini-lsm/week2-03-tiered</loc>
|
||||||
<lastmod>2024-01-25T02:56:28.249Z</lastmod>
|
<lastmod>2024-01-30T08:17:56.748Z</lastmod>
|
||||||
</url>
|
</url>
|
||||||
<url>
|
<url>
|
||||||
<loc>https://skyzh.github.io/mini-lsm/week2-04-leveled</loc>
|
<loc>https://skyzh.github.io/mini-lsm/week2-04-leveled</loc>
|
||||||
<lastmod>2024-01-25T02:56:28.250Z</lastmod>
|
<lastmod>2024-01-30T08:17:56.749Z</lastmod>
|
||||||
</url>
|
</url>
|
||||||
<url>
|
<url>
|
||||||
<loc>https://skyzh.github.io/mini-lsm/week2-05-manifest</loc>
|
<loc>https://skyzh.github.io/mini-lsm/week2-05-manifest</loc>
|
||||||
<lastmod>2024-01-25T02:56:28.251Z</lastmod>
|
<lastmod>2024-01-30T08:17:56.750Z</lastmod>
|
||||||
</url>
|
</url>
|
||||||
<url>
|
<url>
|
||||||
<loc>https://skyzh.github.io/mini-lsm/week2-06-wal</loc>
|
<loc>https://skyzh.github.io/mini-lsm/week2-06-wal</loc>
|
||||||
<lastmod>2024-01-25T02:56:28.252Z</lastmod>
|
<lastmod>2024-01-30T08:17:56.751Z</lastmod>
|
||||||
</url>
|
</url>
|
||||||
<url>
|
<url>
|
||||||
<loc>https://skyzh.github.io/mini-lsm/week2-07-snacks</loc>
|
<loc>https://skyzh.github.io/mini-lsm/week2-07-snacks</loc>
|
||||||
<lastmod>2024-01-25T02:56:28.253Z</lastmod>
|
<lastmod>2024-01-30T08:17:56.752Z</lastmod>
|
||||||
</url>
|
</url>
|
||||||
<url>
|
<url>
|
||||||
<loc>https://skyzh.github.io/mini-lsm/week2-overview</loc>
|
<loc>https://skyzh.github.io/mini-lsm/week2-overview</loc>
|
||||||
<lastmod>2024-01-25T02:56:28.245Z</lastmod>
|
<lastmod>2024-01-30T08:17:56.745Z</lastmod>
|
||||||
|
</url>
|
||||||
|
<url>
|
||||||
|
<loc>https://skyzh.github.io/mini-lsm/week3-01-ts-key-refactor</loc>
|
||||||
|
<lastmod>2024-01-30T08:17:56.754Z</lastmod>
|
||||||
|
</url>
|
||||||
|
<url>
|
||||||
|
<loc>https://skyzh.github.io/mini-lsm/week3-02-snapshot-read-part-1</loc>
|
||||||
|
<lastmod>2024-01-30T08:17:56.755Z</lastmod>
|
||||||
|
</url>
|
||||||
|
<url>
|
||||||
|
<loc>https://skyzh.github.io/mini-lsm/week3-03-snapshot-read-part-2</loc>
|
||||||
|
<lastmod>2024-01-30T08:17:56.756Z</lastmod>
|
||||||
|
</url>
|
||||||
|
<url>
|
||||||
|
<loc>https://skyzh.github.io/mini-lsm/week3-04-watermark</loc>
|
||||||
|
<lastmod>2024-01-30T08:17:56.757Z</lastmod>
|
||||||
|
</url>
|
||||||
|
<url>
|
||||||
|
<loc>https://skyzh.github.io/mini-lsm/week3-05-txn-occ</loc>
|
||||||
|
<lastmod>2024-01-30T08:17:56.758Z</lastmod>
|
||||||
|
</url>
|
||||||
|
<url>
|
||||||
|
<loc>https://skyzh.github.io/mini-lsm/week3-06-serializable</loc>
|
||||||
|
<lastmod>2024-01-30T08:17:56.759Z</lastmod>
|
||||||
|
</url>
|
||||||
|
<url>
|
||||||
|
<loc>https://skyzh.github.io/mini-lsm/week3-07-compaction-filter</loc>
|
||||||
|
<lastmod>2024-01-30T08:17:56.760Z</lastmod>
|
||||||
</url>
|
</url>
|
||||||
<url>
|
<url>
|
||||||
<loc>https://skyzh.github.io/mini-lsm/week3-overview</loc>
|
<loc>https://skyzh.github.io/mini-lsm/week3-overview</loc>
|
||||||
<lastmod>2024-01-25T02:56:28.254Z</lastmod>
|
<lastmod>2024-01-30T08:17:56.753Z</lastmod>
|
||||||
</url>
|
</url>
|
||||||
<url>
|
<url>
|
||||||
<loc>https://skyzh.github.io/mini-lsm/week4-overview</loc>
|
<loc>https://skyzh.github.io/mini-lsm/week4-overview</loc>
|
||||||
<lastmod>2024-01-25T02:56:28.255Z</lastmod>
|
<lastmod>2024-01-30T08:17:56.761Z</lastmod>
|
||||||
</url>
|
</url>
|
||||||
</urlset>
|
</urlset>
|
||||||
|
|||||||
Reference in New Issue
Block a user