add preface

Signed-off-by: Alex Chi <iskyzh@gmail.com>
This commit is contained in:
Alex Chi
2024-01-20 19:27:36 +08:00
parent 10ff2c3970
commit ab5c3ec38c
22 changed files with 469 additions and 151 deletions

View File

@@ -1,4 +1,4 @@
# Get Started # Environment Setup
The starter code and reference solution is available at [https://github.com/skyzh/mini-lsm](https://github.com/skyzh/mini-lsm). The starter code and reference solution is available at [https://github.com/skyzh/mini-lsm](https://github.com/skyzh/mini-lsm).

View File

@@ -1,44 +1,4 @@
# Overview # Mini-LSM Overview
<!-- toc -->
In this tutorial, you will learn how to build a simple LSM-Tree storage engine in the Rust programming language.
## What is LSM, and Why LSM?
Log-structured merge tree is a data structure to maintain key-value pairs. This data structure is widely used in
distributed database systems like [TiDB](https://www.pingcap.com) and [CockroachDB](https://www.cockroachlabs.com) as
their underlying storage engine. [RocksDB](http://rocksdb.org), based on [LevelDB](https://github.com/google/leveldb),
is an implementation of LSM-Tree storage engine. It provides a wide range of key-value access functionalities and is
used in a lot of production systems.
Generally speaking, LSM Tree is an append-friendly data structure. It is more intuitive to compare LSM to other
key-value data structure like RB-Tree and B-Tree. For RB-Tree and B-Tree, all data operations are in-place. That is to
say, when you update the value corresponding to the key, the value will be overwritten at its original memory or disk
space. But in an LSM Tree, all write operations, i.e., insertions, updates, deletions, are performed in somewhere else.
These operations will be batched into SST (sorted string table) files and be written to the disk. Once written to the
disk, the file will not be changed. These operations are applied lazily on disk with a special task called compaction.
The compaction job will merge multiple SST files and remove unused data.
This architectural design makes LSM tree easy to work with.
1. Data are immutable on persistent storage, which means that it is easier to offload the background tasks (compaction)
to remote servers. It is also feasible to directly store and serve data from cloud-native storage systems like S3.
2. An LSM tree can balance between read, write and space amplification by changing the compaction algorithm. The data
structure itself is super versatile and can be optimized for different workloads.
In this tutorial, we will learn how to build an LSM-Tree-based storage engine in the Rust programming language.
## Prerequisites of this Tutorial
* You should know the basics of the Rust programming language. Reading [the Rust book](https://doc.rust-lang.org/book/)
is enough.
* You should know the basic concepts of key-value storage engines, i.e., why we need somehow complex design to achieve
persistence. If you have no experience with database systems and storage systems before, you can implement Bitcask
in [PingCAP Talent Plan](https://github.com/pingcap/talent-plan/tree/master/courses/rust/projects/project-2).
* Knowing the basics of an LSM tree is not a requirement but we recommend you to read something about it, e.g., the
overall idea of LevelDB. This would familiarize you with concepts like mutable and immutable mem-tables, SST,
compaction, WAL, etc.
## Overview of LSM ## Overview of LSM
@@ -65,95 +25,34 @@ of key value pairs.
In this tutorial, we assume the LSM tree is using leveled compaction algorithm, which is commonly used in real-world In this tutorial, we assume the LSM tree is using leveled compaction algorithm, which is commonly used in real-world
systems. systems.
## Write Flow ### Write Path
![Write Flow](lsm-tutorial/00-lsm-write-flow.svg) ![Write Path](lsm-tutorial/00-lsm-write-flow.svg)
The write flow of LSM contains 4 steps: The write path of LSM contains 4 steps:
1. Write the key-value pair to write-ahead log, so that it can be recovered after the storage engine crashes. 1. Write the key-value pair to write-ahead log, so that it can be recovered after the storage engine crashes.
2. Write the key-value pair to memtable. After (1) and (2) completes, we can notify the user that the write operation 2. Write the key-value pair to memtable. After (1) and (2) completes, we can notify the user that the write operation
is completed. is completed.
3. When a memtable is full, we will flush it to the disk as an SST file in the background. 3. When a memtable is full, we will freeze them into immutable memtables, and will flush them to the disk as SST files in the background.
4. We will compact some files in some level into lower levels to maintain a good shape for the LSM tree, so that read 4. We will compact some files in some level into lower levels to maintain a good shape for the LSM tree, so that read
amplification is low. amplification is low.
## Read Flow ### Read Path
![Read Flow](lsm-tutorial/00-lsm-read-flow.svg) ![Read Path](lsm-tutorial/00-lsm-read-flow.svg)
When we want to read a key, When we want to read a key,
1. We will first probe all the memtables from latest to oldest. 1. We will first probe all the memtables from latest to oldest.
2. If the key is not found, we will then search the entire LSM tree containing SSTs to find the data. 2. If the key is not found, we will then search the entire LSM tree containing SSTs to find the data.
## Community There are two types of read: lookup and scan. Lookup finds one key in the LSM tree, while scan iterates all keys within a range in the storage engine. We will cover both of them throughout the tutorial.
You may join skyzh's Discord server and study with the mini-lsm community. ## Tutorial Structure
[![Join skyzh's Discord Server](https://dcbadge.vercel.app/api/server/ZgXzxpua3H)](https://skyzh.dev/join/discord) ![Tutorial Overview](lsm-tutorial/00-full-overview.svg)
## About the Author We have 3 parts (weeks) for this tutorial. In the first week, we will focus on the storage structure and the storage format of an LSM storage engine. In the second week, we will dive into compactions in depth and implement persistence support for the storage engine. In the third week, we will implement multi-version concurrency control.
As of writing (at the end of 2022), Chi is a first-year master's student in Carnegie Mellon University. He has 5 years'
experience with the Rust programming language since 2018. He has been working on a variety of database systems including
[TiKV][db1], [AgateDB][db2], [TerarkDB][db3], [RisingLight][db4], and [RisingWave][db5]. In his first semester in CMU,
he worked as a teaching assistant for CMU's [15-445/645 Intro to Database Systems][15445-course] course, where he built
a new SQL processing layer for the [BusTub][bustub] educational database system, added more query optimization stuff into
the course, and made the course [more challenging than ever before][tweet]. Chi is interested in exploring how the Rust
programming language can fit in the database world. Check out his [previous tutorial][type-exercise] on building a
vectorized expression framework if you are also interested in that topic.
[db1]: https://github.com/tikv/tikv
[db2]: https://github.com/tikv/agatedb
[db3]: https://github.com/bytedance/terarkdb
[db4]: https://github.com/risinglightdb/risinglight
[db5]: https://github.com/risingwavelabs/risingwave
[15445-course]: https://15445.courses.cs.cmu.edu/fall2022/
[tweet]: https://twitter.com/andy_pavlo/status/1598137241016360961
[type-exercise]: https://github.com/skyzh/type-exercise-in-rust
[bustub]: https://github.com/cmu-db/bustub
<!--
## Structure
chapters + snacks, clear goal
implement, think, try by yourself
required tasks, check your understanding questions, bonus tasks
## Testing
exploring and understanding is more important than passing all the test cases
testing basic requirements, not the internal structure or something
## Solution
### Checkpoints
the final version, but many things can be simplified, read the docs
comments / tests / not up-to-date with the starter code
### How to use the solutions
## Feedbacks
join the Discord server, your feedback is important, thank GitHub users
## License
### Free forever?
### Video lectures + Review Service + Office Hour?
should have a separate preface (before you start) chapter? and what's new with v2?
## Target audience?
## What will you get after taking this course...
-->
{{#include copyright.md}} {{#include copyright.md}}

View File

@@ -0,0 +1,98 @@
# Preface
![Tutorial Overview](lsm-tutorial/00-full-overview.svg)
In this tutorial, you will learn how to build a simple LSM-Tree storage engine in the Rust programming language.
## What is LSM, and Why LSM?
Log-structured merge tree is a data structure to maintain key-value pairs. This data structure is widely used in
distributed database systems like [TiDB](https://www.pingcap.com) and [CockroachDB](https://www.cockroachlabs.com) as
their underlying storage engine. [RocksDB](http://rocksdb.org), based on [LevelDB](https://github.com/google/leveldb),
is an implementation of LSM-Tree storage engine. It provides a wide range of key-value access functionalities and is
used in a lot of production systems.
Generally speaking, LSM Tree is an append-friendly data structure. It is more intuitive to compare LSM to other
key-value data structure like RB-Tree and B-Tree. For RB-Tree and B-Tree, all data operations are in-place. That is to
say, when you update the value corresponding to the key, the value will be overwritten at its original memory or disk
space. But in an LSM Tree, all write operations, i.e., insertions, updates, deletions, are performed in somewhere else.
These operations will be batched into SST (sorted string table) files and be written to the disk. Once written to the
disk, the file will not be changed. These operations are applied lazily on disk with a special task called compaction.
The compaction job will merge multiple SST files and remove unused data.
This architectural design makes LSM tree easy to work with.
1. Data are immutable on persistent storage, which means that it is easier to offload the background tasks (compaction)
to remote servers. It is also feasible to directly store and serve data from cloud-native storage systems like S3.
2. An LSM tree can balance between read, write and space amplification by changing the compaction algorithm. The data
structure itself is super versatile and can be optimized for different workloads.
In this tutorial, we will learn how to build an LSM-Tree-based storage engine in the Rust programming language.
## Prerequisites
* You should know the basics of the Rust programming language. Reading [the Rust book](https://doc.rust-lang.org/book/)
is enough.
* You should know the basic concepts of key-value storage engines, i.e., why we need somehow complex design to achieve
persistence. If you have no experience with database systems and storage systems before, you can implement Bitcask
in [PingCAP Talent Plan](https://github.com/pingcap/talent-plan/tree/master/courses/rust/projects/project-2).
* Knowing the basics of an LSM tree is not a requirement but we recommend you to read something about it, e.g., the
overall idea of LevelDB. This would familiarize you with concepts like mutable and immutable mem-tables, SST,
compaction, WAL, etc.
## What should you expect from this tutorial...
After learning this course, you should have a deep understanding of how a LSM-based storage system works, gain hands-on experience of designing such systems, and apply what you have learned in your study and career. You will become an expert of LSM storage systems, understand the design tradeoffs in such storage systems, and find optimal ways to design a LSM-based storage system to meet your workload requirements/goals. This is a very in-depth tutorial that covers all the important implementation details and design choices of modern storage systems (i.e., RocksDB) based on the author's experience in several LSM-like storage systems, and you will be able to directly apply what you have learned in both industry and academia.
### Structure
The tutorial is a large course that is split into several parts (weeks). Each week usually has seven chapters, and each of the chapter can be finished within 2-3 hours. The first six chapters of each part will instruct you to build a working system, and the last chapter of each week will be a *snack time* chapter that implements some easy things over what you have built in the previous six days. In each chapter, there will be required tasks, *check you understanding* questions, and bonus tasks.
### Testing
We provide full test suite and some cli tools for you to validate if your solution is correct. Note that the test suite is not exhaustive, and your solution might not be 100% correct after passing all test cases. You might need to fix earlier bugs when implementing later parts of the system. We recommend you to think thoroughly about your implementation, especially when there are multi-thread operations and race conditions.
### Solution
We have a solution that implements all the functionalities as required in the tutorial in the mini-lsm main repo. At the same time, we also have a mini-lsm solution checkpoint repo where each commit corresponds to a chapter in the tutorial.
Keeping such checkpoint repo up-to-date to the mini-lsm tutorial is hard because each bug fix or new feature will need to go through all commits (or checkpoints). Therefore, this repo might not be using the latest starter code or incorporating the latest features from the mini-lsm tutorial.
**TL;DR: We do not guarantee the solution checkpoint repo contains a correct solution, passes all tests, or has the correct doc comments.** For a correct implementation and the solution after implementing all things, please take a look at the solution in the main repo instead. [https://github.com/skyzh/mini-lsm/tree/main/mini-lsm](https://github.com/skyzh/mini-lsm/tree/main/mini-lsm).
If you are stuck at some part of the tutorial or do not know where to implement a functionality, you can refer to this repo for help. You may compare the diff between commits to know what has been changed. Some functions in the mini-lsm tutorial might be changed multiple times throughout the chapters, and you can know what exactly are expected to be implemented for each chapter in this repo.
You may access the solution checkpoint repo at [https://github.com/skyzh/mini-lsm-solution-checkpoint](https://github.com/skyzh/mini-lsm-solution-checkpoint).
### Feedbacks
Your feedback is greatly appreciated. We have rewritten the whole course from scratch in 2024 based on the feedbacks from the students. We hope you can share your learning experience and help us continuously improve the learning experience. Please join the [Discord community](https://skyzh.dev/join/discord) to share your experience.
The tutorial was originally planned as a general guidance that students start from an empty directory and implement whatever they want based on the specification we had. We had minimal tests that checks if the behavior is correct. However, the original tutorial is too open-ended that caused huge obstacles with the learning experience. As students do not have an overview of the whole system beforehand and the instructions are kind of vague, sometimes it is hard for the students to know why a design decision is made and what they need to achieve a goal. And some part of the course is too compact that it is impossible to deliver expected contents within just one chapter. Therefore, we completely redesigned the course to have a easier learning curve and clearer learning goals. The original one-week tutorial is now split into two weeks (first week on storage format, and second week on deep-dive compaction), with an extra part on MVCC. We hope you find this course interesting and helpful in your study and career. We would like to thank everyone who commented in [Feedback after coding day 1](https://github.com/skyzh/mini-lsm/issues/11) and [Hello, when is the next update plan for the tutorial?](https://github.com/skyzh/mini-lsm/issues/7) -- your feedback greatly helped us improve the course.
### License
The source code of this course is licensed under Apache 2.0, while the author owns the full copyright of the tutorial itself (markdown files + figures).
### Will this tutorial be free forever?
Yes! Everything publicly available now will be free forever and will receive lifetime updates and bug fixes. Meanwhile, we might provide paid code review and office hour services in the future. For the DLC part (*rest of your life* chapters), we do not have plans to finish them as of 2024, and have not decided whether they will be public available or not.
## Community
You may join skyzh's Discord server and study with the mini-lsm community.
[![Join skyzh's Discord Server](https://dcbadge.vercel.app/api/server/ZgXzxpua3H)](https://skyzh.dev/join/discord)
## About the Author
As of writing (at the beginning of 2024), Chi obtained his master's degree in Computer Science from Carnegie Mellon University and his bachelor's degree from Shanghai Jiao Tong University. He has been working on a variety of database systems including [TiKV][db1], [AgateDB][db2], [TerarkDB][db3], [RisingWave][db4], and [Neon][db5]. Since 2022, he worked as a teaching assistant for [CMU's Database Systems course](https://15445.courses.cs.cmu) for three semesters on the BusTub educational system, where he added a lot of new features and more challenges to the course (check out the re-designed [query execution](https://15445.courses.cs.cmu.edu/fall2022/project3/) project and the super challenging [multi-version concurrency control](https://15445.courses.cs.cmu.edu/fall2023/project4/) project). Besides working on the BusTub educational system, he is also a maintainer of the [RisingLight](https://github.com/risinglightdb/risinglight) educational database system. Chi is interested in exploring how the Rust programming language can fit in the database world. Check out his previous tutorial on building a vectorized expression framework [type-exercise-in-rust](https://github.com/skyzh/type-exercise-in-rust) and on building a vector database [write-you-a-vector-db](https://github.com/skyzh/write-you-a-vector-db) if you are also interested in that topic.
[db1]: https://github.com/tikv/tikv
[db2]: https://github.com/tikv/agatedb
[db3]: https://github.com/bytedance/terarkdb
[db4]: https://github.com/risingwavelabs/risingwave
[db5]: https://github.com/neondatabase/neon
{{#include copyright.md}}

View File

@@ -1,7 +1,8 @@
# LSM in a Week # LSM in a Week
[Overview](./00-overview.md) [Preface](./00-preface.md)
[Get Started](./00-get-started.md) [Mini-LSM Overview](./00-overview.md)
[Environment Setup](./00-get-started.md)
--- ---
@@ -22,7 +23,7 @@
# Mini-LSM v2 # Mini-LSM v2
- [Week 1: Mini-LSM](./week1-overview.md) - [Week 1 Overview: Mini-LSM](./week1-overview.md)
- [Memtable](./week1-01-memtable.md) - [Memtable](./week1-01-memtable.md)
- [Merge Iterator](./week1-02-merge-iterator.md) - [Merge Iterator](./week1-02-merge-iterator.md)
- [Block](./week1-03-block.md) - [Block](./week1-03-block.md)
@@ -31,7 +32,7 @@
- [Write Path](./week1-06-write-path.md) - [Write Path](./week1-06-write-path.md)
- [Snack Time: SST Optimizations](./week1-07-sst-optimizations.md) - [Snack Time: SST Optimizations](./week1-07-sst-optimizations.md)
- [Week 2: Compaction and Persistence](./week2-overview.md) - [Week 2 Overview: Compaction and Persistence](./week2-overview.md)
- [Compaction Implementation](./week2-01-compaction.md) - [Compaction Implementation](./week2-01-compaction.md)
- [Simple Compaction Strategy](./week2-02-simple.md) - [Simple Compaction Strategy](./week2-02-simple.md)
- [Tiered Compaction Strategy](./week2-03-tiered.md) - [Tiered Compaction Strategy](./week2-03-tiered.md)
@@ -40,7 +41,7 @@
- [Write-Ahead Log (WAL)](./week2-06-wal.md) - [Write-Ahead Log (WAL)](./week2-06-wal.md)
- [Snack Time: Batch Write and Checksums](./week2-07-snacks.md) - [Snack Time: Batch Write and Checksums](./week2-07-snacks.md)
- [Week 3: MVCC](./week3-overview.md) - [Week 3 Overview: MVCC](./week3-overview.md)
# The Rest of Your Life (TBD) # The Rest of Your Life (TBD)

View File

@@ -1 +1 @@
<p style="text-color: grey; text-align: center; margin-top: 3em"><small>Copyright © 2022 - 2024 Alex Chi Z. All Rights Reserved.</small></p> <p style="text-align: center; margin-top: 3em"><small>Your feedback is greatly appreciated. Welcome to join our <a href="https://skyzh.dev/join/discord">Discord Community</a>.<br>Copyright © 2022 - 2024 Alex Chi Z. All Rights Reserved.</small></p>

View File

@@ -0,0 +1,308 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<svg xmlns="http://www.w3.org/2000/svg" version="1.1" xmlns:xl="http://www.w3.org/1999/xlink" xmlns:dc="http://purl.org/dc/elements/1.1/" viewBox="-5 170 704 599" width="704" height="599">
<defs>
<marker orient="auto" overflow="visible" markerUnits="strokeWidth" id="FilledArrow_Marker" stroke-linejoin="miter" stroke-miterlimit="10" viewBox="-1 -4 10 8" markerWidth="10" markerHeight="8" color="black">
<g>
<path d="M 8 0 L 0 -3 L 0 3 Z" fill="currentColor" stroke="currentColor" stroke-width="1"/>
</g>
</marker>
</defs>
<g id="00-full-overview" fill-opacity="1" stroke-dasharray="none" fill="none" stroke-opacity="1" stroke="none">
<title>00-full-overview</title>
<rect fill="white" x="-5" y="170" width="704" height="599"/>
<g id="00-full-overview_Layer_1">
<title>Layer 1</title>
<g id="Graphic_73">
<rect x="80.01552" y="343.5" width="553.4845" height="181.45023" fill="white"/>
<path d="M 80.01552 343.5 L 633.5 343.5 L 633.5 524.9502 L 80.01552 524.9502 Z" stroke="gray" stroke-linecap="round" stroke-linejoin="round" stroke-dasharray="4.0,4.0" stroke-width="1"/>
</g>
<g id="Graphic_67">
<rect x="117.31032" y="171.12778" width="363.6897" height="233.39822" fill="white"/>
<path d="M 117.31032 171.12778 L 481 171.12778 L 481 404.526 L 117.31032 404.526 Z" stroke="gray" stroke-linecap="round" stroke-linejoin="round" stroke-dasharray="4.0,4.0" stroke-width="1"/>
</g>
<g id="Graphic_48">
<rect x="-3.6741593" y="553.5" width="541" height="214.51611" fill="white"/>
<path d="M -3.6741593 553.5 L 537.32584 553.5 L 537.32584 768.0161 L -3.6741593 768.0161 Z" stroke="gray" stroke-linecap="round" stroke-linejoin="round" stroke-dasharray="4.0,4.0" stroke-width="1"/>
</g>
<g id="Graphic_47">
<rect x="128.292" y="358.5" width="62.81032" height="41.5" fill="white"/>
<rect x="128.292" y="358.5" width="62.81032" height="41.5" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
<text transform="translate(133.292 370.026)" fill="black">
<tspan font-family="Helvetica Neue" font-size="16" fill="black" x="11.445159" y="15">SST</tspan>
</text>
</g>
<g id="Graphic_46">
<text transform="translate(90 370.026)" fill="black">
<tspan font-family="Helvetica Neue" font-size="16" fill="black" x="6394885e-19" y="15">L0</tspan>
</text>
</g>
<g id="Graphic_45">
<rect x="128.5" y="410.026" width="62.81032" height="41.5" fill="white"/>
<rect x="128.5" y="410.026" width="62.81032" height="41.5" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
<text transform="translate(133.5 421.552)" fill="black">
<tspan font-family="Helvetica Neue" font-size="16" fill="black" x="11.445159" y="15">SST</tspan>
</text>
</g>
<g id="Graphic_44">
<rect x="201.09484" y="410.026" width="62.81032" height="41.5" fill="white"/>
<rect x="201.09484" y="410.026" width="62.81032" height="41.5" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
<text transform="translate(206.09484 421.552)" fill="black">
<tspan font-family="Helvetica Neue" font-size="16" fill="black" x="11.445159" y="15">SST</tspan>
</text>
</g>
<g id="Graphic_43">
<text transform="translate(90.208 421.552)" fill="black">
<tspan font-family="Helvetica Neue" font-size="16" fill="black" x="6394885e-19" y="15">L1</tspan>
</text>
</g>
<g id="Graphic_42">
<rect x="200.88684" y="461.552" width="62.81032" height="41.5" fill="white"/>
<rect x="200.88684" y="461.552" width="62.81032" height="41.5" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
<text transform="translate(205.88684 473.078)" fill="black">
<tspan font-family="Helvetica Neue" font-size="16" fill="black" x="11.445159" y="15">SST</tspan>
</text>
</g>
<g id="Graphic_41">
<rect x="273.48168" y="461.552" width="62.81032" height="41.5" fill="white"/>
<rect x="273.48168" y="461.552" width="62.81032" height="41.5" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
<text transform="translate(278.48168 473.078)" fill="black">
<tspan font-family="Helvetica Neue" font-size="16" fill="black" x="11.445159" y="15">SST</tspan>
</text>
</g>
<g id="Graphic_40">
<text transform="translate(90 473.078)" fill="black">
<tspan font-family="Helvetica Neue" font-size="16" fill="black" x="6394885e-19" y="15">Ln</tspan>
</text>
</g>
<g id="Graphic_39">
<rect x="128.292" y="461.552" width="62.81032" height="41.5" fill="white"/>
<rect x="128.292" y="461.552" width="62.81032" height="41.5" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
<text transform="translate(133.292 473.078)" fill="black">
<tspan font-family="Helvetica Neue" font-size="16" fill="black" x="11.445159" y="15">SST</tspan>
</text>
</g>
<g id="Graphic_38">
<rect x="346.07652" y="461.552" width="62.81032" height="41.5" fill="white"/>
<rect x="346.07652" y="461.552" width="62.81032" height="41.5" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
<text transform="translate(351.07652 473.078)" fill="black">
<tspan font-family="Helvetica Neue" font-size="16" fill="black" x="11.445159" y="15">SST</tspan>
</text>
</g>
<g id="Line_37">
<line x1="7" y1="319" x2="697.696" y2="319" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-dasharray="4.0,4.0" stroke-width="1"/>
</g>
<g id="Graphic_36">
<text transform="translate(12 325)" fill="black">
<tspan font-family="Helvetica Neue" font-size="16" fill="black" x="0" y="15">On Disk</tspan>
</text>
</g>
<g id="Graphic_35">
<text transform="translate(12 296.552)" fill="black">
<tspan font-family="Helvetica Neue" font-size="16" fill="black" x="0" y="15">In Memory</tspan>
</text>
</g>
<g id="Graphic_34">
<rect x="128.5" y="245.974" width="62.81032" height="41.5" fill="white"/>
<rect x="128.5" y="245.974" width="62.81032" height="41.5" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
<text transform="translate(133.5 248.276)" fill="black">
<tspan font-family="Helvetica Neue" font-size="16" fill="black" x="8.317159" y="15">Mem </tspan>
<tspan font-family="Helvetica Neue" font-size="16" fill="black" x="7.589159" y="33.448">Table</tspan>
</text>
</g>
<g id="Graphic_33">
<rect x="201.09484" y="245.974" width="62.81032" height="41.5" fill="white"/>
<rect x="201.09484" y="245.974" width="62.81032" height="41.5" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
<text transform="translate(206.09484 248.276)" fill="black">
<tspan font-family="Helvetica Neue" font-size="16" fill="black" x="8.317159" y="15">Mem </tspan>
<tspan font-family="Helvetica Neue" font-size="16" fill="black" x="7.589159" y="33.448">Table</tspan>
</text>
</g>
<g id="Graphic_32">
<rect x="273.68968" y="245.974" width="62.81032" height="41.5" fill="white"/>
<rect x="273.68968" y="245.974" width="62.81032" height="41.5" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
<text transform="translate(278.68968 248.276)" fill="black">
<tspan font-family="Helvetica Neue" font-size="16" fill="black" x="8.317159" y="15">Mem </tspan>
<tspan font-family="Helvetica Neue" font-size="16" fill="black" x="7.589159" y="33.448">Table</tspan>
</text>
</g>
<g id="Graphic_31">
<rect x="530.95384" y="365.026" width="62.81032" height="41.5" fill="white"/>
<rect x="530.95384" y="365.026" width="62.81032" height="41.5" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
<text transform="translate(535.95384 376.552)" fill="black">
<tspan font-family="Helvetica Neue" font-size="16" fill="black" x="9.557159" y="15">WAL</tspan>
</text>
</g>
<g id="Graphic_30">
<rect x="500.31726" y="252.052" width="124.08348" height="28.447998" fill="white"/>
<rect x="500.31726" y="252.052" width="124.08348" height="28.447998" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
<text transform="translate(505.31726 257.052)" fill="black">
<tspan font-family="Helvetica Neue" font-size="16" fill="black" x="16.52974" y="15">key + value</tspan>
</text>
</g>
<g id="Line_29">
<line x1="562.359" y1="280.5" x2="562.359" y2="355.126" marker-end="url(#FilledArrow_Marker)" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
</g>
<g id="Line_28">
<line x1="159.8668" y1="287.474" x2="159.75381" y2="348.6" marker-end="url(#FilledArrow_Marker)" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
</g>
<g id="Line_27">
<line x1="500.31726" y1="266.38404" x2="346.4" y2="266.65207" marker-end="url(#FilledArrow_Marker)" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
</g>
<g id="Line_23">
<path d="M 436.92997 383.00897 C 436.92997 383.00897 456.93 404.82856 456.93 434.00897 C 456.93 452.75967 448.67184 468.88394 442.76857 478.01486" marker-end="url(#FilledArrow_Marker)" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
</g>
<g id="Graphic_21">
<rect x="80.01552" y="571.13" width="62.81032" height="41.5" fill="white"/>
<rect x="80.01552" y="571.13" width="62.81032" height="41.5" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
<text transform="translate(85.01552 573.432)" fill="black">
<tspan font-family="Helvetica Neue" font-size="16" fill="black" x="9.661159" y="15">Data </tspan>
<tspan font-family="Helvetica Neue" font-size="16" fill="black" x="6.109159" y="33.448">Block</tspan>
</text>
</g>
<g id="Graphic_20">
<rect x="142.82584" y="571.13" width="62.81032" height="41.5" fill="white"/>
<rect x="142.82584" y="571.13" width="62.81032" height="41.5" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
<text transform="translate(147.82584 573.432)" fill="black">
<tspan font-family="Helvetica Neue" font-size="16" fill="black" x="9.661159" y="15">Data </tspan>
<tspan font-family="Helvetica Neue" font-size="16" fill="black" x="6.109159" y="33.448">Block</tspan>
</text>
</g>
<g id="Graphic_19">
<rect x="204.82584" y="571.13" width="62.81032" height="41.5" fill="white"/>
<rect x="204.82584" y="571.13" width="62.81032" height="41.5" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
<text transform="translate(209.82584 573.432)" fill="black">
<tspan font-family="Helvetica Neue" font-size="16" fill="black" x="9.661159" y="15">Data </tspan>
<tspan font-family="Helvetica Neue" font-size="16" fill="black" x="6.109159" y="33.448">Block</tspan>
</text>
</g>
<g id="Graphic_18">
<rect x="266.82584" y="571.13" width="62.81032" height="41.5" fill="white"/>
<rect x="266.82584" y="571.13" width="62.81032" height="41.5" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
<text transform="translate(271.82584 573.432)" fill="black">
<tspan font-family="Helvetica Neue" font-size="16" fill="black" x="9.661159" y="15">Data </tspan>
<tspan font-family="Helvetica Neue" font-size="16" fill="black" x="6.109159" y="33.448">Block</tspan>
</text>
</g>
<g id="Graphic_17">
<rect x="328.82584" y="571.13" width="62.81032" height="41.5" fill="white"/>
<rect x="328.82584" y="571.13" width="62.81032" height="41.5" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
<text transform="translate(333.82584 573.432)" fill="black">
<tspan font-family="Helvetica Neue" font-size="16" fill="black" x="9.661159" y="15">Data </tspan>
<tspan font-family="Helvetica Neue" font-size="16" fill="black" x="6.109159" y="33.448">Block</tspan>
</text>
</g>
<g id="Graphic_16">
<rect x="390.82584" y="571.13" width="62.81032" height="41.5" fill="white"/>
<rect x="390.82584" y="571.13" width="62.81032" height="41.5" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
<text transform="translate(395.82584 582.656)" fill="black">
<tspan font-family="Helvetica Neue" font-size="16" fill="black" x="6.701159" y="15">Index</tspan>
</text>
</g>
<g id="Line_15">
<line x1="128.292" y1="503.052" x2="80.01552" y2="571.13" stroke="#7f8080" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
</g>
<g id="Line_14">
<line x1="191.10232" y1="503.052" x2="453.63616" y2="571.13" stroke="#7f8080" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
</g>
<g id="Graphic_13">
<rect x="7" y="697.76" width="44.48448" height="26.391998" fill="white"/>
<rect x="7" y="697.76" width="44.48448" height="26.391998" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
<text transform="translate(12 701.732)" fill="black">
<tspan font-family="Helvetica Neue" font-size="16" fill="black" x="3.610239" y="15">Key</tspan>
</text>
</g>
<g id="Graphic_12">
<rect x="51.48448" y="697.76" width="62.81032" height="26.391998" fill="white"/>
<rect x="51.48448" y="697.76" width="62.81032" height="26.391998" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
<text transform="translate(56.48448 701.732)" fill="black">
<tspan font-family="Helvetica Neue" font-size="16" fill="black" x="7.141159" y="15">Value</tspan>
</text>
</g>
<g id="Graphic_11">
<rect x="109.56808" y="697.76" width="44.48448" height="26.391998" fill="white"/>
<rect x="109.56808" y="697.76" width="44.48448" height="26.391998" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
<text transform="translate(114.56808 701.732)" fill="black">
<tspan font-family="Helvetica Neue" font-size="16" fill="black" x="3.610239" y="15">Key</tspan>
</text>
</g>
<g id="Graphic_10">
<rect x="154.05256" y="697.76" width="62.81032" height="26.391998" fill="white"/>
<rect x="154.05256" y="697.76" width="62.81032" height="26.391998" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
<text transform="translate(159.05256 701.732)" fill="black">
<tspan font-family="Helvetica Neue" font-size="16" fill="black" x="7.141159" y="15">Value</tspan>
</text>
</g>
<g id="Graphic_9">
<rect x="211.56808" y="697.76" width="44.48448" height="26.391998" fill="white"/>
<rect x="211.56808" y="697.76" width="44.48448" height="26.391998" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
<text transform="translate(216.56808 701.732)" fill="black">
<tspan font-family="Helvetica Neue" font-size="16" fill="black" x="3.610239" y="15">Key</tspan>
</text>
</g>
<g id="Graphic_8">
<rect x="256.05256" y="697.76" width="62.81032" height="26.391998" fill="white"/>
<rect x="256.05256" y="697.76" width="62.81032" height="26.391998" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
<text transform="translate(261.05256 701.732)" fill="black">
<tspan font-family="Helvetica Neue" font-size="16" fill="black" x="7.141159" y="15">Value</tspan>
</text>
</g>
<g id="Graphic_7">
<rect x="313.56808" y="697.76" width="44.48448" height="26.391998" fill="white"/>
<rect x="313.56808" y="697.76" width="44.48448" height="26.391998" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
<text transform="translate(318.56808 701.732)" fill="black">
<tspan font-family="Helvetica Neue" font-size="16" fill="black" x="3.610239" y="15">Key</tspan>
</text>
</g>
<g id="Graphic_6">
<rect x="358.05256" y="697.76" width="62.81032" height="26.391998" fill="white"/>
<rect x="358.05256" y="697.76" width="62.81032" height="26.391998" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
<text transform="translate(363.05256 701.732)" fill="black">
<tspan font-family="Helvetica Neue" font-size="16" fill="black" x="7.141159" y="15">Value</tspan>
</text>
</g>
<g id="Graphic_5">
<rect x="415.5681" y="697.76" width="44.48448" height="26.391998" fill="white"/>
<rect x="415.5681" y="697.76" width="44.48448" height="26.391998" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
<text transform="translate(420.5681 701.732)" fill="black">
<tspan font-family="Helvetica Neue" font-size="16" fill="black" x="3.610239" y="15">Key</tspan>
</text>
</g>
<g id="Graphic_4">
<rect x="460.05256" y="697.76" width="62.81032" height="26.391998" fill="white"/>
<rect x="460.05256" y="697.76" width="62.81032" height="26.391998" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
<text transform="translate(465.05256 701.732)" fill="black">
<tspan font-family="Helvetica Neue" font-size="16" fill="black" x="7.141159" y="15">Value</tspan>
</text>
</g>
<g id="Line_3">
<line x1="80.01552" y1="612.63" x2="7" y2="697.76" stroke="#7f8080" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
</g>
<g id="Line_2">
<line x1="142.82584" y1="612.63" x2="522.8629" y2="697.76" stroke="#7f8080" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
</g>
<g id="Graphic_49">
<text transform="translate(477.618 743.552)" fill="black">
<tspan font-family="Helvetica Neue" font-weight="bold" font-size="16" fill="black" x="33679726e-19" y="16">Week 1</tspan>
</text>
</g>
<g id="Graphic_59">
<rect x="511.8359" y="421.304" width="101.04616" height="41.5" fill="white"/>
<rect x="511.8359" y="421.304" width="101.04616" height="41.5" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
<text transform="translate(516.8359 432.83)" fill="black">
<tspan font-family="Helvetica Neue" font-size="16" fill="black" x="14.851079" y="15">Manifest</tspan>
</text>
</g>
<g id="Graphic_68">
<text transform="translate(415.9537 182)" fill="black">
<tspan font-family="Helvetica Neue" font-weight="bold" font-size="16" fill="black" x="33679726e-19" y="16">Week 1</tspan>
</text>
</g>
<g id="Graphic_74">
<text transform="translate(573.412 500.4861)" fill="black">
<tspan font-family="Helvetica Neue" font-weight="bold" font-size="16" fill="black" x="33679726e-19" y="16">Week 2</tspan>
</text>
</g>
</g>
</g>
</svg>

After

Width:  |  Height:  |  Size: 20 KiB

View File

@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?> <?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"> <!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<svg xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:xl="http://www.w3.org/1999/xlink" version="1.1" xmlns="http://www.w3.org/2000/svg" viewBox="6 194 693 361" width="693" height="361"> <svg xmlns="http://www.w3.org/2000/svg" version="1.1" xmlns:xl="http://www.w3.org/1999/xlink" xmlns:dc="http://purl.org/dc/elements/1.1/" viewBox="6 194 693 361" width="693" height="361">
<defs> <defs>
<marker orient="auto" overflow="visible" markerUnits="strokeWidth" id="FilledArrow_Marker" stroke-linejoin="miter" stroke-miterlimit="10" viewBox="-1 -4 10 8" markerWidth="10" markerHeight="8" color="black"> <marker orient="auto" overflow="visible" markerUnits="strokeWidth" id="FilledArrow_Marker" stroke-linejoin="miter" stroke-miterlimit="10" viewBox="-1 -4 10 8" markerWidth="10" markerHeight="8" color="black">
<g> <g>
@@ -18,7 +18,7 @@
</g> </g>
</marker> </marker>
</defs> </defs>
<g id="00-lsm-read-flow" stroke="none" fill-opacity="1" stroke-opacity="1" stroke-dasharray="none" fill="none"> <g id="00-lsm-read-flow" fill-opacity="1" stroke-dasharray="none" fill="none" stroke-opacity="1" stroke="none">
<title>00-lsm-read-flow</title> <title>00-lsm-read-flow</title>
<rect fill="white" x="6" y="194" width="693" height="361"/> <rect fill="white" x="6" y="194" width="693" height="361"/>
<g id="00-lsm-read-flow_Layer_1"> <g id="00-lsm-read-flow_Layer_1">

Before

Width:  |  Height:  |  Size: 12 KiB

After

Width:  |  Height:  |  Size: 12 KiB

View File

@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?> <?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"> <!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<svg xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:xl="http://www.w3.org/1999/xlink" version="1.1" xmlns="http://www.w3.org/2000/svg" viewBox="6 245 693 273" width="693" height="273"> <svg xmlns="http://www.w3.org/2000/svg" version="1.1" xmlns:xl="http://www.w3.org/1999/xlink" xmlns:dc="http://purl.org/dc/elements/1.1/" viewBox="6 245 693 273" width="693" height="273">
<defs> <defs>
<marker orient="auto" overflow="visible" markerUnits="strokeWidth" id="FilledArrow_Marker" stroke-linejoin="miter" stroke-miterlimit="10" viewBox="-1 -4 10 8" markerWidth="10" markerHeight="8" color="black"> <marker orient="auto" overflow="visible" markerUnits="strokeWidth" id="FilledArrow_Marker" stroke-linejoin="miter" stroke-miterlimit="10" viewBox="-1 -4 10 8" markerWidth="10" markerHeight="8" color="black">
<g> <g>
@@ -13,7 +13,7 @@
</g> </g>
</marker> </marker>
</defs> </defs>
<g id="00-lsm-write-flow" stroke="none" fill-opacity="1" stroke-opacity="1" stroke-dasharray="none" fill="none"> <g id="00-lsm-write-flow" fill-opacity="1" stroke-dasharray="none" fill="none" stroke-opacity="1" stroke="none">
<title>00-lsm-write-flow</title> <title>00-lsm-write-flow</title>
<rect fill="white" x="6" y="245" width="693" height="273"/> <rect fill="white" x="6" y="245" width="693" height="273"/>
<g id="00-lsm-write-flow_Layer_1"> <g id="00-lsm-write-flow_Layer_1">

Before

Width:  |  Height:  |  Size: 11 KiB

After

Width:  |  Height:  |  Size: 11 KiB

View File

@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?> <?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"> <!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<svg xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:xl="http://www.w3.org/1999/xlink" version="1.1" xmlns="http://www.w3.org/2000/svg" viewBox="6 213 693 136" width="693" height="136"> <svg xmlns="http://www.w3.org/2000/svg" version="1.1" xmlns:xl="http://www.w3.org/1999/xlink" xmlns:dc="http://purl.org/dc/elements/1.1/" viewBox="6 213 693 136" width="693" height="136">
<defs> <defs>
<marker orient="auto" overflow="visible" markerUnits="strokeWidth" id="FilledArrow_Marker" stroke-linejoin="miter" stroke-miterlimit="10" viewBox="-1 -4 10 8" markerWidth="10" markerHeight="8" color="black"> <marker orient="auto" overflow="visible" markerUnits="strokeWidth" id="FilledArrow_Marker" stroke-linejoin="miter" stroke-miterlimit="10" viewBox="-1 -4 10 8" markerWidth="10" markerHeight="8" color="black">
<g> <g>
@@ -8,7 +8,7 @@
</g> </g>
</marker> </marker>
</defs> </defs>
<g id="week1-01-overview" stroke="none" fill-opacity="1" stroke-opacity="1" stroke-dasharray="none" fill="none"> <g id="week1-01-overview" fill-opacity="1" stroke-dasharray="none" fill="none" stroke-opacity="1" stroke="none">
<title>week1-01-overview</title> <title>week1-01-overview</title>
<rect fill="white" x="6" y="213" width="693" height="136"/> <rect fill="white" x="6" y="213" width="693" height="136"/>
<g id="week1-01-overview_Layer_1"> <g id="week1-01-overview_Layer_1">

Before

Width:  |  Height:  |  Size: 4.5 KiB

After

Width:  |  Height:  |  Size: 4.5 KiB

View File

@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?> <?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"> <!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<svg xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:xl="http://www.w3.org/1999/xlink" version="1.1" xmlns="http://www.w3.org/2000/svg" viewBox="6 213 693 136" width="693" height="136"> <svg xmlns="http://www.w3.org/2000/svg" version="1.1" xmlns:xl="http://www.w3.org/1999/xlink" xmlns:dc="http://purl.org/dc/elements/1.1/" viewBox="6 213 693 136" width="693" height="136">
<defs> <defs>
<marker orient="auto" overflow="visible" markerUnits="strokeWidth" id="FilledArrow_Marker" stroke-linejoin="miter" stroke-miterlimit="10" viewBox="-1 -4 10 8" markerWidth="10" markerHeight="8" color="black"> <marker orient="auto" overflow="visible" markerUnits="strokeWidth" id="FilledArrow_Marker" stroke-linejoin="miter" stroke-miterlimit="10" viewBox="-1 -4 10 8" markerWidth="10" markerHeight="8" color="black">
<g> <g>
@@ -8,7 +8,7 @@
</g> </g>
</marker> </marker>
</defs> </defs>
<g id="week1-02-overview" stroke="none" fill-opacity="1" stroke-opacity="1" stroke-dasharray="none" fill="none"> <g id="week1-02-overview" fill-opacity="1" stroke-dasharray="none" fill="none" stroke-opacity="1" stroke="none">
<title>week1-02-overview</title> <title>week1-02-overview</title>
<rect fill="white" x="6" y="213" width="693" height="136"/> <rect fill="white" x="6" y="213" width="693" height="136"/>
<g id="week1-02-overview_Layer_1"> <g id="week1-02-overview_Layer_1">

Before

Width:  |  Height:  |  Size: 4.5 KiB

After

Width:  |  Height:  |  Size: 4.5 KiB

View File

@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?> <?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"> <!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<svg xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:xl="http://www.w3.org/1999/xlink" version="1.1" xmlns="http://www.w3.org/2000/svg" viewBox="-5 245 704 524" width="704" height="524"> <svg xmlns="http://www.w3.org/2000/svg" version="1.1" xmlns:xl="http://www.w3.org/1999/xlink" xmlns:dc="http://purl.org/dc/elements/1.1/" viewBox="-5 245 704 524" width="704" height="524">
<defs> <defs>
<marker orient="auto" overflow="visible" markerUnits="strokeWidth" id="FilledArrow_Marker" stroke-linejoin="miter" stroke-miterlimit="10" viewBox="-1 -4 10 8" markerWidth="10" markerHeight="8" color="black"> <marker orient="auto" overflow="visible" markerUnits="strokeWidth" id="FilledArrow_Marker" stroke-linejoin="miter" stroke-miterlimit="10" viewBox="-1 -4 10 8" markerWidth="10" markerHeight="8" color="black">
<g> <g>
@@ -8,10 +8,10 @@
</g> </g>
</marker> </marker>
</defs> </defs>
<g id="week1-03-overview_1" stroke="none" fill-opacity="1" stroke-opacity="1" stroke-dasharray="none" fill="none"> <g id="week1-03-overview" fill-opacity="1" stroke-dasharray="none" fill="none" stroke-opacity="1" stroke="none">
<title>week1-03-overview</title> <title>week1-03-overview</title>
<rect fill="white" x="-5" y="245" width="704" height="524"/> <rect fill="white" x="-5" y="245" width="704" height="524"/>
<g id="week1-03-overview_1_Layer_1"> <g id="week1-03-overview_Layer_1">
<title>Layer 1</title> <title>Layer 1</title>
<g id="Graphic_48"> <g id="Graphic_48">
<rect x="-3.6741593" y="675.5681" width="541" height="92.448" fill="white"/> <rect x="-3.6741593" y="675.5681" width="541" height="92.448" fill="white"/>

Before

Width:  |  Height:  |  Size: 18 KiB

After

Width:  |  Height:  |  Size: 18 KiB

View File

@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?> <?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"> <!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<svg xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:xl="http://www.w3.org/1999/xlink" version="1.1" xmlns="http://www.w3.org/2000/svg" viewBox="6 245 693 480" width="693" height="480"> <svg xmlns="http://www.w3.org/2000/svg" version="1.1" xmlns:xl="http://www.w3.org/1999/xlink" xmlns:dc="http://purl.org/dc/elements/1.1/" viewBox="6 245 693 480" width="693" height="480">
<defs> <defs>
<marker orient="auto" overflow="visible" markerUnits="strokeWidth" id="FilledArrow_Marker" stroke-linejoin="miter" stroke-miterlimit="10" viewBox="-1 -4 10 8" markerWidth="10" markerHeight="8" color="black"> <marker orient="auto" overflow="visible" markerUnits="strokeWidth" id="FilledArrow_Marker" stroke-linejoin="miter" stroke-miterlimit="10" viewBox="-1 -4 10 8" markerWidth="10" markerHeight="8" color="black">
<g> <g>
@@ -8,10 +8,10 @@
</g> </g>
</marker> </marker>
</defs> </defs>
<g id="week1-04-overview_1" stroke="none" fill-opacity="1" stroke-opacity="1" stroke-dasharray="none" fill="none"> <g id="week1-04-overview" fill-opacity="1" stroke-dasharray="none" fill="none" stroke-opacity="1" stroke="none">
<title>week1-04-overview</title> <title>week1-04-overview</title>
<rect fill="white" x="6" y="245" width="693" height="480"/> <rect fill="white" x="6" y="245" width="693" height="480"/>
<g id="week1-04-overview_1_Layer_1"> <g id="week1-04-overview_Layer_1">
<title>Layer 1</title> <title>Layer 1</title>
<g id="Graphic_52"> <g id="Graphic_52">
<rect x="61.5" y="558.03406" width="475.82584" height="92.448" fill="white"/> <rect x="61.5" y="558.03406" width="475.82584" height="92.448" fill="white"/>

Before

Width:  |  Height:  |  Size: 18 KiB

After

Width:  |  Height:  |  Size: 18 KiB

View File

@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?> <?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"> <!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<svg xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:xl="http://www.w3.org/1999/xlink" version="1.1" xmlns="http://www.w3.org/2000/svg" viewBox="6 170 693 236" width="693" height="236"> <svg xmlns="http://www.w3.org/2000/svg" version="1.1" xmlns:xl="http://www.w3.org/1999/xlink" xmlns:dc="http://purl.org/dc/elements/1.1/" viewBox="6 170 693 236" width="693" height="236">
<defs> <defs>
<marker orient="auto" overflow="visible" markerUnits="strokeWidth" id="FilledArrow_Marker" stroke-linejoin="miter" stroke-miterlimit="10" viewBox="-1 -4 10 8" markerWidth="10" markerHeight="8" color="black"> <marker orient="auto" overflow="visible" markerUnits="strokeWidth" id="FilledArrow_Marker" stroke-linejoin="miter" stroke-miterlimit="10" viewBox="-1 -4 10 8" markerWidth="10" markerHeight="8" color="black">
<g> <g>
@@ -8,7 +8,7 @@
</g> </g>
</marker> </marker>
</defs> </defs>
<g id="week1-05-overview" stroke="none" fill-opacity="1" stroke-opacity="1" stroke-dasharray="none" fill="none"> <g id="week1-05-overview" fill-opacity="1" stroke-dasharray="none" fill="none" stroke-opacity="1" stroke="none">
<title>week1-05-overview</title> <title>week1-05-overview</title>
<rect fill="white" x="6" y="170" width="693" height="236"/> <rect fill="white" x="6" y="170" width="693" height="236"/>
<g id="week1-05-overview_Layer_1"> <g id="week1-05-overview_Layer_1">

Before

Width:  |  Height:  |  Size: 6.9 KiB

After

Width:  |  Height:  |  Size: 6.9 KiB

View File

@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?> <?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"> <!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<svg xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:xl="http://www.w3.org/1999/xlink" version="1.1" xmlns="http://www.w3.org/2000/svg" viewBox="-5 461 784 308" width="784" height="308"> <svg xmlns="http://www.w3.org/2000/svg" version="1.1" xmlns:xl="http://www.w3.org/1999/xlink" xmlns:dc="http://purl.org/dc/elements/1.1/" viewBox="-5 461 784 308" width="784" height="308">
<defs> <defs>
<marker orient="auto" overflow="visible" markerUnits="strokeWidth" id="FilledArrow_Marker" stroke-linejoin="miter" stroke-miterlimit="10" viewBox="-1 -4 10 8" markerWidth="10" markerHeight="8" color="#7f8080"> <marker orient="auto" overflow="visible" markerUnits="strokeWidth" id="FilledArrow_Marker" stroke-linejoin="miter" stroke-miterlimit="10" viewBox="-1 -4 10 8" markerWidth="10" markerHeight="8" color="#7f8080">
<g> <g>
@@ -8,7 +8,7 @@
</g> </g>
</marker> </marker>
</defs> </defs>
<g id="week1-07-overview" stroke="none" fill-opacity="1" stroke-opacity="1" stroke-dasharray="none" fill="none"> <g id="week1-07-overview" fill-opacity="1" stroke-dasharray="none" fill="none" stroke-opacity="1" stroke="none">
<title>week1-07-overview</title> <title>week1-07-overview</title>
<rect fill="white" x="-5" y="461" width="784" height="308"/> <rect fill="white" x="-5" y="461" width="784" height="308"/>
<g id="week1-07-overview_Layer_1"> <g id="week1-07-overview_Layer_1">

Before

Width:  |  Height:  |  Size: 12 KiB

After

Width:  |  Height:  |  Size: 12 KiB

View File

@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?> <?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"> <!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<svg xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:xl="http://www.w3.org/1999/xlink" version="1.1" xmlns="http://www.w3.org/2000/svg" viewBox="-5 170 784 599" width="784" height="599"> <svg xmlns="http://www.w3.org/2000/svg" version="1.1" xmlns:xl="http://www.w3.org/1999/xlink" xmlns:dc="http://purl.org/dc/elements/1.1/" viewBox="-5 170 784 599" width="784" height="599">
<defs> <defs>
<marker orient="auto" overflow="visible" markerUnits="strokeWidth" id="FilledArrow_Marker" stroke-linejoin="miter" stroke-miterlimit="10" viewBox="-1 -4 10 8" markerWidth="10" markerHeight="8" color="black"> <marker orient="auto" overflow="visible" markerUnits="strokeWidth" id="FilledArrow_Marker" stroke-linejoin="miter" stroke-miterlimit="10" viewBox="-1 -4 10 8" markerWidth="10" markerHeight="8" color="black">
<g> <g>
@@ -13,7 +13,7 @@
</g> </g>
</marker> </marker>
</defs> </defs>
<g id="week1-overview" stroke="none" fill-opacity="1" stroke-opacity="1" stroke-dasharray="none" fill="none"> <g id="week1-overview" fill-opacity="1" stroke-dasharray="none" fill="none" stroke-opacity="1" stroke="none">
<title>week1-overview</title> <title>week1-overview</title>
<rect fill="white" x="-5" y="170" width="784" height="599"/> <rect fill="white" x="-5" y="170" width="784" height="599"/>
<g id="week1-overview_Layer_1"> <g id="week1-overview_Layer_1">

Before

Width:  |  Height:  |  Size: 22 KiB

After

Width:  |  Height:  |  Size: 22 KiB

View File

@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?> <?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"> <!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<svg xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:xl="http://www.w3.org/1999/xlink" version="1.1" xmlns="http://www.w3.org/2000/svg" viewBox="6 245 693 269" width="693" height="269"> <svg xmlns="http://www.w3.org/2000/svg" version="1.1" xmlns:xl="http://www.w3.org/1999/xlink" xmlns:dc="http://purl.org/dc/elements/1.1/" viewBox="6 245 693 269" width="693" height="269">
<defs> <defs>
<marker orient="auto" overflow="visible" markerUnits="strokeWidth" id="FilledArrow_Marker" stroke-linejoin="miter" stroke-miterlimit="10" viewBox="-1 -4 10 8" markerWidth="10" markerHeight="8" color="black"> <marker orient="auto" overflow="visible" markerUnits="strokeWidth" id="FilledArrow_Marker" stroke-linejoin="miter" stroke-miterlimit="10" viewBox="-1 -4 10 8" markerWidth="10" markerHeight="8" color="black">
<g> <g>
@@ -8,7 +8,7 @@
</g> </g>
</marker> </marker>
</defs> </defs>
<g id="week2-01-overview" stroke="none" fill-opacity="1" stroke-opacity="1" stroke-dasharray="none" fill="none"> <g id="week2-01-overview" fill-opacity="1" stroke-dasharray="none" fill="none" stroke-opacity="1" stroke="none">
<title>week2-01-overview</title> <title>week2-01-overview</title>
<rect fill="white" x="6" y="245" width="693" height="269"/> <rect fill="white" x="6" y="245" width="693" height="269"/>
<g id="week2-01-overview_Layer_1"> <g id="week2-01-overview_Layer_1">

Before

Width:  |  Height:  |  Size: 8.9 KiB

After

Width:  |  Height:  |  Size: 8.9 KiB

View File

@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?> <?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"> <!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<svg xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:xl="http://www.w3.org/1999/xlink" version="1.1" xmlns="http://www.w3.org/2000/svg" viewBox="6 245 765 268" width="765" height="268"> <svg xmlns="http://www.w3.org/2000/svg" version="1.1" xmlns:xl="http://www.w3.org/1999/xlink" xmlns:dc="http://purl.org/dc/elements/1.1/" viewBox="6 245 765 268" width="765" height="268">
<defs> <defs>
<marker orient="auto" overflow="visible" markerUnits="strokeWidth" id="FilledArrow_Marker" stroke-linejoin="miter" stroke-miterlimit="10" viewBox="-1 -4 10 8" markerWidth="10" markerHeight="8" color="black"> <marker orient="auto" overflow="visible" markerUnits="strokeWidth" id="FilledArrow_Marker" stroke-linejoin="miter" stroke-miterlimit="10" viewBox="-1 -4 10 8" markerWidth="10" markerHeight="8" color="black">
<g> <g>
@@ -13,7 +13,7 @@
</g> </g>
</marker> </marker>
</defs> </defs>
<g id="week2-05-overview" stroke="none" fill-opacity="1" stroke-opacity="1" stroke-dasharray="none" fill="none"> <g id="week2-05-overview" fill-opacity="1" stroke-dasharray="none" fill="none" stroke-opacity="1" stroke="none">
<title>week2-05-overview</title> <title>week2-05-overview</title>
<rect fill="white" x="6" y="245" width="765" height="268"/> <rect fill="white" x="6" y="245" width="765" height="268"/>
<g id="week2-05-overview_Layer_1"> <g id="week2-05-overview_Layer_1">

Before

Width:  |  Height:  |  Size: 9.6 KiB

After

Width:  |  Height:  |  Size: 9.6 KiB

View File

@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?> <?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"> <!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<svg xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:xl="http://www.w3.org/1999/xlink" version="1.1" xmlns="http://www.w3.org/2000/svg" viewBox="6 213 693 291" width="693" height="291"> <svg xmlns="http://www.w3.org/2000/svg" version="1.1" xmlns:xl="http://www.w3.org/1999/xlink" xmlns:dc="http://purl.org/dc/elements/1.1/" viewBox="6 213 693 291" width="693" height="291">
<defs> <defs>
<marker orient="auto" overflow="visible" markerUnits="strokeWidth" id="FilledArrow_Marker" stroke-linejoin="miter" stroke-miterlimit="10" viewBox="-1 -4 10 8" markerWidth="10" markerHeight="8" color="black"> <marker orient="auto" overflow="visible" markerUnits="strokeWidth" id="FilledArrow_Marker" stroke-linejoin="miter" stroke-miterlimit="10" viewBox="-1 -4 10 8" markerWidth="10" markerHeight="8" color="black">
<g> <g>
@@ -8,7 +8,7 @@
</g> </g>
</marker> </marker>
</defs> </defs>
<g id="week2-06-overview" stroke="none" fill-opacity="1" stroke-opacity="1" stroke-dasharray="none" fill="none"> <g id="week2-06-overview" fill-opacity="1" stroke-dasharray="none" fill="none" stroke-opacity="1" stroke="none">
<title>week2-06-overview</title> <title>week2-06-overview</title>
<rect fill="white" x="6" y="213" width="693" height="291"/> <rect fill="white" x="6" y="213" width="693" height="291"/>
<g id="week2-06-overview_Layer_1"> <g id="week2-06-overview_Layer_1">

Before

Width:  |  Height:  |  Size: 10 KiB

After

Width:  |  Height:  |  Size: 10 KiB

View File

@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?> <?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"> <!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<svg xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:xl="http://www.w3.org/1999/xlink" version="1.1" xmlns="http://www.w3.org/2000/svg" viewBox="6 156 788 358" width="788" height="358"> <svg xmlns="http://www.w3.org/2000/svg" version="1.1" xmlns:xl="http://www.w3.org/1999/xlink" xmlns:dc="http://purl.org/dc/elements/1.1/" viewBox="6 156 788 358" width="788" height="358">
<defs> <defs>
<marker orient="auto" overflow="visible" markerUnits="strokeWidth" id="FilledArrow_Marker" stroke-linejoin="miter" stroke-miterlimit="10" viewBox="-1 -4 10 8" markerWidth="10" markerHeight="8" color="black"> <marker orient="auto" overflow="visible" markerUnits="strokeWidth" id="FilledArrow_Marker" stroke-linejoin="miter" stroke-miterlimit="10" viewBox="-1 -4 10 8" markerWidth="10" markerHeight="8" color="black">
<g> <g>
@@ -13,7 +13,7 @@
</g> </g>
</marker> </marker>
</defs> </defs>
<g id="week2-overview" stroke="none" fill-opacity="1" stroke-opacity="1" stroke-dasharray="none" fill="none"> <g id="week2-overview" fill-opacity="1" stroke-dasharray="none" fill="none" stroke-opacity="1" stroke="none">
<title>week2-overview</title> <title>week2-overview</title>
<rect fill="white" x="6" y="156" width="788" height="358"/> <rect fill="white" x="6" y="156" width="788" height="358"/>
<g id="week2-overview_Layer_1"> <g id="week2-overview_Layer_1">

Before

Width:  |  Height:  |  Size: 12 KiB

After

Width:  |  Height:  |  Size: 12 KiB

View File

@@ -10,8 +10,8 @@ In this chapter, you will:
## Task 1: Two Merge Iterator ## Task 1: Two Merge Iterator
## Task 2: Read Path - Get ## Task 3: Read Path - Get
## Task 3: Read Path - Scan ## Task 4: Read Path - Scan
{{#include copyright.md}} {{#include copyright.md}}

View File

@@ -11,4 +11,6 @@ In this chapter, you will:
## Task 2: Update the LSM State ## Task 2: Update the LSM State
## Task 3: Filtering the SSTs
{{#include copyright.md}} {{#include copyright.md}}

View File

@@ -1,7 +1,17 @@
# Overview # Week 1 Overview: Mini-LSM
![Chapter Overview](./lsm-tutorial/week1-overview.svg) ![Chapter Overview](./lsm-tutorial/week1-overview.svg)
key functionalities index In the first week of the tutorial, you will build necessary storage formats for the storage engine, the read path and the write path of the system, and have a working implementation of an LSM-based key-value store. There are 7 chapters (days) for this part.
* Day 1: Memtable. You will implement the in-memory read and write path of the system.
* Day 2: Merge iterator. You will extend what you have built in day 1 and implement a `scan` interface for your system.
* Day 3: Block encoding. Now we start the first step of the on-disk structure and build the encoding/decoding of the blocks.
* Day 4: SST encoding. SSTs are composed of blocks and at the end of the day, you will have the basic building blocks of the LSM on-disk structure.
* Day 5: Read path. Now that we have both in-memory and on-disk structures, we can combine them together and have a fully-working read path for the storage engine.
* Day 6: Write path. In day 5, the test harness generates the structures, and in day 6, you will control the SST flushes by yourself. You will implement flush to level-0 SST and the storage engine is complete.
* Day 7: SST optimizations. We will implement several SST format optimizations and improve the performance of the system.
At the end of the week, your storage engine should be able to handle all get/scan/put requests. The only missing parts are persisting the LSM state to disk and a more efficient way of organizing the SSTs on the disk. You will have a working *Mini-LSM* storage engine.
{{#include copyright.md}} {{#include copyright.md}}