Browse Source

add readseq compare without compaction thread

master
小人鱼 8 months ago
parent
commit
bd42141075
3 changed files with 120 additions and 3 deletions
  1. +108
    -0
      benchmark_data.md
  2. BIN
      设计文档.assets/image-20250107101406637.png
  3. +12
    -3
      设计文档.md

+ 108
- 0
benchmark_data.md View File

@ -1590,3 +1590,111 @@ All steps completed successfully!
# 有无GC db_bench
原leveldb:
```
LevelDB: version 1.23
Date: Tue Jan 7 01:32:40 2025
CPU: 32 * Intel(R) Xeon(R) Silver 4110 CPU @ 2.10GHz
CPUCache: 11264 KB
Keys: 16 bytes each
Values: 5000 bytes each (2500 bytes after compression)
Entries: 1000000
RawSize: 4783.6 MB (estimated)
FileSize: 2399.4 MB (estimated)
WARNING: Snappy compression is not enabled
------------------------------------------------
fillseq : 43.561 micros/op; 109.8 MB/s
fillsync : 8360.610 micros/op; 0.6 MB/s (1000 ops)
fillrandom : 306.627 micros/op; 15.6 MB/s
overwrite : 423.441 micros/op; 11.3 MB/s
readrandom : 41.489 micros/op; (864322 of 1000000 found)
readrandom : 33.352 micros/op; (864083 of 1000000 found)
readseq : 1.077 micros/op; 4442.3 MB/s
readreverse : 2.384 micros/op; 2006.2 MB/s
compact : 27672776.000 micros/op;
readrandom : 23.430 micros/op; (864105 of 1000000 found)
readseq : 1.262 micros/op; 3791.0 MB/s
readreverse : 2.488 micros/op; 1922.6 MB/s
fill100K : 909.619 micros/op; 104.9 MB/s (1000 ops)
crc32c : 1.708 micros/op; 2287.5 MB/s (4K per op)
snappycomp : 9937.000 micros/op; (snappy failure)
snappyuncomp : 10061.000 micros/op; (snappy failure)
zstdcomp : 6627.000 micros/op; (zstd failure)
zstduncomp : 6851.000 micros/op; (zstd failure)
```
无GC:
```
LevelDB: version 1.23
Date: Tue Jan 7 01:12:39 2025
CPU: 32 * Intel(R) Xeon(R) Silver 4110 CPU @ 2.10GHz
CPUCache: 11264 KB
Keys: 16 bytes each
Values: 5000 bytes each (2500 bytes after compression)
Entries: 1000000
RawSize: 4783.6 MB (estimated)
FileSize: 2399.4 MB (estimated)
WARNING: Snappy compression is not enabled
------------------------------------------------
fillseq : 15.353 micros/op; 311.6 MB/s
fillsync : 8307.683 micros/op; 0.6 MB/s (1000 ops)
fillrandom : 16.065 micros/op; 297.8 MB/s
overwrite : 16.277 micros/op; 293.9 MB/s
readrandom : 13.119 micros/op; (864322 of 1000000 found)
readrandom : 12.603 micros/op; (864083 of 1000000 found)
readunorderseq : 4.414 micros/op; 1083.8 MB/s
readseq : 10.850 micros/op; 440.9 MB/s
readreverse : 10.840 micros/op; 441.3 MB/s
compact : 301143.000 micros/op;
readrandom : 11.220 micros/op; (864482 of 1000000 found)
readunorderseq : 3.934 micros/op; 1216.1 MB/s
readseq : 10.685 micros/op; 447.7 MB/s
readreverse : 10.865 micros/op; 440.3 MB/s
fill100K : 114.789 micros/op; 830.9 MB/s (1000 ops)
crc32c : 1.734 micros/op; 2253.3 MB/s (4K per op)
snappycomp : 9693.000 micros/op; (snappy failure)
snappyuncomp : 9252.000 micros/op; (snappy failure)
zstdcomp : 9907.000 micros/op; (zstd failure)
zstduncomp : 8232.000 micros/op; (zstd failure)
```
有GC:
```
LevelDB: version 1.23
Date: Tue Jan 7 01:26:45 2025
CPU: 32 * Intel(R) Xeon(R) Silver 4110 CPU @ 2.10GHz
CPUCache: 11264 KB
Keys: 16 bytes each
Values: 5000 bytes each (2500 bytes after compression)
Entries: 1000000
RawSize: 4783.6 MB (estimated)
FileSize: 2399.4 MB (estimated)
WARNING: Snappy compression is not enabled
------------------------------------------------
fillseq : 15.083 micros/op; 317.2 MB/s
fillsync : 8272.137 micros/op; 0.6 MB/s (1000 ops)
fillrandom : 17.520 micros/op; 273.0 MB/s
overwrite : 23.640 micros/op; 202.3 MB/s
readrandom : 18.831 micros/op; (864322 of 1000000 found)
readrandom : 12.227 micros/op; (864083 of 1000000 found)
readunorderseq : 4.124 micros/op; 1159.9 MB/s
readseq : 10.532 micros/op; 454.2 MB/s
readreverse : 10.518 micros/op; 454.8 MB/s
compact : 344150.000 micros/op;
readrandom : 18.466 micros/op; (864482 of 1000000 found)
readunorderseq : 3.753 micros/op; 1274.6 MB/s
readseq : 9.922 micros/op; 482.1 MB/s
readreverse : 10.438 micros/op; 458.3 MB/s
fill100K : 109.456 micros/op; 871.4 MB/s (1000 ops)
crc32c : 1.621 micros/op; 2410.0 MB/s (4K per op)
snappycomp : 11457.000 micros/op; (snappy failure)
snappyuncomp : 9203.000 micros/op; (snappy failure)
zstdcomp : 6013.000 micros/op; (zstd failure)
zstduncomp : 9923.000 micros/op; (zstd failure)
```

BIN
设计文档.assets/image-20250107101406637.png View File

Before After
Width: 752  |  Height: 452  |  Size: 16 KiB

+ 12
- 3
设计文档.md View File

@ -452,6 +452,8 @@ mode==1表示遍历valuelog_map阶段
mode==2表示读完所有数据,InValid了。
小优化:对于大value会复用内存空间,避免每次都free和new一块大空间。经过测试该优化会带来一些性能提升。(不在以下函数而在MyReadValuelog函数中)
```
void UnorderedIter::Next() {
if(mode==0){
@ -784,7 +786,7 @@ WriteOptions新增:
### 5.1**单元测试(测试用例)**:
我们写了共十个大测试。
我们写了共十个大测试。测试的具体内容请看test/test.cpp。
分别为:
- 写ValueLog完成后未进入WAL日志的崩溃恢复测试(通过查看ValueLog内部可以看到恢复后未完成的数据从ValueLog中删除)
@ -908,8 +910,16 @@ gc valuelog 33 success and write 12334344 bytes
##### 随机读
在有compaction的情况下的读性能对比,可以看到我们的读性能更好。
![image-20250105205822843](设计文档.assets/image-20250105205822843.png)
在无compaction的情况下随机对比:
发现不考虑合并对读取性能影响的情况下,我们的随机读性能仍然更优。
![image-20250107101406637](设计文档.assets/image-20250107101406637.png)
##### 原生范围查询
![image-20250105205545925](设计文档.assets/image-20250105205545925.png)
@ -1053,7 +1063,7 @@ getkeysbyfieldproportion=0.001
![image-20250105220226773](设计文档.assets/image-20250105220226773.png)
## 6. 正确性分析与性能分析
## 6. 内存管理分析、正确性分析与性能分析
#### 1.内存管理分析
@ -1172,4 +1182,3 @@ getkeysbyfieldproportion=0.001
| 各类性能测试数据记录以及分析 | 1月5日 | 徐翔宇 |
| 完成设计文档 | 1月5日 | 徐翔宇&谢瑞阳 |

Loading…
Cancel
Save