diff --git a/benchmark_data.md b/benchmark_data.md index ed39740..44d346d 100644 --- a/benchmark_data.md +++ b/benchmark_data.md @@ -1590,3 +1590,111 @@ All steps completed successfully! +# 有无GC db_bench + +原leveldb： + +``` +LevelDB: version 1.23 +Date: Tue Jan 7 01:32:40 2025 +CPU: 32 * Intel(R) Xeon(R) Silver 4110 CPU @ 2.10GHz +CPUCache: 11264 KB +Keys: 16 bytes each +Values: 5000 bytes each (2500 bytes after compression) +Entries: 1000000 +RawSize: 4783.6 MB (estimated) +FileSize: 2399.4 MB (estimated) +WARNING: Snappy compression is not enabled +------------------------------------------------ +fillseq : 43.561 micros/op; 109.8 MB/s +fillsync : 8360.610 micros/op; 0.6 MB/s (1000 ops) +fillrandom : 306.627 micros/op; 15.6 MB/s +overwrite : 423.441 micros/op; 11.3 MB/s +readrandom : 41.489 micros/op; (864322 of 1000000 found) +readrandom : 33.352 micros/op; (864083 of 1000000 found) +readseq : 1.077 micros/op; 4442.3 MB/s +readreverse : 2.384 micros/op; 2006.2 MB/s +compact : 27672776.000 micros/op; +readrandom : 23.430 micros/op; (864105 of 1000000 found) +readseq : 1.262 micros/op; 3791.0 MB/s +readreverse : 2.488 micros/op; 1922.6 MB/s +fill100K : 909.619 micros/op; 104.9 MB/s (1000 ops) +crc32c : 1.708 micros/op; 2287.5 MB/s (4K per op) +snappycomp : 9937.000 micros/op; (snappy failure) +snappyuncomp : 10061.000 micros/op; (snappy failure) +zstdcomp : 6627.000 micros/op; (zstd failure) +zstduncomp : 6851.000 micros/op; (zstd failure) +``` + +无GC： + +``` +LevelDB: version 1.23 +Date: Tue Jan 7 01:12:39 2025 +CPU: 32 * Intel(R) Xeon(R) Silver 4110 CPU @ 2.10GHz +CPUCache: 11264 KB +Keys: 16 bytes each +Values: 5000 bytes each (2500 bytes after compression) +Entries: 1000000 +RawSize: 4783.6 MB (estimated) +FileSize: 2399.4 MB (estimated) +WARNING: Snappy compression is not enabled +------------------------------------------------ +fillseq : 15.353 micros/op; 311.6 MB/s +fillsync : 8307.683 micros/op; 0.6 MB/s (1000 ops) +fillrandom : 16.065 micros/op; 297.8 MB/s +overwrite : 16.277 micros/op; 293.9 MB/s +readrandom : 13.119 micros/op; (864322 of 1000000 found) +readrandom : 12.603 micros/op; (864083 of 1000000 found) +readunorderseq : 4.414 micros/op; 1083.8 MB/s +readseq : 10.850 micros/op; 440.9 MB/s +readreverse : 10.840 micros/op; 441.3 MB/s +compact : 301143.000 micros/op; +readrandom : 11.220 micros/op; (864482 of 1000000 found) +readunorderseq : 3.934 micros/op; 1216.1 MB/s +readseq : 10.685 micros/op; 447.7 MB/s +readreverse : 10.865 micros/op; 440.3 MB/s +fill100K : 114.789 micros/op; 830.9 MB/s (1000 ops) +crc32c : 1.734 micros/op; 2253.3 MB/s (4K per op) +snappycomp : 9693.000 micros/op; (snappy failure) +snappyuncomp : 9252.000 micros/op; (snappy failure) +zstdcomp : 9907.000 micros/op; (zstd failure) +zstduncomp : 8232.000 micros/op; (zstd failure) +``` + +有GC： + +``` +LevelDB: version 1.23 +Date: Tue Jan 7 01:26:45 2025 +CPU: 32 * Intel(R) Xeon(R) Silver 4110 CPU @ 2.10GHz +CPUCache: 11264 KB +Keys: 16 bytes each +Values: 5000 bytes each (2500 bytes after compression) +Entries: 1000000 +RawSize: 4783.6 MB (estimated) +FileSize: 2399.4 MB (estimated) +WARNING: Snappy compression is not enabled +------------------------------------------------ +fillseq : 15.083 micros/op; 317.2 MB/s +fillsync : 8272.137 micros/op; 0.6 MB/s (1000 ops) +fillrandom : 17.520 micros/op; 273.0 MB/s +overwrite : 23.640 micros/op; 202.3 MB/s +readrandom : 18.831 micros/op; (864322 of 1000000 found) +readrandom : 12.227 micros/op; (864083 of 1000000 found) +readunorderseq : 4.124 micros/op; 1159.9 MB/s +readseq : 10.532 micros/op; 454.2 MB/s +readreverse : 10.518 micros/op; 454.8 MB/s +compact : 344150.000 micros/op; +readrandom : 18.466 micros/op; (864482 of 1000000 found) +readunorderseq : 3.753 micros/op; 1274.6 MB/s +readseq : 9.922 micros/op; 482.1 MB/s +readreverse : 10.438 micros/op; 458.3 MB/s +fill100K : 109.456 micros/op; 871.4 MB/s (1000 ops) +crc32c : 1.621 micros/op; 2410.0 MB/s (4K per op) +snappycomp : 11457.000 micros/op; (snappy failure) +snappyuncomp : 9203.000 micros/op; (snappy failure) +zstdcomp : 6013.000 micros/op; (zstd failure) +zstduncomp : 9923.000 micros/op; (zstd failure) +``` + diff --git a/设计文档.assets/image-20250107101406637.png b/设计文档.assets/image-20250107101406637.png new file mode 100644 index 0000000..25fa034 Binary files /dev/null and b/设计文档.assets/image-20250107101406637.png differ diff --git a/设计文档.md b/设计文档.md index ad92187..f5d1c82 100644 --- a/设计文档.md +++ b/设计文档.md @@ -452,6 +452,8 @@ mode==1表示遍历valuelog_map阶段 mode==2表示读完所有数据，InValid了。 +小优化：对于大value会复用内存空间，避免每次都free和new一块大空间。经过测试该优化会带来一些性能提升。（不在以下函数而在MyReadValuelog函数中） + ``` void UnorderedIter::Next() { if(mode==0){ @@ -784,7 +786,7 @@ WriteOptions新增： ### 5.1**单元测试（测试用例）**： -我们写了共十个大测试。 +我们写了共十个大测试。测试的具体内容请看test/test.cpp。 分别为： - 写ValueLog完成后未进入WAL日志的崩溃恢复测试（通过查看ValueLog内部可以看到恢复后未完成的数据从ValueLog中删除） @@ -908,8 +910,16 @@ gc valuelog 33 success and write 12334344 bytes ##### 随机读 +在有compaction的情况下的读性能对比，可以看到我们的读性能更好。 + ![image-20250105205822843](设计文档.assets/image-20250105205822843.png) +在无compaction的情况下随机对比： + +发现不考虑合并对读取性能影响的情况下，我们的随机读性能仍然更优。 + +![image-20250107101406637](设计文档.assets/image-20250107101406637.png) + ##### 原生范围查询 ![image-20250105205545925](设计文档.assets/image-20250105205545925.png) @@ -1053,7 +1063,7 @@ getkeysbyfieldproportion=0.001 ![image-20250105220226773](设计文档.assets/image-20250105220226773.png) -## 6. 正确性分析与性能分析 +## 6. 内存管理分析、正确性分析与性能分析 #### 1.内存管理分析 @@ -1172,4 +1182,3 @@ getkeysbyfieldproportion=0.001 | 各类性能测试数据记录以及分析 | 1月5日 | 徐翔宇 | | 完成设计文档 | 1月5日 | 徐翔宇&谢瑞阳 | -