作者: 韩晨旭 10225101440 李畅 10225102463
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

164 regels
6.2 KiB

Clean up util/coding.{h,cc}. 1) Inline EncodeFixed{32,64}(). They emit single machine instructions on 64-bit processors. 2) Remove size narrowing compiler warnings from DecodeFixed{32,64}(). 3) Add comments explaining the current state of optimizations in compilers we care about. 4) Change C-style includes, like <stdint.h>, to C++ style, like <cstdint>. 5) memcpy -> std::memcpy. The optimization comments are based on https://godbolt.org/z/RdIqS1. The missed optimization opportunities in clang have been reported as https://bugs.llvm.org/show_bug.cgi?id=41761 The change does not have significant impact on benchmarks. Results below. LevelDB: version 1.22 Date: Mon May 6 10:42:18 2019 CPU: 72 * Intel(R) Xeon(R) Gold 6154 CPU @ 3.00GHz CPUCache: 25344 KB Keys: 16 bytes each Values: 100 bytes each (50 bytes after compression) Entries: 1000000 RawSize: 110.6 MB (estimated) FileSize: 62.9 MB (estimated) With change ------------------------------------------------ fillseq : 2.327 micros/op; 47.5 MB/s fillsync : 4185.526 micros/op; 0.0 MB/s (1000 ops) fillrandom : 3.662 micros/op; 30.2 MB/s overwrite : 4.261 micros/op; 26.0 MB/s readrandom : 4.239 micros/op; (1000000 of 1000000 found) readrandom : 3.649 micros/op; (1000000 of 1000000 found) readseq : 0.174 micros/op; 636.7 MB/s readreverse : 0.271 micros/op; 408.7 MB/s compact : 570495.000 micros/op; readrandom : 2.735 micros/op; (1000000 of 1000000 found) readseq : 0.118 micros/op; 937.3 MB/s readreverse : 0.190 micros/op; 583.7 MB/s fill100K : 860.164 micros/op; 110.9 MB/s (1000 ops) crc32c : 1.131 micros/op; 3455.2 MB/s (4K per op) snappycomp : 3.034 micros/op; 1287.5 MB/s (output: 55.1%) snappyuncomp : 0.544 micros/op; 7176.0 MB/s Baseline ------------------------------------------------ fillseq : 2.365 micros/op; 46.8 MB/s fillsync : 4240.165 micros/op; 0.0 MB/s (1000 ops) fillrandom : 3.244 micros/op; 34.1 MB/s overwrite : 4.153 micros/op; 26.6 MB/s readrandom : 4.698 micros/op; (1000000 of 1000000 found) readrandom : 4.065 micros/op; (1000000 of 1000000 found) readseq : 0.192 micros/op; 576.3 MB/s readreverse : 0.286 micros/op; 386.7 MB/s compact : 635979.000 micros/op; readrandom : 3.264 micros/op; (1000000 of 1000000 found) readseq : 0.169 micros/op; 652.8 MB/s readreverse : 0.213 micros/op; 519.5 MB/s fill100K : 1055.367 micros/op; 90.4 MB/s (1000 ops) crc32c : 1.353 micros/op; 2887.3 MB/s (4K per op) snappycomp : 3.036 micros/op; 1286.7 MB/s (output: 55.1%) snappyuncomp : 0.540 micros/op; 7238.6 MB/s PiperOrigin-RevId: 246856811
5 jaren geleden
Clean up util/coding.{h,cc}. 1) Inline EncodeFixed{32,64}(). They emit single machine instructions on 64-bit processors. 2) Remove size narrowing compiler warnings from DecodeFixed{32,64}(). 3) Add comments explaining the current state of optimizations in compilers we care about. 4) Change C-style includes, like <stdint.h>, to C++ style, like <cstdint>. 5) memcpy -> std::memcpy. The optimization comments are based on https://godbolt.org/z/RdIqS1. The missed optimization opportunities in clang have been reported as https://bugs.llvm.org/show_bug.cgi?id=41761 The change does not have significant impact on benchmarks. Results below. LevelDB: version 1.22 Date: Mon May 6 10:42:18 2019 CPU: 72 * Intel(R) Xeon(R) Gold 6154 CPU @ 3.00GHz CPUCache: 25344 KB Keys: 16 bytes each Values: 100 bytes each (50 bytes after compression) Entries: 1000000 RawSize: 110.6 MB (estimated) FileSize: 62.9 MB (estimated) With change ------------------------------------------------ fillseq : 2.327 micros/op; 47.5 MB/s fillsync : 4185.526 micros/op; 0.0 MB/s (1000 ops) fillrandom : 3.662 micros/op; 30.2 MB/s overwrite : 4.261 micros/op; 26.0 MB/s readrandom : 4.239 micros/op; (1000000 of 1000000 found) readrandom : 3.649 micros/op; (1000000 of 1000000 found) readseq : 0.174 micros/op; 636.7 MB/s readreverse : 0.271 micros/op; 408.7 MB/s compact : 570495.000 micros/op; readrandom : 2.735 micros/op; (1000000 of 1000000 found) readseq : 0.118 micros/op; 937.3 MB/s readreverse : 0.190 micros/op; 583.7 MB/s fill100K : 860.164 micros/op; 110.9 MB/s (1000 ops) crc32c : 1.131 micros/op; 3455.2 MB/s (4K per op) snappycomp : 3.034 micros/op; 1287.5 MB/s (output: 55.1%) snappyuncomp : 0.544 micros/op; 7176.0 MB/s Baseline ------------------------------------------------ fillseq : 2.365 micros/op; 46.8 MB/s fillsync : 4240.165 micros/op; 0.0 MB/s (1000 ops) fillrandom : 3.244 micros/op; 34.1 MB/s overwrite : 4.153 micros/op; 26.6 MB/s readrandom : 4.698 micros/op; (1000000 of 1000000 found) readrandom : 4.065 micros/op; (1000000 of 1000000 found) readseq : 0.192 micros/op; 576.3 MB/s readreverse : 0.286 micros/op; 386.7 MB/s compact : 635979.000 micros/op; readrandom : 3.264 micros/op; (1000000 of 1000000 found) readseq : 0.169 micros/op; 652.8 MB/s readreverse : 0.213 micros/op; 519.5 MB/s fill100K : 1055.367 micros/op; 90.4 MB/s (1000 ops) crc32c : 1.353 micros/op; 2887.3 MB/s (4K per op) snappycomp : 3.036 micros/op; 1286.7 MB/s (output: 55.1%) snappyuncomp : 0.540 micros/op; 7238.6 MB/s PiperOrigin-RevId: 246856811
5 jaren geleden
Clean up util/coding.{h,cc}. 1) Inline EncodeFixed{32,64}(). They emit single machine instructions on 64-bit processors. 2) Remove size narrowing compiler warnings from DecodeFixed{32,64}(). 3) Add comments explaining the current state of optimizations in compilers we care about. 4) Change C-style includes, like <stdint.h>, to C++ style, like <cstdint>. 5) memcpy -> std::memcpy. The optimization comments are based on https://godbolt.org/z/RdIqS1. The missed optimization opportunities in clang have been reported as https://bugs.llvm.org/show_bug.cgi?id=41761 The change does not have significant impact on benchmarks. Results below. LevelDB: version 1.22 Date: Mon May 6 10:42:18 2019 CPU: 72 * Intel(R) Xeon(R) Gold 6154 CPU @ 3.00GHz CPUCache: 25344 KB Keys: 16 bytes each Values: 100 bytes each (50 bytes after compression) Entries: 1000000 RawSize: 110.6 MB (estimated) FileSize: 62.9 MB (estimated) With change ------------------------------------------------ fillseq : 2.327 micros/op; 47.5 MB/s fillsync : 4185.526 micros/op; 0.0 MB/s (1000 ops) fillrandom : 3.662 micros/op; 30.2 MB/s overwrite : 4.261 micros/op; 26.0 MB/s readrandom : 4.239 micros/op; (1000000 of 1000000 found) readrandom : 3.649 micros/op; (1000000 of 1000000 found) readseq : 0.174 micros/op; 636.7 MB/s readreverse : 0.271 micros/op; 408.7 MB/s compact : 570495.000 micros/op; readrandom : 2.735 micros/op; (1000000 of 1000000 found) readseq : 0.118 micros/op; 937.3 MB/s readreverse : 0.190 micros/op; 583.7 MB/s fill100K : 860.164 micros/op; 110.9 MB/s (1000 ops) crc32c : 1.131 micros/op; 3455.2 MB/s (4K per op) snappycomp : 3.034 micros/op; 1287.5 MB/s (output: 55.1%) snappyuncomp : 0.544 micros/op; 7176.0 MB/s Baseline ------------------------------------------------ fillseq : 2.365 micros/op; 46.8 MB/s fillsync : 4240.165 micros/op; 0.0 MB/s (1000 ops) fillrandom : 3.244 micros/op; 34.1 MB/s overwrite : 4.153 micros/op; 26.6 MB/s readrandom : 4.698 micros/op; (1000000 of 1000000 found) readrandom : 4.065 micros/op; (1000000 of 1000000 found) readseq : 0.192 micros/op; 576.3 MB/s readreverse : 0.286 micros/op; 386.7 MB/s compact : 635979.000 micros/op; readrandom : 3.264 micros/op; (1000000 of 1000000 found) readseq : 0.169 micros/op; 652.8 MB/s readreverse : 0.213 micros/op; 519.5 MB/s fill100K : 1055.367 micros/op; 90.4 MB/s (1000 ops) crc32c : 1.353 micros/op; 2887.3 MB/s (4K per op) snappycomp : 3.036 micros/op; 1286.7 MB/s (output: 55.1%) snappyuncomp : 0.540 micros/op; 7238.6 MB/s PiperOrigin-RevId: 246856811
5 jaren geleden
Clean up util/coding.{h,cc}. 1) Inline EncodeFixed{32,64}(). They emit single machine instructions on 64-bit processors. 2) Remove size narrowing compiler warnings from DecodeFixed{32,64}(). 3) Add comments explaining the current state of optimizations in compilers we care about. 4) Change C-style includes, like <stdint.h>, to C++ style, like <cstdint>. 5) memcpy -> std::memcpy. The optimization comments are based on https://godbolt.org/z/RdIqS1. The missed optimization opportunities in clang have been reported as https://bugs.llvm.org/show_bug.cgi?id=41761 The change does not have significant impact on benchmarks. Results below. LevelDB: version 1.22 Date: Mon May 6 10:42:18 2019 CPU: 72 * Intel(R) Xeon(R) Gold 6154 CPU @ 3.00GHz CPUCache: 25344 KB Keys: 16 bytes each Values: 100 bytes each (50 bytes after compression) Entries: 1000000 RawSize: 110.6 MB (estimated) FileSize: 62.9 MB (estimated) With change ------------------------------------------------ fillseq : 2.327 micros/op; 47.5 MB/s fillsync : 4185.526 micros/op; 0.0 MB/s (1000 ops) fillrandom : 3.662 micros/op; 30.2 MB/s overwrite : 4.261 micros/op; 26.0 MB/s readrandom : 4.239 micros/op; (1000000 of 1000000 found) readrandom : 3.649 micros/op; (1000000 of 1000000 found) readseq : 0.174 micros/op; 636.7 MB/s readreverse : 0.271 micros/op; 408.7 MB/s compact : 570495.000 micros/op; readrandom : 2.735 micros/op; (1000000 of 1000000 found) readseq : 0.118 micros/op; 937.3 MB/s readreverse : 0.190 micros/op; 583.7 MB/s fill100K : 860.164 micros/op; 110.9 MB/s (1000 ops) crc32c : 1.131 micros/op; 3455.2 MB/s (4K per op) snappycomp : 3.034 micros/op; 1287.5 MB/s (output: 55.1%) snappyuncomp : 0.544 micros/op; 7176.0 MB/s Baseline ------------------------------------------------ fillseq : 2.365 micros/op; 46.8 MB/s fillsync : 4240.165 micros/op; 0.0 MB/s (1000 ops) fillrandom : 3.244 micros/op; 34.1 MB/s overwrite : 4.153 micros/op; 26.6 MB/s readrandom : 4.698 micros/op; (1000000 of 1000000 found) readrandom : 4.065 micros/op; (1000000 of 1000000 found) readseq : 0.192 micros/op; 576.3 MB/s readreverse : 0.286 micros/op; 386.7 MB/s compact : 635979.000 micros/op; readrandom : 3.264 micros/op; (1000000 of 1000000 found) readseq : 0.169 micros/op; 652.8 MB/s readreverse : 0.213 micros/op; 519.5 MB/s fill100K : 1055.367 micros/op; 90.4 MB/s (1000 ops) crc32c : 1.353 micros/op; 2887.3 MB/s (4K per op) snappycomp : 3.036 micros/op; 1286.7 MB/s (output: 55.1%) snappyuncomp : 0.540 micros/op; 7238.6 MB/s PiperOrigin-RevId: 246856811
5 jaren geleden
Clean up util/coding.{h,cc}. 1) Inline EncodeFixed{32,64}(). They emit single machine instructions on 64-bit processors. 2) Remove size narrowing compiler warnings from DecodeFixed{32,64}(). 3) Add comments explaining the current state of optimizations in compilers we care about. 4) Change C-style includes, like <stdint.h>, to C++ style, like <cstdint>. 5) memcpy -> std::memcpy. The optimization comments are based on https://godbolt.org/z/RdIqS1. The missed optimization opportunities in clang have been reported as https://bugs.llvm.org/show_bug.cgi?id=41761 The change does not have significant impact on benchmarks. Results below. LevelDB: version 1.22 Date: Mon May 6 10:42:18 2019 CPU: 72 * Intel(R) Xeon(R) Gold 6154 CPU @ 3.00GHz CPUCache: 25344 KB Keys: 16 bytes each Values: 100 bytes each (50 bytes after compression) Entries: 1000000 RawSize: 110.6 MB (estimated) FileSize: 62.9 MB (estimated) With change ------------------------------------------------ fillseq : 2.327 micros/op; 47.5 MB/s fillsync : 4185.526 micros/op; 0.0 MB/s (1000 ops) fillrandom : 3.662 micros/op; 30.2 MB/s overwrite : 4.261 micros/op; 26.0 MB/s readrandom : 4.239 micros/op; (1000000 of 1000000 found) readrandom : 3.649 micros/op; (1000000 of 1000000 found) readseq : 0.174 micros/op; 636.7 MB/s readreverse : 0.271 micros/op; 408.7 MB/s compact : 570495.000 micros/op; readrandom : 2.735 micros/op; (1000000 of 1000000 found) readseq : 0.118 micros/op; 937.3 MB/s readreverse : 0.190 micros/op; 583.7 MB/s fill100K : 860.164 micros/op; 110.9 MB/s (1000 ops) crc32c : 1.131 micros/op; 3455.2 MB/s (4K per op) snappycomp : 3.034 micros/op; 1287.5 MB/s (output: 55.1%) snappyuncomp : 0.544 micros/op; 7176.0 MB/s Baseline ------------------------------------------------ fillseq : 2.365 micros/op; 46.8 MB/s fillsync : 4240.165 micros/op; 0.0 MB/s (1000 ops) fillrandom : 3.244 micros/op; 34.1 MB/s overwrite : 4.153 micros/op; 26.6 MB/s readrandom : 4.698 micros/op; (1000000 of 1000000 found) readrandom : 4.065 micros/op; (1000000 of 1000000 found) readseq : 0.192 micros/op; 576.3 MB/s readreverse : 0.286 micros/op; 386.7 MB/s compact : 635979.000 micros/op; readrandom : 3.264 micros/op; (1000000 of 1000000 found) readseq : 0.169 micros/op; 652.8 MB/s readreverse : 0.213 micros/op; 519.5 MB/s fill100K : 1055.367 micros/op; 90.4 MB/s (1000 ops) crc32c : 1.353 micros/op; 2887.3 MB/s (4K per op) snappycomp : 3.036 micros/op; 1286.7 MB/s (output: 55.1%) snappyuncomp : 0.540 micros/op; 7238.6 MB/s PiperOrigin-RevId: 246856811
5 jaren geleden
Clean up util/coding.{h,cc}. 1) Inline EncodeFixed{32,64}(). They emit single machine instructions on 64-bit processors. 2) Remove size narrowing compiler warnings from DecodeFixed{32,64}(). 3) Add comments explaining the current state of optimizations in compilers we care about. 4) Change C-style includes, like <stdint.h>, to C++ style, like <cstdint>. 5) memcpy -> std::memcpy. The optimization comments are based on https://godbolt.org/z/RdIqS1. The missed optimization opportunities in clang have been reported as https://bugs.llvm.org/show_bug.cgi?id=41761 The change does not have significant impact on benchmarks. Results below. LevelDB: version 1.22 Date: Mon May 6 10:42:18 2019 CPU: 72 * Intel(R) Xeon(R) Gold 6154 CPU @ 3.00GHz CPUCache: 25344 KB Keys: 16 bytes each Values: 100 bytes each (50 bytes after compression) Entries: 1000000 RawSize: 110.6 MB (estimated) FileSize: 62.9 MB (estimated) With change ------------------------------------------------ fillseq : 2.327 micros/op; 47.5 MB/s fillsync : 4185.526 micros/op; 0.0 MB/s (1000 ops) fillrandom : 3.662 micros/op; 30.2 MB/s overwrite : 4.261 micros/op; 26.0 MB/s readrandom : 4.239 micros/op; (1000000 of 1000000 found) readrandom : 3.649 micros/op; (1000000 of 1000000 found) readseq : 0.174 micros/op; 636.7 MB/s readreverse : 0.271 micros/op; 408.7 MB/s compact : 570495.000 micros/op; readrandom : 2.735 micros/op; (1000000 of 1000000 found) readseq : 0.118 micros/op; 937.3 MB/s readreverse : 0.190 micros/op; 583.7 MB/s fill100K : 860.164 micros/op; 110.9 MB/s (1000 ops) crc32c : 1.131 micros/op; 3455.2 MB/s (4K per op) snappycomp : 3.034 micros/op; 1287.5 MB/s (output: 55.1%) snappyuncomp : 0.544 micros/op; 7176.0 MB/s Baseline ------------------------------------------------ fillseq : 2.365 micros/op; 46.8 MB/s fillsync : 4240.165 micros/op; 0.0 MB/s (1000 ops) fillrandom : 3.244 micros/op; 34.1 MB/s overwrite : 4.153 micros/op; 26.6 MB/s readrandom : 4.698 micros/op; (1000000 of 1000000 found) readrandom : 4.065 micros/op; (1000000 of 1000000 found) readseq : 0.192 micros/op; 576.3 MB/s readreverse : 0.286 micros/op; 386.7 MB/s compact : 635979.000 micros/op; readrandom : 3.264 micros/op; (1000000 of 1000000 found) readseq : 0.169 micros/op; 652.8 MB/s readreverse : 0.213 micros/op; 519.5 MB/s fill100K : 1055.367 micros/op; 90.4 MB/s (1000 ops) crc32c : 1.353 micros/op; 2887.3 MB/s (4K per op) snappycomp : 3.036 micros/op; 1286.7 MB/s (output: 55.1%) snappyuncomp : 0.540 micros/op; 7238.6 MB/s PiperOrigin-RevId: 246856811
5 jaren geleden
Clean up util/coding.{h,cc}. 1) Inline EncodeFixed{32,64}(). They emit single machine instructions on 64-bit processors. 2) Remove size narrowing compiler warnings from DecodeFixed{32,64}(). 3) Add comments explaining the current state of optimizations in compilers we care about. 4) Change C-style includes, like <stdint.h>, to C++ style, like <cstdint>. 5) memcpy -> std::memcpy. The optimization comments are based on https://godbolt.org/z/RdIqS1. The missed optimization opportunities in clang have been reported as https://bugs.llvm.org/show_bug.cgi?id=41761 The change does not have significant impact on benchmarks. Results below. LevelDB: version 1.22 Date: Mon May 6 10:42:18 2019 CPU: 72 * Intel(R) Xeon(R) Gold 6154 CPU @ 3.00GHz CPUCache: 25344 KB Keys: 16 bytes each Values: 100 bytes each (50 bytes after compression) Entries: 1000000 RawSize: 110.6 MB (estimated) FileSize: 62.9 MB (estimated) With change ------------------------------------------------ fillseq : 2.327 micros/op; 47.5 MB/s fillsync : 4185.526 micros/op; 0.0 MB/s (1000 ops) fillrandom : 3.662 micros/op; 30.2 MB/s overwrite : 4.261 micros/op; 26.0 MB/s readrandom : 4.239 micros/op; (1000000 of 1000000 found) readrandom : 3.649 micros/op; (1000000 of 1000000 found) readseq : 0.174 micros/op; 636.7 MB/s readreverse : 0.271 micros/op; 408.7 MB/s compact : 570495.000 micros/op; readrandom : 2.735 micros/op; (1000000 of 1000000 found) readseq : 0.118 micros/op; 937.3 MB/s readreverse : 0.190 micros/op; 583.7 MB/s fill100K : 860.164 micros/op; 110.9 MB/s (1000 ops) crc32c : 1.131 micros/op; 3455.2 MB/s (4K per op) snappycomp : 3.034 micros/op; 1287.5 MB/s (output: 55.1%) snappyuncomp : 0.544 micros/op; 7176.0 MB/s Baseline ------------------------------------------------ fillseq : 2.365 micros/op; 46.8 MB/s fillsync : 4240.165 micros/op; 0.0 MB/s (1000 ops) fillrandom : 3.244 micros/op; 34.1 MB/s overwrite : 4.153 micros/op; 26.6 MB/s readrandom : 4.698 micros/op; (1000000 of 1000000 found) readrandom : 4.065 micros/op; (1000000 of 1000000 found) readseq : 0.192 micros/op; 576.3 MB/s readreverse : 0.286 micros/op; 386.7 MB/s compact : 635979.000 micros/op; readrandom : 3.264 micros/op; (1000000 of 1000000 found) readseq : 0.169 micros/op; 652.8 MB/s readreverse : 0.213 micros/op; 519.5 MB/s fill100K : 1055.367 micros/op; 90.4 MB/s (1000 ops) crc32c : 1.353 micros/op; 2887.3 MB/s (4K per op) snappycomp : 3.036 micros/op; 1286.7 MB/s (output: 55.1%) snappyuncomp : 0.540 micros/op; 7238.6 MB/s PiperOrigin-RevId: 246856811
5 jaren geleden
Clean up util/coding.{h,cc}. 1) Inline EncodeFixed{32,64}(). They emit single machine instructions on 64-bit processors. 2) Remove size narrowing compiler warnings from DecodeFixed{32,64}(). 3) Add comments explaining the current state of optimizations in compilers we care about. 4) Change C-style includes, like <stdint.h>, to C++ style, like <cstdint>. 5) memcpy -> std::memcpy. The optimization comments are based on https://godbolt.org/z/RdIqS1. The missed optimization opportunities in clang have been reported as https://bugs.llvm.org/show_bug.cgi?id=41761 The change does not have significant impact on benchmarks. Results below. LevelDB: version 1.22 Date: Mon May 6 10:42:18 2019 CPU: 72 * Intel(R) Xeon(R) Gold 6154 CPU @ 3.00GHz CPUCache: 25344 KB Keys: 16 bytes each Values: 100 bytes each (50 bytes after compression) Entries: 1000000 RawSize: 110.6 MB (estimated) FileSize: 62.9 MB (estimated) With change ------------------------------------------------ fillseq : 2.327 micros/op; 47.5 MB/s fillsync : 4185.526 micros/op; 0.0 MB/s (1000 ops) fillrandom : 3.662 micros/op; 30.2 MB/s overwrite : 4.261 micros/op; 26.0 MB/s readrandom : 4.239 micros/op; (1000000 of 1000000 found) readrandom : 3.649 micros/op; (1000000 of 1000000 found) readseq : 0.174 micros/op; 636.7 MB/s readreverse : 0.271 micros/op; 408.7 MB/s compact : 570495.000 micros/op; readrandom : 2.735 micros/op; (1000000 of 1000000 found) readseq : 0.118 micros/op; 937.3 MB/s readreverse : 0.190 micros/op; 583.7 MB/s fill100K : 860.164 micros/op; 110.9 MB/s (1000 ops) crc32c : 1.131 micros/op; 3455.2 MB/s (4K per op) snappycomp : 3.034 micros/op; 1287.5 MB/s (output: 55.1%) snappyuncomp : 0.544 micros/op; 7176.0 MB/s Baseline ------------------------------------------------ fillseq : 2.365 micros/op; 46.8 MB/s fillsync : 4240.165 micros/op; 0.0 MB/s (1000 ops) fillrandom : 3.244 micros/op; 34.1 MB/s overwrite : 4.153 micros/op; 26.6 MB/s readrandom : 4.698 micros/op; (1000000 of 1000000 found) readrandom : 4.065 micros/op; (1000000 of 1000000 found) readseq : 0.192 micros/op; 576.3 MB/s readreverse : 0.286 micros/op; 386.7 MB/s compact : 635979.000 micros/op; readrandom : 3.264 micros/op; (1000000 of 1000000 found) readseq : 0.169 micros/op; 652.8 MB/s readreverse : 0.213 micros/op; 519.5 MB/s fill100K : 1055.367 micros/op; 90.4 MB/s (1000 ops) crc32c : 1.353 micros/op; 2887.3 MB/s (4K per op) snappycomp : 3.036 micros/op; 1286.7 MB/s (output: 55.1%) snappyuncomp : 0.540 micros/op; 7238.6 MB/s PiperOrigin-RevId: 246856811
5 jaren geleden
Clean up util/coding.{h,cc}. 1) Inline EncodeFixed{32,64}(). They emit single machine instructions on 64-bit processors. 2) Remove size narrowing compiler warnings from DecodeFixed{32,64}(). 3) Add comments explaining the current state of optimizations in compilers we care about. 4) Change C-style includes, like <stdint.h>, to C++ style, like <cstdint>. 5) memcpy -> std::memcpy. The optimization comments are based on https://godbolt.org/z/RdIqS1. The missed optimization opportunities in clang have been reported as https://bugs.llvm.org/show_bug.cgi?id=41761 The change does not have significant impact on benchmarks. Results below. LevelDB: version 1.22 Date: Mon May 6 10:42:18 2019 CPU: 72 * Intel(R) Xeon(R) Gold 6154 CPU @ 3.00GHz CPUCache: 25344 KB Keys: 16 bytes each Values: 100 bytes each (50 bytes after compression) Entries: 1000000 RawSize: 110.6 MB (estimated) FileSize: 62.9 MB (estimated) With change ------------------------------------------------ fillseq : 2.327 micros/op; 47.5 MB/s fillsync : 4185.526 micros/op; 0.0 MB/s (1000 ops) fillrandom : 3.662 micros/op; 30.2 MB/s overwrite : 4.261 micros/op; 26.0 MB/s readrandom : 4.239 micros/op; (1000000 of 1000000 found) readrandom : 3.649 micros/op; (1000000 of 1000000 found) readseq : 0.174 micros/op; 636.7 MB/s readreverse : 0.271 micros/op; 408.7 MB/s compact : 570495.000 micros/op; readrandom : 2.735 micros/op; (1000000 of 1000000 found) readseq : 0.118 micros/op; 937.3 MB/s readreverse : 0.190 micros/op; 583.7 MB/s fill100K : 860.164 micros/op; 110.9 MB/s (1000 ops) crc32c : 1.131 micros/op; 3455.2 MB/s (4K per op) snappycomp : 3.034 micros/op; 1287.5 MB/s (output: 55.1%) snappyuncomp : 0.544 micros/op; 7176.0 MB/s Baseline ------------------------------------------------ fillseq : 2.365 micros/op; 46.8 MB/s fillsync : 4240.165 micros/op; 0.0 MB/s (1000 ops) fillrandom : 3.244 micros/op; 34.1 MB/s overwrite : 4.153 micros/op; 26.6 MB/s readrandom : 4.698 micros/op; (1000000 of 1000000 found) readrandom : 4.065 micros/op; (1000000 of 1000000 found) readseq : 0.192 micros/op; 576.3 MB/s readreverse : 0.286 micros/op; 386.7 MB/s compact : 635979.000 micros/op; readrandom : 3.264 micros/op; (1000000 of 1000000 found) readseq : 0.169 micros/op; 652.8 MB/s readreverse : 0.213 micros/op; 519.5 MB/s fill100K : 1055.367 micros/op; 90.4 MB/s (1000 ops) crc32c : 1.353 micros/op; 2887.3 MB/s (4K per op) snappycomp : 3.036 micros/op; 1286.7 MB/s (output: 55.1%) snappyuncomp : 0.540 micros/op; 7238.6 MB/s PiperOrigin-RevId: 246856811
5 jaren geleden
Clean up util/coding.{h,cc}. 1) Inline EncodeFixed{32,64}(). They emit single machine instructions on 64-bit processors. 2) Remove size narrowing compiler warnings from DecodeFixed{32,64}(). 3) Add comments explaining the current state of optimizations in compilers we care about. 4) Change C-style includes, like <stdint.h>, to C++ style, like <cstdint>. 5) memcpy -> std::memcpy. The optimization comments are based on https://godbolt.org/z/RdIqS1. The missed optimization opportunities in clang have been reported as https://bugs.llvm.org/show_bug.cgi?id=41761 The change does not have significant impact on benchmarks. Results below. LevelDB: version 1.22 Date: Mon May 6 10:42:18 2019 CPU: 72 * Intel(R) Xeon(R) Gold 6154 CPU @ 3.00GHz CPUCache: 25344 KB Keys: 16 bytes each Values: 100 bytes each (50 bytes after compression) Entries: 1000000 RawSize: 110.6 MB (estimated) FileSize: 62.9 MB (estimated) With change ------------------------------------------------ fillseq : 2.327 micros/op; 47.5 MB/s fillsync : 4185.526 micros/op; 0.0 MB/s (1000 ops) fillrandom : 3.662 micros/op; 30.2 MB/s overwrite : 4.261 micros/op; 26.0 MB/s readrandom : 4.239 micros/op; (1000000 of 1000000 found) readrandom : 3.649 micros/op; (1000000 of 1000000 found) readseq : 0.174 micros/op; 636.7 MB/s readreverse : 0.271 micros/op; 408.7 MB/s compact : 570495.000 micros/op; readrandom : 2.735 micros/op; (1000000 of 1000000 found) readseq : 0.118 micros/op; 937.3 MB/s readreverse : 0.190 micros/op; 583.7 MB/s fill100K : 860.164 micros/op; 110.9 MB/s (1000 ops) crc32c : 1.131 micros/op; 3455.2 MB/s (4K per op) snappycomp : 3.034 micros/op; 1287.5 MB/s (output: 55.1%) snappyuncomp : 0.544 micros/op; 7176.0 MB/s Baseline ------------------------------------------------ fillseq : 2.365 micros/op; 46.8 MB/s fillsync : 4240.165 micros/op; 0.0 MB/s (1000 ops) fillrandom : 3.244 micros/op; 34.1 MB/s overwrite : 4.153 micros/op; 26.6 MB/s readrandom : 4.698 micros/op; (1000000 of 1000000 found) readrandom : 4.065 micros/op; (1000000 of 1000000 found) readseq : 0.192 micros/op; 576.3 MB/s readreverse : 0.286 micros/op; 386.7 MB/s compact : 635979.000 micros/op; readrandom : 3.264 micros/op; (1000000 of 1000000 found) readseq : 0.169 micros/op; 652.8 MB/s readreverse : 0.213 micros/op; 519.5 MB/s fill100K : 1055.367 micros/op; 90.4 MB/s (1000 ops) crc32c : 1.353 micros/op; 2887.3 MB/s (4K per op) snappycomp : 3.036 micros/op; 1286.7 MB/s (output: 55.1%) snappyuncomp : 0.540 micros/op; 7238.6 MB/s PiperOrigin-RevId: 246856811
5 jaren geleden
  1. // Copyright (c) 2011 The LevelDB Authors. All rights reserved.
  2. // Use of this source code is governed by a BSD-style license that can be
  3. // found in the LICENSE file. See the AUTHORS file for names of contributors.
  4. //
  5. // Endian-neutral encoding:
  6. // * Fixed-length numbers are encoded with least-significant byte first
  7. // * In addition we support variable length "varint" encoding
  8. // * Strings are encoded prefixed by their length in varint format
  9. #ifndef STORAGE_LEVELDB_UTIL_CODING_H_
  10. #define STORAGE_LEVELDB_UTIL_CODING_H_
  11. #include <cstdint>
  12. #include <cstring>
  13. #include <string>
  14. #include "leveldb/slice.h"
  15. #include "port/port.h"
  16. namespace leveldb {
  17. // Standard Put... routines append to a string
  18. void PutFixed32(std::string* dst, uint32_t value);
  19. void PutFixed64(std::string* dst, uint64_t value);
  20. void PutVarint32(std::string* dst, uint32_t value);
  21. void PutVarint64(std::string* dst, uint64_t value);
  22. void PutLengthPrefixedSlice(std::string* dst, const Slice& value);
  23. // Standard Get... routines parse a value from the beginning of a Slice
  24. // and advance the slice past the parsed value.
  25. bool GetVarint32(Slice* input, uint32_t* value);
  26. bool GetVarint64(Slice* input, uint64_t* value);
  27. bool GetLengthPrefixedSlice(Slice* input, Slice* result);
  28. // Pointer-based variants of GetVarint... These either store a value
  29. // in *v and return a pointer just past the parsed value, or return
  30. // nullptr on error. These routines only look at bytes in the range
  31. // [p..limit-1]
  32. const char* GetVarint32Ptr(const char* p, const char* limit, uint32_t* v);
  33. const char* GetVarint64Ptr(const char* p, const char* limit, uint64_t* v);
  34. // Returns the length of the varint32 or varint64 encoding of "v"
  35. int VarintLength(uint64_t v);
  36. // Lower-level versions of Put... that write directly into a character buffer
  37. // and return a pointer just past the last byte written.
  38. // REQUIRES: dst has enough space for the value being written
  39. char* EncodeVarint32(char* dst, uint32_t value);
  40. char* EncodeVarint64(char* dst, uint64_t value);
  41. // TODO(costan): Remove port::kLittleEndian and the fast paths based on
  42. // std::memcpy when clang learns to optimize the generic code, as
  43. // described in https://bugs.llvm.org/show_bug.cgi?id=41761
  44. //
  45. // The platform-independent code in DecodeFixed{32,64}() gets optimized to mov
  46. // on x86 and ldr on ARM64, by both clang and gcc. However, only gcc optimizes
  47. // the platform-independent code in EncodeFixed{32,64}() to mov / str.
  48. // Lower-level versions of Put... that write directly into a character buffer
  49. // REQUIRES: dst has enough space for the value being written
  50. inline void EncodeFixed32(char* dst, uint32_t value) {
  51. uint8_t* const buffer = reinterpret_cast<uint8_t*>(dst);
  52. if (port::kLittleEndian) {
  53. // Fast path for little-endian CPUs. All major compilers optimize this to a
  54. // single mov (x86_64) / str (ARM) instruction.
  55. std::memcpy(buffer, &value, sizeof(uint32_t));
  56. return;
  57. }
  58. // Platform-independent code.
  59. // Currently, only gcc optimizes this to a single mov / str instruction.
  60. buffer[0] = static_cast<uint8_t>(value);
  61. buffer[1] = static_cast<uint8_t>(value >> 8);
  62. buffer[2] = static_cast<uint8_t>(value >> 16);
  63. buffer[3] = static_cast<uint8_t>(value >> 24);
  64. }
  65. inline void EncodeFixed64(char* dst, uint64_t value) {
  66. uint8_t* const buffer = reinterpret_cast<uint8_t*>(dst);
  67. if (port::kLittleEndian) {
  68. // Fast path for little-endian CPUs. All major compilers optimize this to a
  69. // single mov (x86_64) / str (ARM) instruction.
  70. std::memcpy(buffer, &value, sizeof(uint64_t));
  71. return;
  72. }
  73. // Platform-independent code.
  74. // Currently, only gcc optimizes this to a single mov / str instruction.
  75. buffer[0] = static_cast<uint8_t>(value);
  76. buffer[1] = static_cast<uint8_t>(value >> 8);
  77. buffer[2] = static_cast<uint8_t>(value >> 16);
  78. buffer[3] = static_cast<uint8_t>(value >> 24);
  79. buffer[4] = static_cast<uint8_t>(value >> 32);
  80. buffer[5] = static_cast<uint8_t>(value >> 40);
  81. buffer[6] = static_cast<uint8_t>(value >> 48);
  82. buffer[7] = static_cast<uint8_t>(value >> 56);
  83. }
  84. // Lower-level versions of Get... that read directly from a character buffer
  85. // without any bounds checking.
  86. inline uint32_t DecodeFixed32(const char* ptr) {
  87. const uint8_t* const buffer = reinterpret_cast<const uint8_t*>(ptr);
  88. if (port::kLittleEndian) {
  89. // Fast path for little-endian CPUs. All major compilers optimize this to a
  90. // single mov (x86_64) / ldr (ARM) instruction.
  91. uint32_t result;
  92. std::memcpy(&result, buffer, sizeof(uint32_t));
  93. return result;
  94. }
  95. // Platform-independent code.
  96. // Clang and gcc optimize this to a single mov / ldr instruction.
  97. return (static_cast<uint32_t>(buffer[0])) |
  98. (static_cast<uint32_t>(buffer[1]) << 8) |
  99. (static_cast<uint32_t>(buffer[2]) << 16) |
  100. (static_cast<uint32_t>(buffer[3]) << 24);
  101. }
  102. inline uint64_t DecodeFixed64(const char* ptr) {
  103. const uint8_t* const buffer = reinterpret_cast<const uint8_t*>(ptr);
  104. if (port::kLittleEndian) {
  105. // Fast path for little-endian CPUs. All major compilers optimize this to a
  106. // single mov (x86_64) / ldr (ARM) instruction.
  107. uint64_t result;
  108. std::memcpy(&result, buffer, sizeof(uint64_t));
  109. return result;
  110. }
  111. // Platform-independent code.
  112. // Clang and gcc optimize this to a single mov / ldr instruction.
  113. return (static_cast<uint64_t>(buffer[0])) |
  114. (static_cast<uint64_t>(buffer[1]) << 8) |
  115. (static_cast<uint64_t>(buffer[2]) << 16) |
  116. (static_cast<uint64_t>(buffer[3]) << 24) |
  117. (static_cast<uint64_t>(buffer[4]) << 32) |
  118. (static_cast<uint64_t>(buffer[5]) << 40) |
  119. (static_cast<uint64_t>(buffer[6]) << 48) |
  120. (static_cast<uint64_t>(buffer[7]) << 56);
  121. }
  122. // Internal routine for use by fallback path of GetVarint32Ptr
  123. const char* GetVarint32PtrFallback(const char* p, const char* limit,
  124. uint32_t* value);
  125. inline const char* GetVarint32Ptr(const char* p, const char* limit,
  126. uint32_t* value) {
  127. if (p < limit) {
  128. uint32_t result = *(reinterpret_cast<const unsigned char*>(p));
  129. if ((result & 128) == 0) {
  130. *value = result;
  131. return p + 1;
  132. }
  133. }
  134. return GetVarint32PtrFallback(p, limit, value);
  135. }
  136. } // namespace leveldb
  137. #endif // STORAGE_LEVELDB_UTIL_CODING_H_