作者: 韩晨旭 10225101440 李畅 10225102463

218 lines
7.4 KiB

Release 1.18 Changes are: * Update version number to 1.18 * Replace the basic fprintf call with a call to fwrite in order to work around the apparent compiler optimization/rewrite failure that we are seeing with the new toolchain/iOS SDKs provided with Xcode6 and iOS8. * Fix ALL the header guards. * Createed a README.md with the LevelDB project description. * A new CONTRIBUTING file. * Don't implicitly convert uint64_t to size_t or int. Either preserve it as uint64_t, or explicitly cast. This fixes MSVC warnings about possible value truncation when compiling this code in Chromium. * Added a DumpFile() library function that encapsulates the guts of the "leveldbutil dump" command. This will allow clients to dump data to their log files instead of stdout. It will also allow clients to supply their own environment. * leveldb: Remove unused function 'ConsumeChar'. * leveldbutil: Remove unused member variables from WriteBatchItemPrinter. * OpenBSD, NetBSD and DragonflyBSD have _LITTLE_ENDIAN, so define PLATFORM_IS_LITTLE_ENDIAN like on FreeBSD. This fixes: * issue #143 * issue #198 * issue #249 * Switch from <cstdatomic> to <atomic>. The former never made it into the standard and doesn't exist in modern gcc versions at all. The later contains everything that leveldb was using from the former. This problem was noticed when porting to Portable Native Client where no memory barrier is defined. The fact that <cstdatomic> is missing normally goes unnoticed since memory barriers are defined for most architectures. * Make Hash() treat its input as unsigned. Before this change LevelDB files from platforms with different signedness of char were not compatible. This change fixes: issue #243 * Verify checksums of index/meta/filter blocks when paranoid_checks set. * Invoke all tools for iOS with xcrun. (This was causing problems with the new XCode 5.1.1 image on pulse.) * include <sys/stat.h> only once, and fix the following linter warning: "Found C system header after C++ system header" * When encountering a corrupted table file, return Status::Corruption instead of Status::InvalidArgument. * Support cygwin as build platform, patch is from https://code.google.com/p/leveldb/issues/detail?id=188 * Fix typo, merge patch from https://code.google.com/p/leveldb/issues/detail?id=159 * Fix typos and comments, and address the following two issues: * issue #166 * issue #241 * Add missing db synchronize after "fillseq" in the benchmark. * Removed unused variable in SeekRandom: value (issue #201)
пре 10 година
Release 1.18 Changes are: * Update version number to 1.18 * Replace the basic fprintf call with a call to fwrite in order to work around the apparent compiler optimization/rewrite failure that we are seeing with the new toolchain/iOS SDKs provided with Xcode6 and iOS8. * Fix ALL the header guards. * Createed a README.md with the LevelDB project description. * A new CONTRIBUTING file. * Don't implicitly convert uint64_t to size_t or int. Either preserve it as uint64_t, or explicitly cast. This fixes MSVC warnings about possible value truncation when compiling this code in Chromium. * Added a DumpFile() library function that encapsulates the guts of the "leveldbutil dump" command. This will allow clients to dump data to their log files instead of stdout. It will also allow clients to supply their own environment. * leveldb: Remove unused function 'ConsumeChar'. * leveldbutil: Remove unused member variables from WriteBatchItemPrinter. * OpenBSD, NetBSD and DragonflyBSD have _LITTLE_ENDIAN, so define PLATFORM_IS_LITTLE_ENDIAN like on FreeBSD. This fixes: * issue #143 * issue #198 * issue #249 * Switch from <cstdatomic> to <atomic>. The former never made it into the standard and doesn't exist in modern gcc versions at all. The later contains everything that leveldb was using from the former. This problem was noticed when porting to Portable Native Client where no memory barrier is defined. The fact that <cstdatomic> is missing normally goes unnoticed since memory barriers are defined for most architectures. * Make Hash() treat its input as unsigned. Before this change LevelDB files from platforms with different signedness of char were not compatible. This change fixes: issue #243 * Verify checksums of index/meta/filter blocks when paranoid_checks set. * Invoke all tools for iOS with xcrun. (This was causing problems with the new XCode 5.1.1 image on pulse.) * include <sys/stat.h> only once, and fix the following linter warning: "Found C system header after C++ system header" * When encountering a corrupted table file, return Status::Corruption instead of Status::InvalidArgument. * Support cygwin as build platform, patch is from https://code.google.com/p/leveldb/issues/detail?id=188 * Fix typo, merge patch from https://code.google.com/p/leveldb/issues/detail?id=159 * Fix typos and comments, and address the following two issues: * issue #166 * issue #241 * Add missing db synchronize after "fillseq" in the benchmark. * Removed unused variable in SeekRandom: value (issue #201)
пре 10 година
  1. // Copyright (c) 2011 The LevelDB Authors. All rights reserved.
  2. // Use of this source code is governed by a BSD-style license that can be
  3. // found in the LICENSE file. See the AUTHORS file for names of contributors.
  4. #ifndef STORAGE_LEVELDB_DB_DBFORMAT_H_
  5. #define STORAGE_LEVELDB_DB_DBFORMAT_H_
  6. #include <stdio.h>
  7. #include "leveldb/comparator.h"
  8. #include "leveldb/db.h"
  9. #include "leveldb/filter_policy.h"
  10. #include "leveldb/slice.h"
  11. #include "leveldb/table_builder.h"
  12. #include "util/coding.h"
  13. #include "util/logging.h"
  14. namespace leveldb {
  15. // Grouping of constants. We may want to make some of these
  16. // parameters set via options.
  17. namespace config {
  18. static const int kNumLevels = 7;
  19. // Level-0 compaction is started when we hit this many files.
  20. static const int kL0_CompactionTrigger = 4;
  21. // Soft limit on number of level-0 files. We slow down writes at this point.
  22. static const int kL0_SlowdownWritesTrigger = 8;
  23. // Maximum number of level-0 files. We stop writes at this point.
  24. static const int kL0_StopWritesTrigger = 12;
  25. // Maximum level to which a new compacted memtable is pushed if it
  26. // does not create overlap. We try to push to level 2 to avoid the
  27. // relatively expensive level 0=>1 compactions and to avoid some
  28. // expensive manifest file operations. We do not push all the way to
  29. // the largest level since that can generate a lot of wasted disk
  30. // space if the same key space is being repeatedly overwritten.
  31. static const int kMaxMemCompactLevel = 2;
  32. // Approximate gap in bytes between samples of data read during iteration.
  33. static const int kReadBytesPeriod = 1048576;
  34. } // namespace config
  35. class InternalKey;
  36. // Value types encoded as the last component of internal keys.
  37. // DO NOT CHANGE THESE ENUM VALUES: they are embedded in the on-disk
  38. // data structures.
  39. enum ValueType { kTypeDeletion = 0x0, kTypeValue = 0x1 };
  40. // kValueTypeForSeek defines the ValueType that should be passed when
  41. // constructing a ParsedInternalKey object for seeking to a particular
  42. // sequence number (since we sort sequence numbers in decreasing order
  43. // and the value type is embedded as the low 8 bits in the sequence
  44. // number in internal keys, we need to use the highest-numbered
  45. // ValueType, not the lowest).
  46. static const ValueType kValueTypeForSeek = kTypeValue;
  47. typedef uint64_t SequenceNumber;
  48. // We leave eight bits empty at the bottom so a type and sequence#
  49. // can be packed together into 64-bits.
  50. static const SequenceNumber kMaxSequenceNumber = ((0x1ull << 56) - 1);
  51. struct ParsedInternalKey {
  52. Slice user_key;
  53. SequenceNumber sequence;
  54. ValueType type;
  55. ParsedInternalKey() {} // Intentionally left uninitialized (for speed)
  56. ParsedInternalKey(const Slice& u, const SequenceNumber& seq, ValueType t)
  57. : user_key(u), sequence(seq), type(t) {}
  58. std::string DebugString() const;
  59. };
  60. // Return the length of the encoding of "key".
  61. inline size_t InternalKeyEncodingLength(const ParsedInternalKey& key) {
  62. return key.user_key.size() + 8;
  63. }
  64. // Append the serialization of "key" to *result.
  65. void AppendInternalKey(std::string* result, const ParsedInternalKey& key);
  66. // Attempt to parse an internal key from "internal_key". On success,
  67. // stores the parsed data in "*result", and returns true.
  68. //
  69. // On error, returns false, leaves "*result" in an undefined state.
  70. bool ParseInternalKey(const Slice& internal_key, ParsedInternalKey* result);
  71. // Returns the user key portion of an internal key.
  72. inline Slice ExtractUserKey(const Slice& internal_key) {
  73. assert(internal_key.size() >= 8);
  74. return Slice(internal_key.data(), internal_key.size() - 8);
  75. }
  76. // A comparator for internal keys that uses a specified comparator for
  77. // the user key portion and breaks ties by decreasing sequence number.
  78. class InternalKeyComparator : public Comparator {
  79. private:
  80. const Comparator* user_comparator_;
  81. public:
  82. explicit InternalKeyComparator(const Comparator* c) : user_comparator_(c) {}
  83. virtual const char* Name() const;
  84. virtual int Compare(const Slice& a, const Slice& b) const;
  85. virtual void FindShortestSeparator(std::string* start,
  86. const Slice& limit) const;
  87. virtual void FindShortSuccessor(std::string* key) const;
  88. const Comparator* user_comparator() const { return user_comparator_; }
  89. int Compare(const InternalKey& a, const InternalKey& b) const;
  90. };
  91. // Filter policy wrapper that converts from internal keys to user keys
  92. class InternalFilterPolicy : public FilterPolicy {
  93. private:
  94. const FilterPolicy* const user_policy_;
  95. public:
  96. explicit InternalFilterPolicy(const FilterPolicy* p) : user_policy_(p) {}
  97. virtual const char* Name() const;
  98. virtual void CreateFilter(const Slice* keys, int n, std::string* dst) const;
  99. virtual bool KeyMayMatch(const Slice& key, const Slice& filter) const;
  100. };
  101. // Modules in this directory should keep internal keys wrapped inside
  102. // the following class instead of plain strings so that we do not
  103. // incorrectly use string comparisons instead of an InternalKeyComparator.
  104. class InternalKey {
  105. private:
  106. std::string rep_;
  107. public:
  108. InternalKey() {} // Leave rep_ as empty to indicate it is invalid
  109. InternalKey(const Slice& user_key, SequenceNumber s, ValueType t) {
  110. AppendInternalKey(&rep_, ParsedInternalKey(user_key, s, t));
  111. }
  112. void DecodeFrom(const Slice& s) { rep_.assign(s.data(), s.size()); }
  113. Slice Encode() const {
  114. assert(!rep_.empty());
  115. return rep_;
  116. }
  117. Slice user_key() const { return ExtractUserKey(rep_); }
  118. void SetFrom(const ParsedInternalKey& p) {
  119. rep_.clear();
  120. AppendInternalKey(&rep_, p);
  121. }
  122. void Clear() { rep_.clear(); }
  123. std::string DebugString() const;
  124. };
  125. inline int InternalKeyComparator::Compare(const InternalKey& a,
  126. const InternalKey& b) const {
  127. return Compare(a.Encode(), b.Encode());
  128. }
  129. inline bool ParseInternalKey(const Slice& internal_key,
  130. ParsedInternalKey* result) {
  131. const size_t n = internal_key.size();
  132. if (n < 8) return false;
  133. uint64_t num = DecodeFixed64(internal_key.data() + n - 8);
  134. unsigned char c = num & 0xff;
  135. result->sequence = num >> 8;
  136. result->type = static_cast<ValueType>(c);
  137. result->user_key = Slice(internal_key.data(), n - 8);
  138. return (c <= static_cast<unsigned char>(kTypeValue));
  139. }
  140. // A helper class useful for DBImpl::Get()
  141. class LookupKey {
  142. public:
  143. // Initialize *this for looking up user_key at a snapshot with
  144. // the specified sequence number.
  145. LookupKey(const Slice& user_key, SequenceNumber sequence);
  146. LookupKey(const LookupKey&) = delete;
  147. LookupKey& operator=(const LookupKey&) = delete;
  148. ~LookupKey();
  149. // Return a key suitable for lookup in a MemTable.
  150. Slice memtable_key() const { return Slice(start_, end_ - start_); }
  151. // Return an internal key (suitable for passing to an internal iterator)
  152. Slice internal_key() const { return Slice(kstart_, end_ - kstart_); }
  153. // Return the user key
  154. Slice user_key() const { return Slice(kstart_, end_ - kstart_ - 8); }
  155. private:
  156. // We construct a char array of the form:
  157. // klength varint32 <-- start_
  158. // userkey char[klength] <-- kstart_
  159. // tag uint64
  160. // <-- end_
  161. // The array is a suitable MemTable key.
  162. // The suffix starting with "userkey" can be used as an InternalKey.
  163. const char* start_;
  164. const char* kstart_;
  165. const char* end_;
  166. char space_[200]; // Avoid allocation for short keys
  167. };
  168. inline LookupKey::~LookupKey() {
  169. if (start_ != space_) delete[] start_;
  170. }
  171. } // namespace leveldb
  172. #endif // STORAGE_LEVELDB_DB_DBFORMAT_H_