作者: 韩晨旭 10225101440 李畅 10225102463

228 lignes
6.2 KiB

Release 1.18 Changes are: * Update version number to 1.18 * Replace the basic fprintf call with a call to fwrite in order to work around the apparent compiler optimization/rewrite failure that we are seeing with the new toolchain/iOS SDKs provided with Xcode6 and iOS8. * Fix ALL the header guards. * Createed a README.md with the LevelDB project description. * A new CONTRIBUTING file. * Don't implicitly convert uint64_t to size_t or int. Either preserve it as uint64_t, or explicitly cast. This fixes MSVC warnings about possible value truncation when compiling this code in Chromium. * Added a DumpFile() library function that encapsulates the guts of the "leveldbutil dump" command. This will allow clients to dump data to their log files instead of stdout. It will also allow clients to supply their own environment. * leveldb: Remove unused function 'ConsumeChar'. * leveldbutil: Remove unused member variables from WriteBatchItemPrinter. * OpenBSD, NetBSD and DragonflyBSD have _LITTLE_ENDIAN, so define PLATFORM_IS_LITTLE_ENDIAN like on FreeBSD. This fixes: * issue #143 * issue #198 * issue #249 * Switch from <cstdatomic> to <atomic>. The former never made it into the standard and doesn't exist in modern gcc versions at all. The later contains everything that leveldb was using from the former. This problem was noticed when porting to Portable Native Client where no memory barrier is defined. The fact that <cstdatomic> is missing normally goes unnoticed since memory barriers are defined for most architectures. * Make Hash() treat its input as unsigned. Before this change LevelDB files from platforms with different signedness of char were not compatible. This change fixes: issue #243 * Verify checksums of index/meta/filter blocks when paranoid_checks set. * Invoke all tools for iOS with xcrun. (This was causing problems with the new XCode 5.1.1 image on pulse.) * include <sys/stat.h> only once, and fix the following linter warning: "Found C system header after C++ system header" * When encountering a corrupted table file, return Status::Corruption instead of Status::InvalidArgument. * Support cygwin as build platform, patch is from https://code.google.com/p/leveldb/issues/detail?id=188 * Fix typo, merge patch from https://code.google.com/p/leveldb/issues/detail?id=159 * Fix typos and comments, and address the following two issues: * issue #166 * issue #241 * Add missing db synchronize after "fillseq" in the benchmark. * Removed unused variable in SeekRandom: value (issue #201)
il y a 10 ans
Release 1.18 Changes are: * Update version number to 1.18 * Replace the basic fprintf call with a call to fwrite in order to work around the apparent compiler optimization/rewrite failure that we are seeing with the new toolchain/iOS SDKs provided with Xcode6 and iOS8. * Fix ALL the header guards. * Createed a README.md with the LevelDB project description. * A new CONTRIBUTING file. * Don't implicitly convert uint64_t to size_t or int. Either preserve it as uint64_t, or explicitly cast. This fixes MSVC warnings about possible value truncation when compiling this code in Chromium. * Added a DumpFile() library function that encapsulates the guts of the "leveldbutil dump" command. This will allow clients to dump data to their log files instead of stdout. It will also allow clients to supply their own environment. * leveldb: Remove unused function 'ConsumeChar'. * leveldbutil: Remove unused member variables from WriteBatchItemPrinter. * OpenBSD, NetBSD and DragonflyBSD have _LITTLE_ENDIAN, so define PLATFORM_IS_LITTLE_ENDIAN like on FreeBSD. This fixes: * issue #143 * issue #198 * issue #249 * Switch from <cstdatomic> to <atomic>. The former never made it into the standard and doesn't exist in modern gcc versions at all. The later contains everything that leveldb was using from the former. This problem was noticed when porting to Portable Native Client where no memory barrier is defined. The fact that <cstdatomic> is missing normally goes unnoticed since memory barriers are defined for most architectures. * Make Hash() treat its input as unsigned. Before this change LevelDB files from platforms with different signedness of char were not compatible. This change fixes: issue #243 * Verify checksums of index/meta/filter blocks when paranoid_checks set. * Invoke all tools for iOS with xcrun. (This was causing problems with the new XCode 5.1.1 image on pulse.) * include <sys/stat.h> only once, and fix the following linter warning: "Found C system header after C++ system header" * When encountering a corrupted table file, return Status::Corruption instead of Status::InvalidArgument. * Support cygwin as build platform, patch is from https://code.google.com/p/leveldb/issues/detail?id=188 * Fix typo, merge patch from https://code.google.com/p/leveldb/issues/detail?id=159 * Fix typos and comments, and address the following two issues: * issue #166 * issue #241 * Add missing db synchronize after "fillseq" in the benchmark. * Removed unused variable in SeekRandom: value (issue #201)
il y a 10 ans
Release 1.18 Changes are: * Update version number to 1.18 * Replace the basic fprintf call with a call to fwrite in order to work around the apparent compiler optimization/rewrite failure that we are seeing with the new toolchain/iOS SDKs provided with Xcode6 and iOS8. * Fix ALL the header guards. * Createed a README.md with the LevelDB project description. * A new CONTRIBUTING file. * Don't implicitly convert uint64_t to size_t or int. Either preserve it as uint64_t, or explicitly cast. This fixes MSVC warnings about possible value truncation when compiling this code in Chromium. * Added a DumpFile() library function that encapsulates the guts of the "leveldbutil dump" command. This will allow clients to dump data to their log files instead of stdout. It will also allow clients to supply their own environment. * leveldb: Remove unused function 'ConsumeChar'. * leveldbutil: Remove unused member variables from WriteBatchItemPrinter. * OpenBSD, NetBSD and DragonflyBSD have _LITTLE_ENDIAN, so define PLATFORM_IS_LITTLE_ENDIAN like on FreeBSD. This fixes: * issue #143 * issue #198 * issue #249 * Switch from <cstdatomic> to <atomic>. The former never made it into the standard and doesn't exist in modern gcc versions at all. The later contains everything that leveldb was using from the former. This problem was noticed when porting to Portable Native Client where no memory barrier is defined. The fact that <cstdatomic> is missing normally goes unnoticed since memory barriers are defined for most architectures. * Make Hash() treat its input as unsigned. Before this change LevelDB files from platforms with different signedness of char were not compatible. This change fixes: issue #243 * Verify checksums of index/meta/filter blocks when paranoid_checks set. * Invoke all tools for iOS with xcrun. (This was causing problems with the new XCode 5.1.1 image on pulse.) * include <sys/stat.h> only once, and fix the following linter warning: "Found C system header after C++ system header" * When encountering a corrupted table file, return Status::Corruption instead of Status::InvalidArgument. * Support cygwin as build platform, patch is from https://code.google.com/p/leveldb/issues/detail?id=188 * Fix typo, merge patch from https://code.google.com/p/leveldb/issues/detail?id=159 * Fix typos and comments, and address the following two issues: * issue #166 * issue #241 * Add missing db synchronize after "fillseq" in the benchmark. * Removed unused variable in SeekRandom: value (issue #201)
il y a 10 ans
Release 1.18 Changes are: * Update version number to 1.18 * Replace the basic fprintf call with a call to fwrite in order to work around the apparent compiler optimization/rewrite failure that we are seeing with the new toolchain/iOS SDKs provided with Xcode6 and iOS8. * Fix ALL the header guards. * Createed a README.md with the LevelDB project description. * A new CONTRIBUTING file. * Don't implicitly convert uint64_t to size_t or int. Either preserve it as uint64_t, or explicitly cast. This fixes MSVC warnings about possible value truncation when compiling this code in Chromium. * Added a DumpFile() library function that encapsulates the guts of the "leveldbutil dump" command. This will allow clients to dump data to their log files instead of stdout. It will also allow clients to supply their own environment. * leveldb: Remove unused function 'ConsumeChar'. * leveldbutil: Remove unused member variables from WriteBatchItemPrinter. * OpenBSD, NetBSD and DragonflyBSD have _LITTLE_ENDIAN, so define PLATFORM_IS_LITTLE_ENDIAN like on FreeBSD. This fixes: * issue #143 * issue #198 * issue #249 * Switch from <cstdatomic> to <atomic>. The former never made it into the standard and doesn't exist in modern gcc versions at all. The later contains everything that leveldb was using from the former. This problem was noticed when porting to Portable Native Client where no memory barrier is defined. The fact that <cstdatomic> is missing normally goes unnoticed since memory barriers are defined for most architectures. * Make Hash() treat its input as unsigned. Before this change LevelDB files from platforms with different signedness of char were not compatible. This change fixes: issue #243 * Verify checksums of index/meta/filter blocks when paranoid_checks set. * Invoke all tools for iOS with xcrun. (This was causing problems with the new XCode 5.1.1 image on pulse.) * include <sys/stat.h> only once, and fix the following linter warning: "Found C system header after C++ system header" * When encountering a corrupted table file, return Status::Corruption instead of Status::InvalidArgument. * Support cygwin as build platform, patch is from https://code.google.com/p/leveldb/issues/detail?id=188 * Fix typo, merge patch from https://code.google.com/p/leveldb/issues/detail?id=159 * Fix typos and comments, and address the following two issues: * issue #166 * issue #241 * Add missing db synchronize after "fillseq" in the benchmark. * Removed unused variable in SeekRandom: value (issue #201)
il y a 10 ans
  1. // Copyright (c) 2012 The LevelDB Authors. All rights reserved.
  2. // Use of this source code is governed by a BSD-style license that can be
  3. // found in the LICENSE file. See the AUTHORS file for names of contributors.
  4. #include "leveldb/dumpfile.h"
  5. #include <stdio.h>
  6. #include "db/dbformat.h"
  7. #include "db/filename.h"
  8. #include "db/log_reader.h"
  9. #include "db/version_edit.h"
  10. #include "db/write_batch_internal.h"
  11. #include "leveldb/env.h"
  12. #include "leveldb/iterator.h"
  13. #include "leveldb/options.h"
  14. #include "leveldb/status.h"
  15. #include "leveldb/table.h"
  16. #include "leveldb/write_batch.h"
  17. #include "util/logging.h"
  18. namespace leveldb {
  19. namespace {
  20. bool GuessType(const std::string& fname, FileType* type) {
  21. size_t pos = fname.rfind('/');
  22. std::string basename;
  23. if (pos == std::string::npos) {
  24. basename = fname;
  25. } else {
  26. basename = std::string(fname.data() + pos + 1, fname.size() - pos - 1);
  27. }
  28. uint64_t ignored;
  29. return ParseFileName(basename, &ignored, type);
  30. }
  31. // Notified when log reader encounters corruption.
  32. class CorruptionReporter : public log::Reader::Reporter {
  33. public:
  34. WritableFile* dst_;
  35. virtual void Corruption(size_t bytes, const Status& status) {
  36. std::string r = "corruption: ";
  37. AppendNumberTo(&r, bytes);
  38. r += " bytes; ";
  39. r += status.ToString();
  40. r.push_back('\n');
  41. dst_->Append(r);
  42. }
  43. };
  44. // Print contents of a log file. (*func)() is called on every record.
  45. Status PrintLogContents(Env* env, const std::string& fname,
  46. void (*func)(uint64_t, Slice, WritableFile*),
  47. WritableFile* dst) {
  48. SequentialFile* file;
  49. Status s = env->NewSequentialFile(fname, &file);
  50. if (!s.ok()) {
  51. return s;
  52. }
  53. CorruptionReporter reporter;
  54. reporter.dst_ = dst;
  55. log::Reader reader(file, &reporter, true, 0);
  56. Slice record;
  57. std::string scratch;
  58. while (reader.ReadRecord(&record, &scratch)) {
  59. (*func)(reader.LastRecordOffset(), record, dst);
  60. }
  61. delete file;
  62. return Status::OK();
  63. }
  64. // Called on every item found in a WriteBatch.
  65. class WriteBatchItemPrinter : public WriteBatch::Handler {
  66. public:
  67. WritableFile* dst_;
  68. virtual void Put(const Slice& key, const Slice& value) {
  69. std::string r = " put '";
  70. AppendEscapedStringTo(&r, key);
  71. r += "' '";
  72. AppendEscapedStringTo(&r, value);
  73. r += "'\n";
  74. dst_->Append(r);
  75. }
  76. virtual void Delete(const Slice& key) {
  77. std::string r = " del '";
  78. AppendEscapedStringTo(&r, key);
  79. r += "'\n";
  80. dst_->Append(r);
  81. }
  82. };
  83. // Called on every log record (each one of which is a WriteBatch)
  84. // found in a kLogFile.
  85. static void WriteBatchPrinter(uint64_t pos, Slice record, WritableFile* dst) {
  86. std::string r = "--- offset ";
  87. AppendNumberTo(&r, pos);
  88. r += "; ";
  89. if (record.size() < 12) {
  90. r += "log record length ";
  91. AppendNumberTo(&r, record.size());
  92. r += " is too small\n";
  93. dst->Append(r);
  94. return;
  95. }
  96. WriteBatch batch;
  97. WriteBatchInternal::SetContents(&batch, record);
  98. r += "sequence ";
  99. AppendNumberTo(&r, WriteBatchInternal::Sequence(&batch));
  100. r.push_back('\n');
  101. dst->Append(r);
  102. WriteBatchItemPrinter batch_item_printer;
  103. batch_item_printer.dst_ = dst;
  104. Status s = batch.Iterate(&batch_item_printer);
  105. if (!s.ok()) {
  106. dst->Append(" error: " + s.ToString() + "\n");
  107. }
  108. }
  109. Status DumpLog(Env* env, const std::string& fname, WritableFile* dst) {
  110. return PrintLogContents(env, fname, WriteBatchPrinter, dst);
  111. }
  112. // Called on every log record (each one of which is a WriteBatch)
  113. // found in a kDescriptorFile.
  114. static void VersionEditPrinter(uint64_t pos, Slice record, WritableFile* dst) {
  115. std::string r = "--- offset ";
  116. AppendNumberTo(&r, pos);
  117. r += "; ";
  118. VersionEdit edit;
  119. Status s = edit.DecodeFrom(record);
  120. if (!s.ok()) {
  121. r += s.ToString();
  122. r.push_back('\n');
  123. } else {
  124. r += edit.DebugString();
  125. }
  126. dst->Append(r);
  127. }
  128. Status DumpDescriptor(Env* env, const std::string& fname, WritableFile* dst) {
  129. return PrintLogContents(env, fname, VersionEditPrinter, dst);
  130. }
  131. Status DumpTable(Env* env, const std::string& fname, WritableFile* dst) {
  132. uint64_t file_size;
  133. RandomAccessFile* file = nullptr;
  134. Table* table = nullptr;
  135. Status s = env->GetFileSize(fname, &file_size);
  136. if (s.ok()) {
  137. s = env->NewRandomAccessFile(fname, &file);
  138. }
  139. if (s.ok()) {
  140. // We use the default comparator, which may or may not match the
  141. // comparator used in this database. However this should not cause
  142. // problems since we only use Table operations that do not require
  143. // any comparisons. In particular, we do not call Seek or Prev.
  144. s = Table::Open(Options(), file, file_size, &table);
  145. }
  146. if (!s.ok()) {
  147. delete table;
  148. delete file;
  149. return s;
  150. }
  151. ReadOptions ro;
  152. ro.fill_cache = false;
  153. Iterator* iter = table->NewIterator(ro);
  154. std::string r;
  155. for (iter->SeekToFirst(); iter->Valid(); iter->Next()) {
  156. r.clear();
  157. ParsedInternalKey key;
  158. if (!ParseInternalKey(iter->key(), &key)) {
  159. r = "badkey '";
  160. AppendEscapedStringTo(&r, iter->key());
  161. r += "' => '";
  162. AppendEscapedStringTo(&r, iter->value());
  163. r += "'\n";
  164. dst->Append(r);
  165. } else {
  166. r = "'";
  167. AppendEscapedStringTo(&r, key.user_key);
  168. r += "' @ ";
  169. AppendNumberTo(&r, key.sequence);
  170. r += " : ";
  171. if (key.type == kTypeDeletion) {
  172. r += "del";
  173. } else if (key.type == kTypeValue) {
  174. r += "val";
  175. } else {
  176. AppendNumberTo(&r, key.type);
  177. }
  178. r += " => '";
  179. AppendEscapedStringTo(&r, iter->value());
  180. r += "'\n";
  181. dst->Append(r);
  182. }
  183. }
  184. s = iter->status();
  185. if (!s.ok()) {
  186. dst->Append("iterator error: " + s.ToString() + "\n");
  187. }
  188. delete iter;
  189. delete table;
  190. delete file;
  191. return Status::OK();
  192. }
  193. } // namespace
  194. Status DumpFile(Env* env, const std::string& fname, WritableFile* dst) {
  195. FileType ftype;
  196. if (!GuessType(fname, &ftype)) {
  197. return Status::InvalidArgument(fname + ": unknown file type");
  198. }
  199. switch (ftype) {
  200. case kLogFile: return DumpLog(env, fname, dst);
  201. case kDescriptorFile: return DumpDescriptor(env, fname, dst);
  202. case kTableFile: return DumpTable(env, fname, dst);
  203. default:
  204. break;
  205. }
  206. return Status::InvalidArgument(fname + ": not a dump-able file type");
  207. }
  208. } // namespace leveldb