作者: 谢瑞阳 10225101483 徐翔宇 10225101535
Nie możesz wybrać więcej, niż 25 tematów Tematy muszą się zaczynać od litery lub cyfry, mogą zawierać myślniki ('-') i mogą mieć do 35 znaków.

392 wiersze
9.7 KiB

Release 1.18 Changes are: * Update version number to 1.18 * Replace the basic fprintf call with a call to fwrite in order to work around the apparent compiler optimization/rewrite failure that we are seeing with the new toolchain/iOS SDKs provided with Xcode6 and iOS8. * Fix ALL the header guards. * Createed a README.md with the LevelDB project description. * A new CONTRIBUTING file. * Don't implicitly convert uint64_t to size_t or int. Either preserve it as uint64_t, or explicitly cast. This fixes MSVC warnings about possible value truncation when compiling this code in Chromium. * Added a DumpFile() library function that encapsulates the guts of the "leveldbutil dump" command. This will allow clients to dump data to their log files instead of stdout. It will also allow clients to supply their own environment. * leveldb: Remove unused function 'ConsumeChar'. * leveldbutil: Remove unused member variables from WriteBatchItemPrinter. * OpenBSD, NetBSD and DragonflyBSD have _LITTLE_ENDIAN, so define PLATFORM_IS_LITTLE_ENDIAN like on FreeBSD. This fixes: * issue #143 * issue #198 * issue #249 * Switch from <cstdatomic> to <atomic>. The former never made it into the standard and doesn't exist in modern gcc versions at all. The later contains everything that leveldb was using from the former. This problem was noticed when porting to Portable Native Client where no memory barrier is defined. The fact that <cstdatomic> is missing normally goes unnoticed since memory barriers are defined for most architectures. * Make Hash() treat its input as unsigned. Before this change LevelDB files from platforms with different signedness of char were not compatible. This change fixes: issue #243 * Verify checksums of index/meta/filter blocks when paranoid_checks set. * Invoke all tools for iOS with xcrun. (This was causing problems with the new XCode 5.1.1 image on pulse.) * include <sys/stat.h> only once, and fix the following linter warning: "Found C system header after C++ system header" * When encountering a corrupted table file, return Status::Corruption instead of Status::InvalidArgument. * Support cygwin as build platform, patch is from https://code.google.com/p/leveldb/issues/detail?id=188 * Fix typo, merge patch from https://code.google.com/p/leveldb/issues/detail?id=159 * Fix typos and comments, and address the following two issues: * issue #166 * issue #241 * Add missing db synchronize after "fillseq" in the benchmark. * Removed unused variable in SeekRandom: value (issue #201)
10 lat temu
Release 1.18 Changes are: * Update version number to 1.18 * Replace the basic fprintf call with a call to fwrite in order to work around the apparent compiler optimization/rewrite failure that we are seeing with the new toolchain/iOS SDKs provided with Xcode6 and iOS8. * Fix ALL the header guards. * Createed a README.md with the LevelDB project description. * A new CONTRIBUTING file. * Don't implicitly convert uint64_t to size_t or int. Either preserve it as uint64_t, or explicitly cast. This fixes MSVC warnings about possible value truncation when compiling this code in Chromium. * Added a DumpFile() library function that encapsulates the guts of the "leveldbutil dump" command. This will allow clients to dump data to their log files instead of stdout. It will also allow clients to supply their own environment. * leveldb: Remove unused function 'ConsumeChar'. * leveldbutil: Remove unused member variables from WriteBatchItemPrinter. * OpenBSD, NetBSD and DragonflyBSD have _LITTLE_ENDIAN, so define PLATFORM_IS_LITTLE_ENDIAN like on FreeBSD. This fixes: * issue #143 * issue #198 * issue #249 * Switch from <cstdatomic> to <atomic>. The former never made it into the standard and doesn't exist in modern gcc versions at all. The later contains everything that leveldb was using from the former. This problem was noticed when porting to Portable Native Client where no memory barrier is defined. The fact that <cstdatomic> is missing normally goes unnoticed since memory barriers are defined for most architectures. * Make Hash() treat its input as unsigned. Before this change LevelDB files from platforms with different signedness of char were not compatible. This change fixes: issue #243 * Verify checksums of index/meta/filter blocks when paranoid_checks set. * Invoke all tools for iOS with xcrun. (This was causing problems with the new XCode 5.1.1 image on pulse.) * include <sys/stat.h> only once, and fix the following linter warning: "Found C system header after C++ system header" * When encountering a corrupted table file, return Status::Corruption instead of Status::InvalidArgument. * Support cygwin as build platform, patch is from https://code.google.com/p/leveldb/issues/detail?id=188 * Fix typo, merge patch from https://code.google.com/p/leveldb/issues/detail?id=159 * Fix typos and comments, and address the following two issues: * issue #166 * issue #241 * Add missing db synchronize after "fillseq" in the benchmark. * Removed unused variable in SeekRandom: value (issue #201)
10 lat temu
Release 1.18 Changes are: * Update version number to 1.18 * Replace the basic fprintf call with a call to fwrite in order to work around the apparent compiler optimization/rewrite failure that we are seeing with the new toolchain/iOS SDKs provided with Xcode6 and iOS8. * Fix ALL the header guards. * Createed a README.md with the LevelDB project description. * A new CONTRIBUTING file. * Don't implicitly convert uint64_t to size_t or int. Either preserve it as uint64_t, or explicitly cast. This fixes MSVC warnings about possible value truncation when compiling this code in Chromium. * Added a DumpFile() library function that encapsulates the guts of the "leveldbutil dump" command. This will allow clients to dump data to their log files instead of stdout. It will also allow clients to supply their own environment. * leveldb: Remove unused function 'ConsumeChar'. * leveldbutil: Remove unused member variables from WriteBatchItemPrinter. * OpenBSD, NetBSD and DragonflyBSD have _LITTLE_ENDIAN, so define PLATFORM_IS_LITTLE_ENDIAN like on FreeBSD. This fixes: * issue #143 * issue #198 * issue #249 * Switch from <cstdatomic> to <atomic>. The former never made it into the standard and doesn't exist in modern gcc versions at all. The later contains everything that leveldb was using from the former. This problem was noticed when porting to Portable Native Client where no memory barrier is defined. The fact that <cstdatomic> is missing normally goes unnoticed since memory barriers are defined for most architectures. * Make Hash() treat its input as unsigned. Before this change LevelDB files from platforms with different signedness of char were not compatible. This change fixes: issue #243 * Verify checksums of index/meta/filter blocks when paranoid_checks set. * Invoke all tools for iOS with xcrun. (This was causing problems with the new XCode 5.1.1 image on pulse.) * include <sys/stat.h> only once, and fix the following linter warning: "Found C system header after C++ system header" * When encountering a corrupted table file, return Status::Corruption instead of Status::InvalidArgument. * Support cygwin as build platform, patch is from https://code.google.com/p/leveldb/issues/detail?id=188 * Fix typo, merge patch from https://code.google.com/p/leveldb/issues/detail?id=159 * Fix typos and comments, and address the following two issues: * issue #166 * issue #241 * Add missing db synchronize after "fillseq" in the benchmark. * Removed unused variable in SeekRandom: value (issue #201)
10 lat temu
Release 1.18 Changes are: * Update version number to 1.18 * Replace the basic fprintf call with a call to fwrite in order to work around the apparent compiler optimization/rewrite failure that we are seeing with the new toolchain/iOS SDKs provided with Xcode6 and iOS8. * Fix ALL the header guards. * Createed a README.md with the LevelDB project description. * A new CONTRIBUTING file. * Don't implicitly convert uint64_t to size_t or int. Either preserve it as uint64_t, or explicitly cast. This fixes MSVC warnings about possible value truncation when compiling this code in Chromium. * Added a DumpFile() library function that encapsulates the guts of the "leveldbutil dump" command. This will allow clients to dump data to their log files instead of stdout. It will also allow clients to supply their own environment. * leveldb: Remove unused function 'ConsumeChar'. * leveldbutil: Remove unused member variables from WriteBatchItemPrinter. * OpenBSD, NetBSD and DragonflyBSD have _LITTLE_ENDIAN, so define PLATFORM_IS_LITTLE_ENDIAN like on FreeBSD. This fixes: * issue #143 * issue #198 * issue #249 * Switch from <cstdatomic> to <atomic>. The former never made it into the standard and doesn't exist in modern gcc versions at all. The later contains everything that leveldb was using from the former. This problem was noticed when porting to Portable Native Client where no memory barrier is defined. The fact that <cstdatomic> is missing normally goes unnoticed since memory barriers are defined for most architectures. * Make Hash() treat its input as unsigned. Before this change LevelDB files from platforms with different signedness of char were not compatible. This change fixes: issue #243 * Verify checksums of index/meta/filter blocks when paranoid_checks set. * Invoke all tools for iOS with xcrun. (This was causing problems with the new XCode 5.1.1 image on pulse.) * include <sys/stat.h> only once, and fix the following linter warning: "Found C system header after C++ system header" * When encountering a corrupted table file, return Status::Corruption instead of Status::InvalidArgument. * Support cygwin as build platform, patch is from https://code.google.com/p/leveldb/issues/detail?id=188 * Fix typo, merge patch from https://code.google.com/p/leveldb/issues/detail?id=159 * Fix typos and comments, and address the following two issues: * issue #166 * issue #241 * Add missing db synchronize after "fillseq" in the benchmark. * Removed unused variable in SeekRandom: value (issue #201)
10 lat temu
  1. // Copyright (c) 2011 The LevelDB Authors. All rights reserved.
  2. // Use of this source code is governed by a BSD-style license that can be
  3. // found in the LICENSE file. See the AUTHORS file for names of contributors.
  4. #include "helpers/memenv/memenv.h"
  5. #include <string.h>
  6. #include <limits>
  7. #include <map>
  8. #include <string>
  9. #include <vector>
  10. #include "leveldb/env.h"
  11. #include "leveldb/status.h"
  12. #include "port/port.h"
  13. #include "port/thread_annotations.h"
  14. #include "util/mutexlock.h"
  15. namespace leveldb {
  16. namespace {
  17. class FileState {
  18. public:
  19. // FileStates are reference counted. The initial reference count is zero
  20. // and the caller must call Ref() at least once.
  21. FileState() : refs_(0), size_(0) {}
  22. // No copying allowed.
  23. FileState(const FileState&) = delete;
  24. FileState& operator=(const FileState&) = delete;
  25. // Increase the reference count.
  26. void Ref() {
  27. MutexLock lock(&refs_mutex_);
  28. ++refs_;
  29. }
  30. // Decrease the reference count. Delete if this is the last reference.
  31. void Unref() {
  32. bool do_delete = false;
  33. {
  34. MutexLock lock(&refs_mutex_);
  35. --refs_;
  36. assert(refs_ >= 0);
  37. if (refs_ <= 0) {
  38. do_delete = true;
  39. }
  40. }
  41. if (do_delete) {
  42. delete this;
  43. }
  44. }
  45. uint64_t Size() const {
  46. MutexLock lock(&blocks_mutex_);
  47. return size_;
  48. }
  49. void Truncate() {
  50. MutexLock lock(&blocks_mutex_);
  51. for (char*& block : blocks_) {
  52. delete[] block;
  53. }
  54. blocks_.clear();
  55. size_ = 0;
  56. }
  57. Status Read(uint64_t offset, size_t n, Slice* result, char* scratch) const {
  58. MutexLock lock(&blocks_mutex_);
  59. if (offset > size_) {
  60. return Status::IOError("Offset greater than file size.");
  61. }
  62. const uint64_t available = size_ - offset;
  63. if (n > available) {
  64. n = static_cast<size_t>(available);
  65. }
  66. if (n == 0) {
  67. *result = Slice();
  68. return Status::OK();
  69. }
  70. assert(offset / kBlockSize <= std::numeric_limits<size_t>::max());
  71. size_t block = static_cast<size_t>(offset / kBlockSize);
  72. size_t block_offset = offset % kBlockSize;
  73. size_t bytes_to_copy = n;
  74. char* dst = scratch;
  75. while (bytes_to_copy > 0) {
  76. size_t avail = kBlockSize - block_offset;
  77. if (avail > bytes_to_copy) {
  78. avail = bytes_to_copy;
  79. }
  80. memcpy(dst, blocks_[block] + block_offset, avail);
  81. bytes_to_copy -= avail;
  82. dst += avail;
  83. block++;
  84. block_offset = 0;
  85. }
  86. *result = Slice(scratch, n);
  87. return Status::OK();
  88. }
  89. Status Append(const Slice& data) {
  90. const char* src = data.data();
  91. size_t src_len = data.size();
  92. MutexLock lock(&blocks_mutex_);
  93. while (src_len > 0) {
  94. size_t avail;
  95. size_t offset = size_ % kBlockSize;
  96. if (offset != 0) {
  97. // There is some room in the last block.
  98. avail = kBlockSize - offset;
  99. } else {
  100. // No room in the last block; push new one.
  101. blocks_.push_back(new char[kBlockSize]);
  102. avail = kBlockSize;
  103. }
  104. if (avail > src_len) {
  105. avail = src_len;
  106. }
  107. memcpy(blocks_.back() + offset, src, avail);
  108. src_len -= avail;
  109. src += avail;
  110. size_ += avail;
  111. }
  112. return Status::OK();
  113. }
  114. private:
  115. enum { kBlockSize = 8 * 1024 };
  116. // Private since only Unref() should be used to delete it.
  117. ~FileState() { Truncate(); }
  118. port::Mutex refs_mutex_;
  119. int refs_ GUARDED_BY(refs_mutex_);
  120. mutable port::Mutex blocks_mutex_;
  121. std::vector<char*> blocks_ GUARDED_BY(blocks_mutex_);
  122. uint64_t size_ GUARDED_BY(blocks_mutex_);
  123. };
  124. class SequentialFileImpl : public SequentialFile {
  125. public:
  126. explicit SequentialFileImpl(FileState* file) : file_(file), pos_(0) {
  127. file_->Ref();
  128. }
  129. ~SequentialFileImpl() { file_->Unref(); }
  130. virtual Status Read(size_t n, Slice* result, char* scratch) {
  131. Status s = file_->Read(pos_, n, result, scratch);
  132. if (s.ok()) {
  133. pos_ += result->size();
  134. }
  135. return s;
  136. }
  137. virtual Status Skip(uint64_t n) {
  138. if (pos_ > file_->Size()) {
  139. return Status::IOError("pos_ > file_->Size()");
  140. }
  141. const uint64_t available = file_->Size() - pos_;
  142. if (n > available) {
  143. n = available;
  144. }
  145. pos_ += n;
  146. return Status::OK();
  147. }
  148. private:
  149. FileState* file_;
  150. uint64_t pos_;
  151. };
  152. class RandomAccessFileImpl : public RandomAccessFile {
  153. public:
  154. explicit RandomAccessFileImpl(FileState* file) : file_(file) { file_->Ref(); }
  155. ~RandomAccessFileImpl() { file_->Unref(); }
  156. virtual Status Read(uint64_t offset, size_t n, Slice* result,
  157. char* scratch) const {
  158. return file_->Read(offset, n, result, scratch);
  159. }
  160. private:
  161. FileState* file_;
  162. };
  163. class WritableFileImpl : public WritableFile {
  164. public:
  165. WritableFileImpl(FileState* file) : file_(file) { file_->Ref(); }
  166. ~WritableFileImpl() { file_->Unref(); }
  167. virtual Status Append(const Slice& data) { return file_->Append(data); }
  168. virtual Status Close() { return Status::OK(); }
  169. virtual Status Flush() { return Status::OK(); }
  170. virtual Status Sync() { return Status::OK(); }
  171. private:
  172. FileState* file_;
  173. };
  174. class NoOpLogger : public Logger {
  175. public:
  176. virtual void Logv(const char* format, va_list ap) {}
  177. };
  178. class InMemoryEnv : public EnvWrapper {
  179. public:
  180. explicit InMemoryEnv(Env* base_env) : EnvWrapper(base_env) {}
  181. virtual ~InMemoryEnv() {
  182. for (FileSystem::iterator i = file_map_.begin(); i != file_map_.end();
  183. ++i) {
  184. i->second->Unref();
  185. }
  186. }
  187. // Partial implementation of the Env interface.
  188. virtual Status NewSequentialFile(const std::string& fname,
  189. SequentialFile** result) {
  190. MutexLock lock(&mutex_);
  191. if (file_map_.find(fname) == file_map_.end()) {
  192. *result = nullptr;
  193. return Status::IOError(fname, "File not found");
  194. }
  195. *result = new SequentialFileImpl(file_map_[fname]);
  196. return Status::OK();
  197. }
  198. virtual Status NewRandomAccessFile(const std::string& fname,
  199. RandomAccessFile** result) {
  200. MutexLock lock(&mutex_);
  201. if (file_map_.find(fname) == file_map_.end()) {
  202. *result = nullptr;
  203. return Status::IOError(fname, "File not found");
  204. }
  205. *result = new RandomAccessFileImpl(file_map_[fname]);
  206. return Status::OK();
  207. }
  208. virtual Status NewWritableFile(const std::string& fname,
  209. WritableFile** result) {
  210. MutexLock lock(&mutex_);
  211. FileSystem::iterator it = file_map_.find(fname);
  212. FileState* file;
  213. if (it == file_map_.end()) {
  214. // File is not currently open.
  215. file = new FileState();
  216. file->Ref();
  217. file_map_[fname] = file;
  218. } else {
  219. file = it->second;
  220. file->Truncate();
  221. }
  222. *result = new WritableFileImpl(file);
  223. return Status::OK();
  224. }
  225. virtual Status NewAppendableFile(const std::string& fname,
  226. WritableFile** result) {
  227. MutexLock lock(&mutex_);
  228. FileState** sptr = &file_map_[fname];
  229. FileState* file = *sptr;
  230. if (file == nullptr) {
  231. file = new FileState();
  232. file->Ref();
  233. }
  234. *result = new WritableFileImpl(file);
  235. return Status::OK();
  236. }
  237. virtual bool FileExists(const std::string& fname) {
  238. MutexLock lock(&mutex_);
  239. return file_map_.find(fname) != file_map_.end();
  240. }
  241. virtual Status GetChildren(const std::string& dir,
  242. std::vector<std::string>* result) {
  243. MutexLock lock(&mutex_);
  244. result->clear();
  245. for (FileSystem::iterator i = file_map_.begin(); i != file_map_.end();
  246. ++i) {
  247. const std::string& filename = i->first;
  248. if (filename.size() >= dir.size() + 1 && filename[dir.size()] == '/' &&
  249. Slice(filename).starts_with(Slice(dir))) {
  250. result->push_back(filename.substr(dir.size() + 1));
  251. }
  252. }
  253. return Status::OK();
  254. }
  255. void DeleteFileInternal(const std::string& fname)
  256. EXCLUSIVE_LOCKS_REQUIRED(mutex_) {
  257. if (file_map_.find(fname) == file_map_.end()) {
  258. return;
  259. }
  260. file_map_[fname]->Unref();
  261. file_map_.erase(fname);
  262. }
  263. virtual Status DeleteFile(const std::string& fname) {
  264. MutexLock lock(&mutex_);
  265. if (file_map_.find(fname) == file_map_.end()) {
  266. return Status::IOError(fname, "File not found");
  267. }
  268. DeleteFileInternal(fname);
  269. return Status::OK();
  270. }
  271. virtual Status CreateDir(const std::string& dirname) { return Status::OK(); }
  272. virtual Status DeleteDir(const std::string& dirname) { return Status::OK(); }
  273. virtual Status GetFileSize(const std::string& fname, uint64_t* file_size) {
  274. MutexLock lock(&mutex_);
  275. if (file_map_.find(fname) == file_map_.end()) {
  276. return Status::IOError(fname, "File not found");
  277. }
  278. *file_size = file_map_[fname]->Size();
  279. return Status::OK();
  280. }
  281. virtual Status RenameFile(const std::string& src, const std::string& target) {
  282. MutexLock lock(&mutex_);
  283. if (file_map_.find(src) == file_map_.end()) {
  284. return Status::IOError(src, "File not found");
  285. }
  286. DeleteFileInternal(target);
  287. file_map_[target] = file_map_[src];
  288. file_map_.erase(src);
  289. return Status::OK();
  290. }
  291. virtual Status LockFile(const std::string& fname, FileLock** lock) {
  292. *lock = new FileLock;
  293. return Status::OK();
  294. }
  295. virtual Status UnlockFile(FileLock* lock) {
  296. delete lock;
  297. return Status::OK();
  298. }
  299. virtual Status GetTestDirectory(std::string* path) {
  300. *path = "/test";
  301. return Status::OK();
  302. }
  303. virtual Status NewLogger(const std::string& fname, Logger** result) {
  304. *result = new NoOpLogger;
  305. return Status::OK();
  306. }
  307. private:
  308. // Map from filenames to FileState objects, representing a simple file system.
  309. typedef std::map<std::string, FileState*> FileSystem;
  310. port::Mutex mutex_;
  311. FileSystem file_map_ GUARDED_BY(mutex_);
  312. };
  313. } // namespace
  314. Env* NewMemEnv(Env* base_env) { return new InMemoryEnv(base_env); }
  315. } // namespace leveldb