小组成员: 曹可心-10223903406 朴祉燕-10224602413
Nelze vybrat více než 25 témat Téma musí začínat písmenem nebo číslem, může obsahovat pomlčky („-“) a může být dlouhé až 35 znaků.

341 řádky
11 KiB

Release 1.18 Changes are: * Update version number to 1.18 * Replace the basic fprintf call with a call to fwrite in order to work around the apparent compiler optimization/rewrite failure that we are seeing with the new toolchain/iOS SDKs provided with Xcode6 and iOS8. * Fix ALL the header guards. * Createed a README.md with the LevelDB project description. * A new CONTRIBUTING file. * Don't implicitly convert uint64_t to size_t or int. Either preserve it as uint64_t, or explicitly cast. This fixes MSVC warnings about possible value truncation when compiling this code in Chromium. * Added a DumpFile() library function that encapsulates the guts of the "leveldbutil dump" command. This will allow clients to dump data to their log files instead of stdout. It will also allow clients to supply their own environment. * leveldb: Remove unused function 'ConsumeChar'. * leveldbutil: Remove unused member variables from WriteBatchItemPrinter. * OpenBSD, NetBSD and DragonflyBSD have _LITTLE_ENDIAN, so define PLATFORM_IS_LITTLE_ENDIAN like on FreeBSD. This fixes: * issue #143 * issue #198 * issue #249 * Switch from <cstdatomic> to <atomic>. The former never made it into the standard and doesn't exist in modern gcc versions at all. The later contains everything that leveldb was using from the former. This problem was noticed when porting to Portable Native Client where no memory barrier is defined. The fact that <cstdatomic> is missing normally goes unnoticed since memory barriers are defined for most architectures. * Make Hash() treat its input as unsigned. Before this change LevelDB files from platforms with different signedness of char were not compatible. This change fixes: issue #243 * Verify checksums of index/meta/filter blocks when paranoid_checks set. * Invoke all tools for iOS with xcrun. (This was causing problems with the new XCode 5.1.1 image on pulse.) * include <sys/stat.h> only once, and fix the following linter warning: "Found C system header after C++ system header" * When encountering a corrupted table file, return Status::Corruption instead of Status::InvalidArgument. * Support cygwin as build platform, patch is from https://code.google.com/p/leveldb/issues/detail?id=188 * Fix typo, merge patch from https://code.google.com/p/leveldb/issues/detail?id=159 * Fix typos and comments, and address the following two issues: * issue #166 * issue #241 * Add missing db synchronize after "fillseq" in the benchmark. * Removed unused variable in SeekRandom: value (issue #201)
před 10 roky
Release 1.18 Changes are: * Update version number to 1.18 * Replace the basic fprintf call with a call to fwrite in order to work around the apparent compiler optimization/rewrite failure that we are seeing with the new toolchain/iOS SDKs provided with Xcode6 and iOS8. * Fix ALL the header guards. * Createed a README.md with the LevelDB project description. * A new CONTRIBUTING file. * Don't implicitly convert uint64_t to size_t or int. Either preserve it as uint64_t, or explicitly cast. This fixes MSVC warnings about possible value truncation when compiling this code in Chromium. * Added a DumpFile() library function that encapsulates the guts of the "leveldbutil dump" command. This will allow clients to dump data to their log files instead of stdout. It will also allow clients to supply their own environment. * leveldb: Remove unused function 'ConsumeChar'. * leveldbutil: Remove unused member variables from WriteBatchItemPrinter. * OpenBSD, NetBSD and DragonflyBSD have _LITTLE_ENDIAN, so define PLATFORM_IS_LITTLE_ENDIAN like on FreeBSD. This fixes: * issue #143 * issue #198 * issue #249 * Switch from <cstdatomic> to <atomic>. The former never made it into the standard and doesn't exist in modern gcc versions at all. The later contains everything that leveldb was using from the former. This problem was noticed when porting to Portable Native Client where no memory barrier is defined. The fact that <cstdatomic> is missing normally goes unnoticed since memory barriers are defined for most architectures. * Make Hash() treat its input as unsigned. Before this change LevelDB files from platforms with different signedness of char were not compatible. This change fixes: issue #243 * Verify checksums of index/meta/filter blocks when paranoid_checks set. * Invoke all tools for iOS with xcrun. (This was causing problems with the new XCode 5.1.1 image on pulse.) * include <sys/stat.h> only once, and fix the following linter warning: "Found C system header after C++ system header" * When encountering a corrupted table file, return Status::Corruption instead of Status::InvalidArgument. * Support cygwin as build platform, patch is from https://code.google.com/p/leveldb/issues/detail?id=188 * Fix typo, merge patch from https://code.google.com/p/leveldb/issues/detail?id=159 * Fix typos and comments, and address the following two issues: * issue #166 * issue #241 * Add missing db synchronize after "fillseq" in the benchmark. * Removed unused variable in SeekRandom: value (issue #201)
před 10 roky
Release 1.18 Changes are: * Update version number to 1.18 * Replace the basic fprintf call with a call to fwrite in order to work around the apparent compiler optimization/rewrite failure that we are seeing with the new toolchain/iOS SDKs provided with Xcode6 and iOS8. * Fix ALL the header guards. * Createed a README.md with the LevelDB project description. * A new CONTRIBUTING file. * Don't implicitly convert uint64_t to size_t or int. Either preserve it as uint64_t, or explicitly cast. This fixes MSVC warnings about possible value truncation when compiling this code in Chromium. * Added a DumpFile() library function that encapsulates the guts of the "leveldbutil dump" command. This will allow clients to dump data to their log files instead of stdout. It will also allow clients to supply their own environment. * leveldb: Remove unused function 'ConsumeChar'. * leveldbutil: Remove unused member variables from WriteBatchItemPrinter. * OpenBSD, NetBSD and DragonflyBSD have _LITTLE_ENDIAN, so define PLATFORM_IS_LITTLE_ENDIAN like on FreeBSD. This fixes: * issue #143 * issue #198 * issue #249 * Switch from <cstdatomic> to <atomic>. The former never made it into the standard and doesn't exist in modern gcc versions at all. The later contains everything that leveldb was using from the former. This problem was noticed when porting to Portable Native Client where no memory barrier is defined. The fact that <cstdatomic> is missing normally goes unnoticed since memory barriers are defined for most architectures. * Make Hash() treat its input as unsigned. Before this change LevelDB files from platforms with different signedness of char were not compatible. This change fixes: issue #243 * Verify checksums of index/meta/filter blocks when paranoid_checks set. * Invoke all tools for iOS with xcrun. (This was causing problems with the new XCode 5.1.1 image on pulse.) * include <sys/stat.h> only once, and fix the following linter warning: "Found C system header after C++ system header" * When encountering a corrupted table file, return Status::Corruption instead of Status::InvalidArgument. * Support cygwin as build platform, patch is from https://code.google.com/p/leveldb/issues/detail?id=188 * Fix typo, merge patch from https://code.google.com/p/leveldb/issues/detail?id=159 * Fix typos and comments, and address the following two issues: * issue #166 * issue #241 * Add missing db synchronize after "fillseq" in the benchmark. * Removed unused variable in SeekRandom: value (issue #201)
před 10 roky
  1. // Copyright (c) 2011 The LevelDB Authors. All rights reserved.
  2. // Use of this source code is governed by a BSD-style license that can be
  3. // found in the LICENSE file. See the AUTHORS file for names of contributors.
  4. #include "leveldb/table.h"
  5. #include <sstream> // For std::istringstream
  6. #include <iomanip> // For std::get_time
  7. #include <ctime> // For std::tm and std::mktime
  8. #include "leveldb/cache.h"
  9. #include "leveldb/comparator.h"
  10. #include "leveldb/env.h"
  11. #include "leveldb/filter_policy.h"
  12. #include "leveldb/options.h"
  13. #include "table/block.h"
  14. #include "table/filter_block.h"
  15. #include "table/format.h"
  16. #include "table/two_level_iterator.h"
  17. #include "util/coding.h"
  18. namespace leveldb {
  19. struct Table::Rep {
  20. ~Rep() {
  21. delete filter;
  22. delete[] filter_data;
  23. delete index_block;
  24. }
  25. Options options;
  26. Status status;
  27. RandomAccessFile* file;
  28. uint64_t cache_id;
  29. FilterBlockReader* filter;
  30. const char* filter_data;
  31. BlockHandle metaindex_handle; // Handle to metaindex_block: saved from footer
  32. Block* index_block;
  33. };
  34. Status Table::Open(const Options& options, RandomAccessFile* file,
  35. uint64_t size, Table** table) {
  36. *table = nullptr;
  37. if (size < Footer::kEncodedLength) {
  38. return Status::Corruption("file is too short to be an sstable");
  39. }
  40. char footer_space[Footer::kEncodedLength];
  41. Slice footer_input;
  42. Status s = file->Read(size - Footer::kEncodedLength, Footer::kEncodedLength,
  43. &footer_input, footer_space);
  44. if (!s.ok()) return s;
  45. Footer footer;
  46. s = footer.DecodeFrom(&footer_input);
  47. if (!s.ok()) return s;
  48. // Read the index block
  49. BlockContents index_block_contents;
  50. ReadOptions opt;
  51. if (options.paranoid_checks) {
  52. opt.verify_checksums = true;
  53. }
  54. s = ReadBlock(file, opt, footer.index_handle(), &index_block_contents);
  55. if (s.ok()) {
  56. // We've successfully read the footer and the index block: we're
  57. // ready to serve requests.
  58. Block* index_block = new Block(index_block_contents);
  59. Rep* rep = new Table::Rep;
  60. rep->options = options;
  61. rep->file = file;
  62. rep->metaindex_handle = footer.metaindex_handle();
  63. rep->index_block = index_block;
  64. rep->cache_id = (options.block_cache ? options.block_cache->NewId() : 0);
  65. rep->filter_data = nullptr;
  66. rep->filter = nullptr;
  67. *table = new Table(rep);
  68. (*table)->ReadMeta(footer);
  69. }
  70. return s;
  71. }
  72. void Table::ReadMeta(const Footer& footer) {
  73. if (rep_->options.filter_policy == nullptr) {
  74. return; // Do not need any metadata
  75. }
  76. // TODO(sanjay): Skip this if footer.metaindex_handle() size indicates
  77. // it is an empty block.
  78. ReadOptions opt;
  79. if (rep_->options.paranoid_checks) {
  80. opt.verify_checksums = true;
  81. }
  82. BlockContents contents;
  83. if (!ReadBlock(rep_->file, opt, footer.metaindex_handle(), &contents).ok()) {
  84. // Do not propagate errors since meta info is not needed for operation
  85. return;
  86. }
  87. Block* meta = new Block(contents);
  88. Iterator* iter = meta->NewIterator(BytewiseComparator());
  89. std::string key = "filter.";
  90. key.append(rep_->options.filter_policy->Name());
  91. iter->Seek(key);
  92. if (iter->Valid() && iter->key() == Slice(key)) {
  93. ReadFilter(iter->value());
  94. }
  95. delete iter;
  96. delete meta;
  97. }
  98. void Table::ReadFilter(const Slice& filter_handle_value) {
  99. Slice v = filter_handle_value;
  100. BlockHandle filter_handle;
  101. if (!filter_handle.DecodeFrom(&v).ok()) {
  102. return;
  103. }
  104. // We might want to unify with ReadBlock() if we start
  105. // requiring checksum verification in Table::Open.
  106. ReadOptions opt;
  107. if (rep_->options.paranoid_checks) {
  108. opt.verify_checksums = true;
  109. }
  110. BlockContents block;
  111. if (!ReadBlock(rep_->file, opt, filter_handle, &block).ok()) {
  112. return;
  113. }
  114. if (block.heap_allocated) {
  115. rep_->filter_data = block.data.data(); // Will need to delete later
  116. }
  117. rep_->filter = new FilterBlockReader(rep_->options.filter_policy, block.data);
  118. }
  119. Table::~Table() { delete rep_; }
  120. static void DeleteBlock(void* arg, void* ignored) {
  121. delete reinterpret_cast<Block*>(arg);
  122. }
  123. static void DeleteCachedBlock(const Slice& key, void* value) {
  124. Block* block = reinterpret_cast<Block*>(value);
  125. delete block;
  126. }
  127. static void ReleaseBlock(void* arg, void* h) {
  128. Cache* cache = reinterpret_cast<Cache*>(arg);
  129. Cache::Handle* handle = reinterpret_cast<Cache::Handle*>(h);
  130. cache->Release(handle);
  131. }
  132. // Convert an index iterator value (i.e., an encoded BlockHandle)
  133. // into an iterator over the contents of the corresponding block.
  134. Iterator* Table::BlockReader(void* arg, const ReadOptions& options,
  135. const Slice& index_value) {
  136. Table* table = reinterpret_cast<Table*>(arg);
  137. Cache* block_cache = table->rep_->options.block_cache;
  138. Block* block = nullptr;
  139. Cache::Handle* cache_handle = nullptr;
  140. BlockHandle handle;
  141. Slice input = index_value;
  142. Status s = handle.DecodeFrom(&input);
  143. // We intentionally allow extra stuff in index_value so that we
  144. // can add more features in the future.
  145. if (s.ok()) {
  146. BlockContents contents;
  147. if (block_cache != nullptr) {
  148. char cache_key_buffer[16];
  149. EncodeFixed64(cache_key_buffer, table->rep_->cache_id);
  150. EncodeFixed64(cache_key_buffer + 8, handle.offset());
  151. Slice key(cache_key_buffer, sizeof(cache_key_buffer));
  152. cache_handle = block_cache->Lookup(key);
  153. if (cache_handle != nullptr) {
  154. block = reinterpret_cast<Block*>(block_cache->Value(cache_handle));
  155. } else {
  156. s = ReadBlock(table->rep_->file, options, handle, &contents);
  157. if (s.ok()) {
  158. block = new Block(contents);
  159. if (contents.cachable && options.fill_cache) {
  160. cache_handle = block_cache->Insert(key, block, block->size(),
  161. &DeleteCachedBlock);
  162. }
  163. }
  164. }
  165. } else {
  166. s = ReadBlock(table->rep_->file, options, handle, &contents);
  167. if (s.ok()) {
  168. block = new Block(contents);
  169. }
  170. }
  171. }
  172. Iterator* iter;
  173. if (block != nullptr) {
  174. iter = block->NewIterator(table->rep_->options.comparator);
  175. if (cache_handle == nullptr) {
  176. iter->RegisterCleanup(&DeleteBlock, block, nullptr);
  177. } else {
  178. iter->RegisterCleanup(&ReleaseBlock, block_cache, cache_handle);
  179. }
  180. } else {
  181. iter = NewErrorIterator(s);
  182. }
  183. return iter;
  184. }
  185. Iterator* Table::NewIterator(const ReadOptions& options) const {
  186. return NewTwoLevelIterator(
  187. rep_->index_block->NewIterator(rep_->options.comparator),
  188. &Table::BlockReader, const_cast<Table*>(this), options);
  189. }
  190. // Status Table::InternalGet(const ReadOptions& options, const Slice& k, void* arg,
  191. // void (*handle_result)(void*, const Slice&,
  192. // const Slice&)) {
  193. // Status s;
  194. // Iterator* iiter = rep_->index_block->NewIterator(rep_->options.comparator);
  195. // iiter->Seek(k);
  196. // if (iiter->Valid()) {
  197. // Slice handle_value = iiter->value();
  198. // FilterBlockReader* filter = rep_->filter;
  199. // BlockHandle handle;
  200. // if (filter != nullptr && handle.DecodeFrom(&handle_value).ok() &&
  201. // !filter->KeyMayMatch(handle.offset(), k)) {
  202. // // Not found
  203. // } else {
  204. // Iterator* block_iter = BlockReader(this, options, iiter->value());
  205. // block_iter->Seek(k);
  206. // if (block_iter->Valid()) {
  207. // (*handle_result)(arg, block_iter->key(), block_iter->value());
  208. // }
  209. // s = block_iter->status();
  210. // delete block_iter;
  211. // }
  212. // }
  213. // if (s.ok()) {
  214. // s = iiter->status();
  215. // }
  216. // delete iiter;
  217. // return s;
  218. // }
  219. Status Table::InternalGet(const ReadOptions& options, const Slice& k, void* arg,
  220. void (*handle_result)(void*, const Slice&,
  221. const Slice&)) {
  222. Status s;
  223. Iterator* iiter = rep_->index_block->NewIterator(rep_->options.comparator);
  224. iiter->Seek(k);
  225. if (iiter->Valid()) {
  226. Slice handle_value = iiter->value();
  227. FilterBlockReader* filter = rep_->filter;
  228. BlockHandle handle;
  229. if (filter != nullptr && handle.DecodeFrom(&handle_value).ok() &&
  230. !filter->KeyMayMatch(handle.offset(), k)) {
  231. // Not found
  232. } else {
  233. Iterator* block_iter = BlockReader(this, options, iiter->value());
  234. block_iter->Seek(k);
  235. if (block_iter->Valid()) {
  236. // 这里获取存储的组合字符串
  237. s = Status::OK(); // 确保状态为 OK
  238. Slice combined_value = block_iter->value();
  239. // 获取实际的值和过期时间
  240. std::string combined_str = combined_value.ToString();
  241. // 假设过期时间是字符串的最后19个字符
  242. std::string expiration_time_str = combined_str.substr(combined_str.size() - 19, 19); // 获取过期时间字符串
  243. std::string actual_value = combined_str.substr(0, combined_str.size() - 19); // 获取实际值
  244. // 解析过期时间为时间戳
  245. std::tm tm = {};
  246. std::istringstream ss(expiration_time_str);
  247. ss >> std::get_time(&tm, "%Y-%m-%d %H:%M:%S");
  248. std::time_t expiration_time = std::mktime(&tm);
  249. // 获取当前时间并与过期时间进行比较
  250. auto now = std::chrono::system_clock::now();
  251. auto now_time_t = std::chrono::system_clock::to_time_t(now);
  252. // 检查是否过期
  253. if (expiration_time > now_time_t) {
  254. // 调用结果处理函数,返回实际值
  255. (*handle_result)(arg, block_iter->key(), Slice(actual_value));//, Slice(expiration_time_str));
  256. s = block_iter->status();
  257. if (! s.ok()) printf("291\n");
  258. } else {
  259. // 数据已过期,处理过期情况
  260. s = Status::NotFound("Key has expired");
  261. }
  262. }
  263. else {
  264. s = block_iter->status();
  265. if (! s.ok()) printf("299\n");
  266. }
  267. delete block_iter;
  268. }
  269. }
  270. if (s.ok()) {
  271. s = iiter->status();
  272. if (! s.ok()) printf("306\n");
  273. }
  274. delete iiter;
  275. if (! s.ok()) printf("!!!!!!!!!!!"); // 如果不在sstable里
  276. return s;
  277. }
  278. uint64_t Table::ApproximateOffsetOf(const Slice& key) const {
  279. Iterator* index_iter =
  280. rep_->index_block->NewIterator(rep_->options.comparator);
  281. index_iter->Seek(key);
  282. uint64_t result;
  283. if (index_iter->Valid()) {
  284. BlockHandle handle;
  285. Slice input = index_iter->value();
  286. Status s = handle.DecodeFrom(&input);
  287. if (s.ok()) {
  288. result = handle.offset();
  289. } else {
  290. // Strange: we can't decode the block handle in the index block.
  291. // We'll just return the offset of the metaindex block, which is
  292. // close to the whole file size for this case.
  293. result = rep_->metaindex_handle.offset();
  294. }
  295. } else {
  296. // key is past the last key in the file. Approximate the offset
  297. // by returning the offset of the metaindex block (which is
  298. // right near the end of the file).
  299. result = rep_->metaindex_handle.offset();
  300. }
  301. delete index_iter;
  302. return result;
  303. }
  304. } // namespace leveldb