作者: 韩晨旭 10225101440 李畅 10225102463

550 Zeilen
15 KiB

Add Env::Remove{File,Dir} which obsolete Env::Delete{File,Dir}. The "DeleteFile" method name causes pain for Windows developers, because <windows.h> #defines a DeleteFile macro to DeleteFileW or DeleteFileA. Current code uses workarounds, like #undefining DeleteFile everywhere an Env is declared, implemented, or used. This CL removes the need for workarounds by renaming Env::DeleteFile to Env::RemoveFile. For consistency, Env::DeleteDir is also renamed to Env::RemoveDir. A few internal methods are also renamed for consistency. Software that supports Windows is expected to migrate any Env implementations and usage to Remove{File,Dir}, and never use the name Env::Delete{File,Dir} in its code. The renaming is done in a backwards-compatible way, at the risk of making it slightly more difficult to build a new correct Env implementation. The backwards compatibility is achieved using the following hacks: 1) Env::Remove{File,Dir} methods are added, with a default implementation that calls into Env::Delete{File,Dir}. This makes old Env implementations compatible with code that calls into the updated API. 2) The Env::Delete{File,Dir} methods are no longer pure virtuals. Instead, they gain a default implementation that calls into Env::Remove{File,Dir}. This makes updated Env implementations compatible with code that calls into the old API. The cost of this approach is that it's possible to write an Env without overriding either Rename{File,Dir} or Delete{File,Dir}, without getting a compiler warning. However, attempting to run the test suite will immediately fail with an infinite call stack ending in {Remove,Delete}{File,Dir}, making developers aware of the problem. PiperOrigin-RevId: 288710907
vor 5 Jahren
Add Env::Remove{File,Dir} which obsolete Env::Delete{File,Dir}. The "DeleteFile" method name causes pain for Windows developers, because <windows.h> #defines a DeleteFile macro to DeleteFileW or DeleteFileA. Current code uses workarounds, like #undefining DeleteFile everywhere an Env is declared, implemented, or used. This CL removes the need for workarounds by renaming Env::DeleteFile to Env::RemoveFile. For consistency, Env::DeleteDir is also renamed to Env::RemoveDir. A few internal methods are also renamed for consistency. Software that supports Windows is expected to migrate any Env implementations and usage to Remove{File,Dir}, and never use the name Env::Delete{File,Dir} in its code. The renaming is done in a backwards-compatible way, at the risk of making it slightly more difficult to build a new correct Env implementation. The backwards compatibility is achieved using the following hacks: 1) Env::Remove{File,Dir} methods are added, with a default implementation that calls into Env::Delete{File,Dir}. This makes old Env implementations compatible with code that calls into the updated API. 2) The Env::Delete{File,Dir} methods are no longer pure virtuals. Instead, they gain a default implementation that calls into Env::Remove{File,Dir}. This makes updated Env implementations compatible with code that calls into the old API. The cost of this approach is that it's possible to write an Env without overriding either Rename{File,Dir} or Delete{File,Dir}, without getting a compiler warning. However, attempting to run the test suite will immediately fail with an infinite call stack ending in {Remove,Delete}{File,Dir}, making developers aware of the problem. PiperOrigin-RevId: 288710907
vor 5 Jahren
Add Env::Remove{File,Dir} which obsolete Env::Delete{File,Dir}. The "DeleteFile" method name causes pain for Windows developers, because <windows.h> #defines a DeleteFile macro to DeleteFileW or DeleteFileA. Current code uses workarounds, like #undefining DeleteFile everywhere an Env is declared, implemented, or used. This CL removes the need for workarounds by renaming Env::DeleteFile to Env::RemoveFile. For consistency, Env::DeleteDir is also renamed to Env::RemoveDir. A few internal methods are also renamed for consistency. Software that supports Windows is expected to migrate any Env implementations and usage to Remove{File,Dir}, and never use the name Env::Delete{File,Dir} in its code. The renaming is done in a backwards-compatible way, at the risk of making it slightly more difficult to build a new correct Env implementation. The backwards compatibility is achieved using the following hacks: 1) Env::Remove{File,Dir} methods are added, with a default implementation that calls into Env::Delete{File,Dir}. This makes old Env implementations compatible with code that calls into the updated API. 2) The Env::Delete{File,Dir} methods are no longer pure virtuals. Instead, they gain a default implementation that calls into Env::Remove{File,Dir}. This makes updated Env implementations compatible with code that calls into the old API. The cost of this approach is that it's possible to write an Env without overriding either Rename{File,Dir} or Delete{File,Dir}, without getting a compiler warning. However, attempting to run the test suite will immediately fail with an infinite call stack ending in {Remove,Delete}{File,Dir}, making developers aware of the problem. PiperOrigin-RevId: 288710907
vor 5 Jahren
Add Env::Remove{File,Dir} which obsolete Env::Delete{File,Dir}. The "DeleteFile" method name causes pain for Windows developers, because <windows.h> #defines a DeleteFile macro to DeleteFileW or DeleteFileA. Current code uses workarounds, like #undefining DeleteFile everywhere an Env is declared, implemented, or used. This CL removes the need for workarounds by renaming Env::DeleteFile to Env::RemoveFile. For consistency, Env::DeleteDir is also renamed to Env::RemoveDir. A few internal methods are also renamed for consistency. Software that supports Windows is expected to migrate any Env implementations and usage to Remove{File,Dir}, and never use the name Env::Delete{File,Dir} in its code. The renaming is done in a backwards-compatible way, at the risk of making it slightly more difficult to build a new correct Env implementation. The backwards compatibility is achieved using the following hacks: 1) Env::Remove{File,Dir} methods are added, with a default implementation that calls into Env::Delete{File,Dir}. This makes old Env implementations compatible with code that calls into the updated API. 2) The Env::Delete{File,Dir} methods are no longer pure virtuals. Instead, they gain a default implementation that calls into Env::Remove{File,Dir}. This makes updated Env implementations compatible with code that calls into the old API. The cost of this approach is that it's possible to write an Env without overriding either Rename{File,Dir} or Delete{File,Dir}, without getting a compiler warning. However, attempting to run the test suite will immediately fail with an infinite call stack ending in {Remove,Delete}{File,Dir}, making developers aware of the problem. PiperOrigin-RevId: 288710907
vor 5 Jahren
Add Env::Remove{File,Dir} which obsolete Env::Delete{File,Dir}. The "DeleteFile" method name causes pain for Windows developers, because <windows.h> #defines a DeleteFile macro to DeleteFileW or DeleteFileA. Current code uses workarounds, like #undefining DeleteFile everywhere an Env is declared, implemented, or used. This CL removes the need for workarounds by renaming Env::DeleteFile to Env::RemoveFile. For consistency, Env::DeleteDir is also renamed to Env::RemoveDir. A few internal methods are also renamed for consistency. Software that supports Windows is expected to migrate any Env implementations and usage to Remove{File,Dir}, and never use the name Env::Delete{File,Dir} in its code. The renaming is done in a backwards-compatible way, at the risk of making it slightly more difficult to build a new correct Env implementation. The backwards compatibility is achieved using the following hacks: 1) Env::Remove{File,Dir} methods are added, with a default implementation that calls into Env::Delete{File,Dir}. This makes old Env implementations compatible with code that calls into the updated API. 2) The Env::Delete{File,Dir} methods are no longer pure virtuals. Instead, they gain a default implementation that calls into Env::Remove{File,Dir}. This makes updated Env implementations compatible with code that calls into the old API. The cost of this approach is that it's possible to write an Env without overriding either Rename{File,Dir} or Delete{File,Dir}, without getting a compiler warning. However, attempting to run the test suite will immediately fail with an infinite call stack ending in {Remove,Delete}{File,Dir}, making developers aware of the problem. PiperOrigin-RevId: 288710907
vor 5 Jahren
Add Env::Remove{File,Dir} which obsolete Env::Delete{File,Dir}. The "DeleteFile" method name causes pain for Windows developers, because <windows.h> #defines a DeleteFile macro to DeleteFileW or DeleteFileA. Current code uses workarounds, like #undefining DeleteFile everywhere an Env is declared, implemented, or used. This CL removes the need for workarounds by renaming Env::DeleteFile to Env::RemoveFile. For consistency, Env::DeleteDir is also renamed to Env::RemoveDir. A few internal methods are also renamed for consistency. Software that supports Windows is expected to migrate any Env implementations and usage to Remove{File,Dir}, and never use the name Env::Delete{File,Dir} in its code. The renaming is done in a backwards-compatible way, at the risk of making it slightly more difficult to build a new correct Env implementation. The backwards compatibility is achieved using the following hacks: 1) Env::Remove{File,Dir} methods are added, with a default implementation that calls into Env::Delete{File,Dir}. This makes old Env implementations compatible with code that calls into the updated API. 2) The Env::Delete{File,Dir} methods are no longer pure virtuals. Instead, they gain a default implementation that calls into Env::Remove{File,Dir}. This makes updated Env implementations compatible with code that calls into the old API. The cost of this approach is that it's possible to write an Env without overriding either Rename{File,Dir} or Delete{File,Dir}, without getting a compiler warning. However, attempting to run the test suite will immediately fail with an infinite call stack ending in {Remove,Delete}{File,Dir}, making developers aware of the problem. PiperOrigin-RevId: 288710907
vor 5 Jahren
Add Env::Remove{File,Dir} which obsolete Env::Delete{File,Dir}. The "DeleteFile" method name causes pain for Windows developers, because <windows.h> #defines a DeleteFile macro to DeleteFileW or DeleteFileA. Current code uses workarounds, like #undefining DeleteFile everywhere an Env is declared, implemented, or used. This CL removes the need for workarounds by renaming Env::DeleteFile to Env::RemoveFile. For consistency, Env::DeleteDir is also renamed to Env::RemoveDir. A few internal methods are also renamed for consistency. Software that supports Windows is expected to migrate any Env implementations and usage to Remove{File,Dir}, and never use the name Env::Delete{File,Dir} in its code. The renaming is done in a backwards-compatible way, at the risk of making it slightly more difficult to build a new correct Env implementation. The backwards compatibility is achieved using the following hacks: 1) Env::Remove{File,Dir} methods are added, with a default implementation that calls into Env::Delete{File,Dir}. This makes old Env implementations compatible with code that calls into the updated API. 2) The Env::Delete{File,Dir} methods are no longer pure virtuals. Instead, they gain a default implementation that calls into Env::Remove{File,Dir}. This makes updated Env implementations compatible with code that calls into the old API. The cost of this approach is that it's possible to write an Env without overriding either Rename{File,Dir} or Delete{File,Dir}, without getting a compiler warning. However, attempting to run the test suite will immediately fail with an infinite call stack ending in {Remove,Delete}{File,Dir}, making developers aware of the problem. PiperOrigin-RevId: 288710907
vor 5 Jahren
  1. // Copyright 2014 The LevelDB Authors. All rights reserved.
  2. // Use of this source code is governed by a BSD-style license that can be
  3. // found in the LICENSE file. See the AUTHORS file for names of contributors.
  4. // This test uses a custom Env to keep track of the state of a filesystem as of
  5. // the last "sync". It then checks for data loss errors by purposely dropping
  6. // file data (or entire files) not protected by a "sync".
  7. #include <map>
  8. #include <set>
  9. #include "gtest/gtest.h"
  10. #include "db/db_impl.h"
  11. #include "db/filename.h"
  12. #include "db/log_format.h"
  13. #include "db/version_set.h"
  14. #include "leveldb/cache.h"
  15. #include "leveldb/db.h"
  16. #include "leveldb/env.h"
  17. #include "leveldb/table.h"
  18. #include "leveldb/write_batch.h"
  19. #include "port/port.h"
  20. #include "port/thread_annotations.h"
  21. #include "util/logging.h"
  22. #include "util/mutexlock.h"
  23. #include "util/testutil.h"
  24. namespace leveldb {
  25. static const int kValueSize = 1000;
  26. static const int kMaxNumValues = 2000;
  27. static const size_t kNumIterations = 3;
  28. class FaultInjectionTestEnv;
  29. namespace {
  30. // Assume a filename, and not a directory name like "/foo/bar/"
  31. static std::string GetDirName(const std::string& filename) {
  32. size_t found = filename.find_last_of("/\\");
  33. if (found == std::string::npos) {
  34. return "";
  35. } else {
  36. return filename.substr(0, found);
  37. }
  38. }
  39. Status SyncDir(const std::string& dir) {
  40. // As this is a test it isn't required to *actually* sync this directory.
  41. return Status::OK();
  42. }
  43. // A basic file truncation function suitable for this test.
  44. Status Truncate(const std::string& filename, uint64_t length) {
  45. leveldb::Env* env = leveldb::Env::Default();
  46. SequentialFile* orig_file;
  47. Status s = env->NewSequentialFile(filename, &orig_file);
  48. if (!s.ok()) return s;
  49. char* scratch = new char[length];
  50. leveldb::Slice result;
  51. s = orig_file->Read(length, &result, scratch);
  52. delete orig_file;
  53. if (s.ok()) {
  54. std::string tmp_name = GetDirName(filename) + "/truncate.tmp";
  55. WritableFile* tmp_file;
  56. s = env->NewWritableFile(tmp_name, &tmp_file);
  57. if (s.ok()) {
  58. s = tmp_file->Append(result);
  59. delete tmp_file;
  60. if (s.ok()) {
  61. s = env->RenameFile(tmp_name, filename);
  62. } else {
  63. env->RemoveFile(tmp_name);
  64. }
  65. }
  66. }
  67. delete[] scratch;
  68. return s;
  69. }
  70. struct FileState {
  71. std::string filename_;
  72. int64_t pos_;
  73. int64_t pos_at_last_sync_;
  74. int64_t pos_at_last_flush_;
  75. FileState(const std::string& filename)
  76. : filename_(filename),
  77. pos_(-1),
  78. pos_at_last_sync_(-1),
  79. pos_at_last_flush_(-1) {}
  80. FileState() : pos_(-1), pos_at_last_sync_(-1), pos_at_last_flush_(-1) {}
  81. bool IsFullySynced() const { return pos_ <= 0 || pos_ == pos_at_last_sync_; }
  82. Status DropUnsyncedData() const;
  83. };
  84. } // anonymous namespace
  85. // A wrapper around WritableFile which informs another Env whenever this file
  86. // is written to or sync'ed.
  87. class TestWritableFile : public WritableFile {
  88. public:
  89. TestWritableFile(const FileState& state, WritableFile* f,
  90. FaultInjectionTestEnv* env);
  91. ~TestWritableFile() override;
  92. Status Append(const Slice& data) override;
  93. Status Close() override;
  94. Status Flush() override;
  95. Status Sync() override;
  96. private:
  97. FileState state_;
  98. WritableFile* target_;
  99. bool writable_file_opened_;
  100. FaultInjectionTestEnv* env_;
  101. Status SyncParent();
  102. };
  103. class FaultInjectionTestEnv : public EnvWrapper {
  104. public:
  105. FaultInjectionTestEnv()
  106. : EnvWrapper(Env::Default()), filesystem_active_(true) {}
  107. ~FaultInjectionTestEnv() override = default;
  108. Status NewWritableFile(const std::string& fname,
  109. WritableFile** result) override;
  110. Status NewAppendableFile(const std::string& fname,
  111. WritableFile** result) override;
  112. Status RemoveFile(const std::string& f) override;
  113. Status RenameFile(const std::string& s, const std::string& t) override;
  114. void WritableFileClosed(const FileState& state);
  115. Status DropUnsyncedFileData();
  116. Status RemoveFilesCreatedAfterLastDirSync();
  117. void DirWasSynced();
  118. bool IsFileCreatedSinceLastDirSync(const std::string& filename);
  119. void ResetState();
  120. void UntrackFile(const std::string& f);
  121. // Setting the filesystem to inactive is the test equivalent to simulating a
  122. // system reset. Setting to inactive will freeze our saved filesystem state so
  123. // that it will stop being recorded. It can then be reset back to the state at
  124. // the time of the reset.
  125. bool IsFilesystemActive() LOCKS_EXCLUDED(mutex_) {
  126. MutexLock l(&mutex_);
  127. return filesystem_active_;
  128. }
  129. void SetFilesystemActive(bool active) LOCKS_EXCLUDED(mutex_) {
  130. MutexLock l(&mutex_);
  131. filesystem_active_ = active;
  132. }
  133. private:
  134. port::Mutex mutex_;
  135. std::map<std::string, FileState> db_file_state_ GUARDED_BY(mutex_);
  136. std::set<std::string> new_files_since_last_dir_sync_ GUARDED_BY(mutex_);
  137. bool filesystem_active_ GUARDED_BY(mutex_); // Record flushes, syncs, writes
  138. };
  139. TestWritableFile::TestWritableFile(const FileState& state, WritableFile* f,
  140. FaultInjectionTestEnv* env)
  141. : state_(state), target_(f), writable_file_opened_(true), env_(env) {
  142. assert(f != nullptr);
  143. }
  144. TestWritableFile::~TestWritableFile() {
  145. if (writable_file_opened_) {
  146. Close();
  147. }
  148. delete target_;
  149. }
  150. Status TestWritableFile::Append(const Slice& data) {
  151. Status s = target_->Append(data);
  152. if (s.ok() && env_->IsFilesystemActive()) {
  153. state_.pos_ += data.size();
  154. }
  155. return s;
  156. }
  157. Status TestWritableFile::Close() {
  158. writable_file_opened_ = false;
  159. Status s = target_->Close();
  160. if (s.ok()) {
  161. env_->WritableFileClosed(state_);
  162. }
  163. return s;
  164. }
  165. Status TestWritableFile::Flush() {
  166. Status s = target_->Flush();
  167. if (s.ok() && env_->IsFilesystemActive()) {
  168. state_.pos_at_last_flush_ = state_.pos_;
  169. }
  170. return s;
  171. }
  172. Status TestWritableFile::SyncParent() {
  173. Status s = SyncDir(GetDirName(state_.filename_));
  174. if (s.ok()) {
  175. env_->DirWasSynced();
  176. }
  177. return s;
  178. }
  179. Status TestWritableFile::Sync() {
  180. if (!env_->IsFilesystemActive()) {
  181. return Status::OK();
  182. }
  183. // Ensure new files referred to by the manifest are in the filesystem.
  184. Status s = target_->Sync();
  185. if (s.ok()) {
  186. state_.pos_at_last_sync_ = state_.pos_;
  187. }
  188. if (env_->IsFileCreatedSinceLastDirSync(state_.filename_)) {
  189. Status ps = SyncParent();
  190. if (s.ok() && !ps.ok()) {
  191. s = ps;
  192. }
  193. }
  194. return s;
  195. }
  196. Status FaultInjectionTestEnv::NewWritableFile(const std::string& fname,
  197. WritableFile** result) {
  198. WritableFile* actual_writable_file;
  199. Status s = target()->NewWritableFile(fname, &actual_writable_file);
  200. if (s.ok()) {
  201. FileState state(fname);
  202. state.pos_ = 0;
  203. *result = new TestWritableFile(state, actual_writable_file, this);
  204. // NewWritableFile doesn't append to files, so if the same file is
  205. // opened again then it will be truncated - so forget our saved
  206. // state.
  207. UntrackFile(fname);
  208. MutexLock l(&mutex_);
  209. new_files_since_last_dir_sync_.insert(fname);
  210. }
  211. return s;
  212. }
  213. Status FaultInjectionTestEnv::NewAppendableFile(const std::string& fname,
  214. WritableFile** result) {
  215. WritableFile* actual_writable_file;
  216. Status s = target()->NewAppendableFile(fname, &actual_writable_file);
  217. if (s.ok()) {
  218. FileState state(fname);
  219. state.pos_ = 0;
  220. {
  221. MutexLock l(&mutex_);
  222. if (db_file_state_.count(fname) == 0) {
  223. new_files_since_last_dir_sync_.insert(fname);
  224. } else {
  225. state = db_file_state_[fname];
  226. }
  227. }
  228. *result = new TestWritableFile(state, actual_writable_file, this);
  229. }
  230. return s;
  231. }
  232. Status FaultInjectionTestEnv::DropUnsyncedFileData() {
  233. Status s;
  234. MutexLock l(&mutex_);
  235. for (const auto& kvp : db_file_state_) {
  236. if (!s.ok()) {
  237. break;
  238. }
  239. const FileState& state = kvp.second;
  240. if (!state.IsFullySynced()) {
  241. s = state.DropUnsyncedData();
  242. }
  243. }
  244. return s;
  245. }
  246. void FaultInjectionTestEnv::DirWasSynced() {
  247. MutexLock l(&mutex_);
  248. new_files_since_last_dir_sync_.clear();
  249. }
  250. bool FaultInjectionTestEnv::IsFileCreatedSinceLastDirSync(
  251. const std::string& filename) {
  252. MutexLock l(&mutex_);
  253. return new_files_since_last_dir_sync_.find(filename) !=
  254. new_files_since_last_dir_sync_.end();
  255. }
  256. void FaultInjectionTestEnv::UntrackFile(const std::string& f) {
  257. MutexLock l(&mutex_);
  258. db_file_state_.erase(f);
  259. new_files_since_last_dir_sync_.erase(f);
  260. }
  261. Status FaultInjectionTestEnv::RemoveFile(const std::string& f) {
  262. Status s = EnvWrapper::RemoveFile(f);
  263. EXPECT_LEVELDB_OK(s);
  264. if (s.ok()) {
  265. UntrackFile(f);
  266. }
  267. return s;
  268. }
  269. Status FaultInjectionTestEnv::RenameFile(const std::string& s,
  270. const std::string& t) {
  271. Status ret = EnvWrapper::RenameFile(s, t);
  272. if (ret.ok()) {
  273. MutexLock l(&mutex_);
  274. if (db_file_state_.find(s) != db_file_state_.end()) {
  275. db_file_state_[t] = db_file_state_[s];
  276. db_file_state_.erase(s);
  277. }
  278. if (new_files_since_last_dir_sync_.erase(s) != 0) {
  279. assert(new_files_since_last_dir_sync_.find(t) ==
  280. new_files_since_last_dir_sync_.end());
  281. new_files_since_last_dir_sync_.insert(t);
  282. }
  283. }
  284. return ret;
  285. }
  286. void FaultInjectionTestEnv::ResetState() {
  287. // Since we are not destroying the database, the existing files
  288. // should keep their recorded synced/flushed state. Therefore
  289. // we do not reset db_file_state_ and new_files_since_last_dir_sync_.
  290. SetFilesystemActive(true);
  291. }
  292. Status FaultInjectionTestEnv::RemoveFilesCreatedAfterLastDirSync() {
  293. // Because RemoveFile access this container make a copy to avoid deadlock
  294. mutex_.Lock();
  295. std::set<std::string> new_files(new_files_since_last_dir_sync_.begin(),
  296. new_files_since_last_dir_sync_.end());
  297. mutex_.Unlock();
  298. Status status;
  299. for (const auto& new_file : new_files) {
  300. Status remove_status = RemoveFile(new_file);
  301. if (!remove_status.ok() && status.ok()) {
  302. status = std::move(remove_status);
  303. }
  304. }
  305. return status;
  306. }
  307. void FaultInjectionTestEnv::WritableFileClosed(const FileState& state) {
  308. MutexLock l(&mutex_);
  309. db_file_state_[state.filename_] = state;
  310. }
  311. Status FileState::DropUnsyncedData() const {
  312. int64_t sync_pos = pos_at_last_sync_ == -1 ? 0 : pos_at_last_sync_;
  313. return Truncate(filename_, sync_pos);
  314. }
  315. class FaultInjectionTest : public testing::Test {
  316. public:
  317. enum ExpectedVerifResult { VAL_EXPECT_NO_ERROR, VAL_EXPECT_ERROR };
  318. enum ResetMethod { RESET_DROP_UNSYNCED_DATA, RESET_DELETE_UNSYNCED_FILES };
  319. FaultInjectionTestEnv* env_;
  320. std::string dbname_;
  321. Cache* tiny_cache_;
  322. Options options_;
  323. DB* db_;
  324. FaultInjectionTest()
  325. : env_(new FaultInjectionTestEnv),
  326. tiny_cache_(NewLRUCache(100)),
  327. db_(nullptr) {
  328. dbname_ = testing::TempDir() + "fault_test";
  329. DestroyDB(dbname_, Options()); // Destroy any db from earlier run
  330. options_.reuse_logs = true;
  331. options_.env = env_;
  332. options_.paranoid_checks = true;
  333. options_.block_cache = tiny_cache_;
  334. options_.create_if_missing = true;
  335. }
  336. ~FaultInjectionTest() {
  337. CloseDB();
  338. DestroyDB(dbname_, Options());
  339. delete tiny_cache_;
  340. delete env_;
  341. }
  342. void ReuseLogs(bool reuse) { options_.reuse_logs = reuse; }
  343. void Build(int start_idx, int num_vals) {
  344. std::string key_space, value_space;
  345. WriteBatch batch;
  346. for (int i = start_idx; i < start_idx + num_vals; i++) {
  347. Slice key = Key(i, &key_space);
  348. batch.Clear();
  349. batch.Put(key, Value(i, &value_space));
  350. WriteOptions options;
  351. ASSERT_LEVELDB_OK(db_->Write(options, &batch));
  352. }
  353. }
  354. Status ReadValue(int i, std::string* val) const {
  355. std::string key_space, value_space;
  356. Slice key = Key(i, &key_space);
  357. Value(i, &value_space);
  358. ReadOptions options;
  359. return db_->Get(options, key, val);
  360. }
  361. Status Verify(int start_idx, int num_vals,
  362. ExpectedVerifResult expected) const {
  363. std::string val;
  364. std::string value_space;
  365. Status s;
  366. for (int i = start_idx; i < start_idx + num_vals && s.ok(); i++) {
  367. Value(i, &value_space);
  368. s = ReadValue(i, &val);
  369. if (expected == VAL_EXPECT_NO_ERROR) {
  370. if (s.ok()) {
  371. EXPECT_EQ(value_space, val);
  372. }
  373. } else if (s.ok()) {
  374. std::fprintf(stderr, "Expected an error at %d, but was OK\n", i);
  375. s = Status::IOError(dbname_, "Expected value error:");
  376. } else {
  377. s = Status::OK(); // An expected error
  378. }
  379. }
  380. return s;
  381. }
  382. // Return the ith key
  383. Slice Key(int i, std::string* storage) const {
  384. char buf[100];
  385. std::snprintf(buf, sizeof(buf), "%016d", i);
  386. storage->assign(buf, strlen(buf));
  387. return Slice(*storage);
  388. }
  389. // Return the value to associate with the specified key
  390. Slice Value(int k, std::string* storage) const {
  391. Random r(k);
  392. return test::RandomString(&r, kValueSize, storage);
  393. }
  394. Status OpenDB() {
  395. delete db_;
  396. db_ = nullptr;
  397. env_->ResetState();
  398. return DB::Open(options_, dbname_, &db_);
  399. }
  400. void CloseDB() {
  401. delete db_;
  402. db_ = nullptr;
  403. }
  404. void DeleteAllData() {
  405. Iterator* iter = db_->NewIterator(ReadOptions());
  406. for (iter->SeekToFirst(); iter->Valid(); iter->Next()) {
  407. ASSERT_LEVELDB_OK(db_->Delete(WriteOptions(), iter->key()));
  408. }
  409. delete iter;
  410. }
  411. void ResetDBState(ResetMethod reset_method) {
  412. switch (reset_method) {
  413. case RESET_DROP_UNSYNCED_DATA:
  414. ASSERT_LEVELDB_OK(env_->DropUnsyncedFileData());
  415. break;
  416. case RESET_DELETE_UNSYNCED_FILES:
  417. ASSERT_LEVELDB_OK(env_->RemoveFilesCreatedAfterLastDirSync());
  418. break;
  419. default:
  420. assert(false);
  421. }
  422. }
  423. void PartialCompactTestPreFault(int num_pre_sync, int num_post_sync) {
  424. DeleteAllData();
  425. Build(0, num_pre_sync);
  426. db_->CompactRange(nullptr, nullptr);
  427. Build(num_pre_sync, num_post_sync);
  428. }
  429. void PartialCompactTestReopenWithFault(ResetMethod reset_method,
  430. int num_pre_sync, int num_post_sync) {
  431. env_->SetFilesystemActive(false);
  432. CloseDB();
  433. ResetDBState(reset_method);
  434. ASSERT_LEVELDB_OK(OpenDB());
  435. ASSERT_LEVELDB_OK(
  436. Verify(0, num_pre_sync, FaultInjectionTest::VAL_EXPECT_NO_ERROR));
  437. ASSERT_LEVELDB_OK(Verify(num_pre_sync, num_post_sync,
  438. FaultInjectionTest::VAL_EXPECT_ERROR));
  439. }
  440. void NoWriteTestPreFault() {}
  441. void NoWriteTestReopenWithFault(ResetMethod reset_method) {
  442. CloseDB();
  443. ResetDBState(reset_method);
  444. ASSERT_LEVELDB_OK(OpenDB());
  445. }
  446. void DoTest() {
  447. Random rnd(0);
  448. ASSERT_LEVELDB_OK(OpenDB());
  449. for (size_t idx = 0; idx < kNumIterations; idx++) {
  450. int num_pre_sync = rnd.Uniform(kMaxNumValues);
  451. int num_post_sync = rnd.Uniform(kMaxNumValues);
  452. PartialCompactTestPreFault(num_pre_sync, num_post_sync);
  453. PartialCompactTestReopenWithFault(RESET_DROP_UNSYNCED_DATA, num_pre_sync,
  454. num_post_sync);
  455. NoWriteTestPreFault();
  456. NoWriteTestReopenWithFault(RESET_DROP_UNSYNCED_DATA);
  457. PartialCompactTestPreFault(num_pre_sync, num_post_sync);
  458. // No new files created so we expect all values since no files will be
  459. // dropped.
  460. PartialCompactTestReopenWithFault(RESET_DELETE_UNSYNCED_FILES,
  461. num_pre_sync + num_post_sync, 0);
  462. NoWriteTestPreFault();
  463. NoWriteTestReopenWithFault(RESET_DELETE_UNSYNCED_FILES);
  464. }
  465. }
  466. };
  467. TEST_F(FaultInjectionTest, FaultTestNoLogReuse) {
  468. ReuseLogs(false);
  469. DoTest();
  470. }
  471. TEST_F(FaultInjectionTest, FaultTestWithLogReuse) {
  472. ReuseLogs(true);
  473. DoTest();
  474. }
  475. } // namespace leveldb