Du kan inte välja fler än 25 ämnen Ämnen måste starta med en bokstav eller siffra, kan innehålla bindestreck ('-') och vara max 35 tecken långa.

387 rader
14 KiB

7 år sedan
Release 1.18 Changes are: * Update version number to 1.18 * Replace the basic fprintf call with a call to fwrite in order to work around the apparent compiler optimization/rewrite failure that we are seeing with the new toolchain/iOS SDKs provided with Xcode6 and iOS8. * Fix ALL the header guards. * Createed a README.md with the LevelDB project description. * A new CONTRIBUTING file. * Don't implicitly convert uint64_t to size_t or int. Either preserve it as uint64_t, or explicitly cast. This fixes MSVC warnings about possible value truncation when compiling this code in Chromium. * Added a DumpFile() library function that encapsulates the guts of the "leveldbutil dump" command. This will allow clients to dump data to their log files instead of stdout. It will also allow clients to supply their own environment. * leveldb: Remove unused function 'ConsumeChar'. * leveldbutil: Remove unused member variables from WriteBatchItemPrinter. * OpenBSD, NetBSD and DragonflyBSD have _LITTLE_ENDIAN, so define PLATFORM_IS_LITTLE_ENDIAN like on FreeBSD. This fixes: * issue #143 * issue #198 * issue #249 * Switch from <cstdatomic> to <atomic>. The former never made it into the standard and doesn't exist in modern gcc versions at all. The later contains everything that leveldb was using from the former. This problem was noticed when porting to Portable Native Client where no memory barrier is defined. The fact that <cstdatomic> is missing normally goes unnoticed since memory barriers are defined for most architectures. * Make Hash() treat its input as unsigned. Before this change LevelDB files from platforms with different signedness of char were not compatible. This change fixes: issue #243 * Verify checksums of index/meta/filter blocks when paranoid_checks set. * Invoke all tools for iOS with xcrun. (This was causing problems with the new XCode 5.1.1 image on pulse.) * include <sys/stat.h> only once, and fix the following linter warning: "Found C system header after C++ system header" * When encountering a corrupted table file, return Status::Corruption instead of Status::InvalidArgument. * Support cygwin as build platform, patch is from https://code.google.com/p/leveldb/issues/detail?id=188 * Fix typo, merge patch from https://code.google.com/p/leveldb/issues/detail?id=159 * Fix typos and comments, and address the following two issues: * issue #166 * issue #241 * Add missing db synchronize after "fillseq" in the benchmark. * Removed unused variable in SeekRandom: value (issue #201)
10 år sedan
  1. // Copyright (c) 2011 The LevelDB Authors. All rights reserved.
  2. // Use of this source code is governed by a BSD-style license that can be
  3. // found in the LICENSE file. See the AUTHORS file for names of contributors.
  4. //
  5. // An Env is an interface used by the leveldb implementation to access
  6. // operating system functionality like the filesystem etc. Callers
  7. // may wish to provide a custom Env object when opening a database to
  8. // get fine gain control; e.g., to rate limit file system operations.
  9. //
  10. // All Env implementations are safe for concurrent access from
  11. // multiple threads without any external synchronization.
  12. #ifndef STORAGE_LEVELDB_INCLUDE_ENV_H_
  13. #define STORAGE_LEVELDB_INCLUDE_ENV_H_
  14. #include <stdarg.h>
  15. #include <stdint.h>
  16. #include <string>
  17. #include <vector>
  18. #include "leveldb/export.h"
  19. #include "leveldb/status.h"
  20. #if defined(_WIN32)
  21. // The leveldb::Env class below contains a DeleteFile method.
  22. // At the same time, <windows.h>, a fairly popular header
  23. // file for Windows applications, defines a DeleteFile macro.
  24. //
  25. // Without any intervention on our part, the result of this
  26. // unfortunate coincidence is that the name of the
  27. // leveldb::Env::DeleteFile method seen by the compiler depends on
  28. // whether <windows.h> was included before or after the LevelDB
  29. // headers.
  30. //
  31. // To avoid headaches, we undefined DeleteFile (if defined) and
  32. // redefine it at the bottom of this file. This way <windows.h>
  33. // can be included before this file (or not at all) and the
  34. // exported method will always be leveldb::Env::DeleteFile.
  35. #if defined(DeleteFile)
  36. #undef DeleteFile
  37. #define LEVELDB_DELETEFILE_UNDEFINED
  38. #endif // defined(DeleteFile)
  39. #endif // defined(_WIN32)
  40. namespace leveldb {
  41. class FileLock;
  42. class Logger;
  43. class RandomAccessFile;
  44. class SequentialFile;
  45. class Slice;
  46. class WritableFile;
  47. class LEVELDB_EXPORT Env {
  48. public:
  49. Env() = default;
  50. Env(const Env&) = delete;
  51. Env& operator=(const Env&) = delete;
  52. virtual ~Env();
  53. // Return a default environment suitable for the current operating
  54. // system. Sophisticated users may wish to provide their own Env
  55. // implementation instead of relying on this default environment.
  56. //
  57. // The result of Default() belongs to leveldb and must never be deleted.
  58. static Env* Default();
  59. // Create an object that sequentially reads the file with the specified name.
  60. // On success, stores a pointer to the new file in *result and returns OK.
  61. // On failure stores nullptr in *result and returns non-OK. If the file does
  62. // not exist, returns a non-OK status. Implementations should return a
  63. // NotFound status when the file does not exist.
  64. //
  65. // The returned file will only be accessed by one thread at a time.
  66. virtual Status NewSequentialFile(const std::string& fname,
  67. SequentialFile** result) = 0;
  68. // Create an object supporting random-access reads from the file with the
  69. // specified name. On success, stores a pointer to the new file in
  70. // *result and returns OK. On failure stores nullptr in *result and
  71. // returns non-OK. If the file does not exist, returns a non-OK
  72. // status. Implementations should return a NotFound status when the file does
  73. // not exist.
  74. //
  75. // The returned file may be concurrently accessed by multiple threads.
  76. virtual Status NewRandomAccessFile(const std::string& fname,
  77. RandomAccessFile** result) = 0;
  78. // Create an object that writes to a new file with the specified
  79. // name. Deletes any existing file with the same name and creates a
  80. // new file. On success, stores a pointer to the new file in
  81. // *result and returns OK. On failure stores nullptr in *result and
  82. // returns non-OK.
  83. //
  84. // The returned file will only be accessed by one thread at a time.
  85. virtual Status NewWritableFile(const std::string& fname,
  86. WritableFile** result) = 0;
  87. // Create an object that either appends to an existing file, or
  88. // writes to a new file (if the file does not exist to begin with).
  89. // On success, stores a pointer to the new file in *result and
  90. // returns OK. On failure stores nullptr in *result and returns
  91. // non-OK.
  92. //
  93. // The returned file will only be accessed by one thread at a time.
  94. //
  95. // May return an IsNotSupportedError error if this Env does
  96. // not allow appending to an existing file. Users of Env (including
  97. // the leveldb implementation) must be prepared to deal with
  98. // an Env that does not support appending.
  99. virtual Status NewAppendableFile(const std::string& fname,
  100. WritableFile** result);
  101. // Returns true iff the named file exists.
  102. virtual bool FileExists(const std::string& fname) = 0;
  103. // Store in *result the names of the children of the specified directory.
  104. // The names are relative to "dir".
  105. // Original contents of *results are dropped.
  106. virtual Status GetChildren(const std::string& dir,
  107. std::vector<std::string>* result) = 0;
  108. // Delete the named file.
  109. virtual Status DeleteFile(const std::string& fname) = 0;
  110. // Create the specified directory.
  111. virtual Status CreateDir(const std::string& dirname) = 0;
  112. // Delete the specified directory.
  113. virtual Status DeleteDir(const std::string& dirname) = 0;
  114. // Store the size of fname in *file_size.
  115. virtual Status GetFileSize(const std::string& fname, uint64_t* file_size) = 0;
  116. // Rename file src to target.
  117. virtual Status RenameFile(const std::string& src,
  118. const std::string& target) = 0;
  119. // Lock the specified file. Used to prevent concurrent access to
  120. // the same db by multiple processes. On failure, stores nullptr in
  121. // *lock and returns non-OK.
  122. //
  123. // On success, stores a pointer to the object that represents the
  124. // acquired lock in *lock and returns OK. The caller should call
  125. // UnlockFile(*lock) to release the lock. If the process exits,
  126. // the lock will be automatically released.
  127. //
  128. // If somebody else already holds the lock, finishes immediately
  129. // with a failure. I.e., this call does not wait for existing locks
  130. // to go away.
  131. //
  132. // May create the named file if it does not already exist.
  133. virtual Status LockFile(const std::string& fname, FileLock** lock) = 0;
  134. // Release the lock acquired by a previous successful call to LockFile.
  135. // REQUIRES: lock was returned by a successful LockFile() call
  136. // REQUIRES: lock has not already been unlocked.
  137. virtual Status UnlockFile(FileLock* lock) = 0;
  138. // Arrange to run "(*function)(arg)" once in a background thread.
  139. //
  140. // "function" may run in an unspecified thread. Multiple functions
  141. // added to the same Env may run concurrently in different threads.
  142. // I.e., the caller may not assume that background work items are
  143. // serialized.
  144. virtual void Schedule(void (*function)(void* arg), void* arg) = 0;
  145. // Start a new thread, invoking "function(arg)" within the new thread.
  146. // When "function(arg)" returns, the thread will be destroyed.
  147. virtual void StartThread(void (*function)(void* arg), void* arg) = 0;
  148. // *path is set to a temporary directory that can be used for testing. It may
  149. // or may not have just been created. The directory may or may not differ
  150. // between runs of the same process, but subsequent calls will return the
  151. // same directory.
  152. virtual Status GetTestDirectory(std::string* path) = 0;
  153. // Create and return a log file for storing informational messages.
  154. virtual Status NewLogger(const std::string& fname, Logger** result) = 0;
  155. // Returns the number of micro-seconds since some fixed point in time. Only
  156. // useful for computing deltas of time.
  157. virtual uint64_t NowMicros() = 0;
  158. // Sleep/delay the thread for the prescribed number of micro-seconds.
  159. virtual void SleepForMicroseconds(int micros) = 0;
  160. };
  161. // A file abstraction for reading sequentially through a file
  162. class LEVELDB_EXPORT SequentialFile {
  163. public:
  164. SequentialFile() = default;
  165. SequentialFile(const SequentialFile&) = delete;
  166. SequentialFile& operator=(const SequentialFile&) = delete;
  167. virtual ~SequentialFile();
  168. // Read up to "n" bytes from the file. "scratch[0..n-1]" may be
  169. // written by this routine. Sets "*result" to the data that was
  170. // read (including if fewer than "n" bytes were successfully read).
  171. // May set "*result" to point at data in "scratch[0..n-1]", so
  172. // "scratch[0..n-1]" must be live when "*result" is used.
  173. // If an error was encountered, returns a non-OK status.
  174. //
  175. // REQUIRES: External synchronization
  176. virtual Status Read(size_t n, Slice* result, char* scratch) = 0;
  177. // Skip "n" bytes from the file. This is guaranteed to be no
  178. // slower that reading the same data, but may be faster.
  179. //
  180. // If end of file is reached, skipping will stop at the end of the
  181. // file, and Skip will return OK.
  182. //
  183. // REQUIRES: External synchronization
  184. virtual Status Skip(uint64_t n) = 0;
  185. };
  186. // A file abstraction for randomly reading the contents of a file.
  187. class LEVELDB_EXPORT RandomAccessFile {
  188. public:
  189. RandomAccessFile() = default;
  190. RandomAccessFile(const RandomAccessFile&) = delete;
  191. RandomAccessFile& operator=(const RandomAccessFile&) = delete;
  192. virtual ~RandomAccessFile();
  193. // Read up to "n" bytes from the file starting at "offset".
  194. // "scratch[0..n-1]" may be written by this routine. Sets "*result"
  195. // to the data that was read (including if fewer than "n" bytes were
  196. // successfully read). May set "*result" to point at data in
  197. // "scratch[0..n-1]", so "scratch[0..n-1]" must be live when
  198. // "*result" is used. If an error was encountered, returns a non-OK
  199. // status.
  200. //
  201. // Safe for concurrent use by multiple threads.
  202. virtual Status Read(uint64_t offset, size_t n, Slice* result,
  203. char* scratch) const = 0;
  204. };
  205. // A file abstraction for sequential writing. The implementation
  206. // must provide buffering since callers may append small fragments
  207. // at a time to the file.
  208. class LEVELDB_EXPORT WritableFile {
  209. public:
  210. WritableFile() = default;
  211. WritableFile(const WritableFile&) = delete;
  212. WritableFile& operator=(const WritableFile&) = delete;
  213. virtual ~WritableFile();
  214. virtual Status Append(const Slice& data) = 0;
  215. virtual Status Close() = 0;
  216. virtual Status Flush() = 0;
  217. virtual Status Sync() = 0;
  218. };
  219. // An interface for writing log messages.
  220. class LEVELDB_EXPORT Logger {
  221. public:
  222. Logger() = default;
  223. Logger(const Logger&) = delete;
  224. Logger& operator=(const Logger&) = delete;
  225. virtual ~Logger();
  226. // Write an entry to the log file with the specified format.
  227. virtual void Logv(const char* format, va_list ap) = 0;
  228. };
  229. // Identifies a locked file.
  230. class LEVELDB_EXPORT FileLock {
  231. public:
  232. FileLock() = default;
  233. FileLock(const FileLock&) = delete;
  234. FileLock& operator=(const FileLock&) = delete;
  235. virtual ~FileLock();
  236. };
  237. // Log the specified data to *info_log if info_log is non-null.
  238. void Log(Logger* info_log, const char* format, ...)
  239. #if defined(__GNUC__) || defined(__clang__)
  240. __attribute__((__format__(__printf__, 2, 3)))
  241. #endif
  242. ;
  243. // A utility routine: write "data" to the named file.
  244. LEVELDB_EXPORT Status WriteStringToFile(Env* env, const Slice& data,
  245. const std::string& fname);
  246. // A utility routine: read contents of named file into *data
  247. LEVELDB_EXPORT Status ReadFileToString(Env* env, const std::string& fname,
  248. std::string* data);
  249. // An implementation of Env that forwards all calls to another Env.
  250. // May be useful to clients who wish to override just part of the
  251. // functionality of another Env.
  252. class LEVELDB_EXPORT EnvWrapper : public Env {
  253. public:
  254. // Initialize an EnvWrapper that delegates all calls to *t.
  255. explicit EnvWrapper(Env* t) : target_(t) {}
  256. virtual ~EnvWrapper();
  257. // Return the target to which this Env forwards all calls.
  258. Env* target() const { return target_; }
  259. // The following text is boilerplate that forwards all methods to target().
  260. Status NewSequentialFile(const std::string& f, SequentialFile** r) override {
  261. return target_->NewSequentialFile(f, r);
  262. }
  263. Status NewRandomAccessFile(const std::string& f,
  264. RandomAccessFile** r) override {
  265. return target_->NewRandomAccessFile(f, r);
  266. }
  267. Status NewWritableFile(const std::string& f, WritableFile** r) override {
  268. return target_->NewWritableFile(f, r);
  269. }
  270. Status NewAppendableFile(const std::string& f, WritableFile** r) override {
  271. return target_->NewAppendableFile(f, r);
  272. }
  273. bool FileExists(const std::string& f) override {
  274. return target_->FileExists(f);
  275. }
  276. Status GetChildren(const std::string& dir,
  277. std::vector<std::string>* r) override {
  278. return target_->GetChildren(dir, r);
  279. }
  280. Status DeleteFile(const std::string& f) override {
  281. return target_->DeleteFile(f);
  282. }
  283. Status CreateDir(const std::string& d) override {
  284. return target_->CreateDir(d);
  285. }
  286. Status DeleteDir(const std::string& d) override {
  287. return target_->DeleteDir(d);
  288. }
  289. Status GetFileSize(const std::string& f, uint64_t* s) override {
  290. return target_->GetFileSize(f, s);
  291. }
  292. Status RenameFile(const std::string& s, const std::string& t) override {
  293. return target_->RenameFile(s, t);
  294. }
  295. Status LockFile(const std::string& f, FileLock** l) override {
  296. return target_->LockFile(f, l);
  297. }
  298. Status UnlockFile(FileLock* l) override { return target_->UnlockFile(l); }
  299. void Schedule(void (*f)(void*), void* a) override {
  300. return target_->Schedule(f, a);
  301. }
  302. void StartThread(void (*f)(void*), void* a) override {
  303. return target_->StartThread(f, a);
  304. }
  305. Status GetTestDirectory(std::string* path) override {
  306. return target_->GetTestDirectory(path);
  307. }
  308. Status NewLogger(const std::string& fname, Logger** result) override {
  309. return target_->NewLogger(fname, result);
  310. }
  311. uint64_t NowMicros() override { return target_->NowMicros(); }
  312. void SleepForMicroseconds(int micros) override {
  313. target_->SleepForMicroseconds(micros);
  314. }
  315. private:
  316. Env* target_;
  317. };
  318. } // namespace leveldb
  319. // Redefine DeleteFile if necessary.
  320. #if defined(_WIN32) && defined(LEVELDB_DELETEFILE_UNDEFINED)
  321. #if defined(UNICODE)
  322. #define DeleteFile DeleteFileW
  323. #else
  324. #define DeleteFile DeleteFileA
  325. #endif // defined(UNICODE)
  326. #endif // defined(_WIN32) && defined(LEVELDB_DELETEFILE_UNDEFINED)
  327. #endif // STORAGE_LEVELDB_INCLUDE_ENV_H_