"Create a brand new [adjective] file" seems like the description for a
method that will create a new file, but is used for methods that open
existing files for read access.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=211468002

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=209345072

WriteBatchInternal has a method for efficiently concatenating two
WriteBatches. This commit exposes the method to the public API.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=208724311

This CL renames the private struct Iterator::Cleanup ->
Iterator::CleanupNode, to better reflect that it's a linked list node,
and extracts duplicated code from its user in IsEmpty() and Run()
methods.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=199175058

This will result in smaller code generation when Status instances are
passed around.

Benchmarks don't indicate a significant change either way.
CPU:        48 * Intel(R) Xeon(R) CPU E5-2690 v3 @ 2.60GHz
CPUCache:   30720 KB
Keys:       16 bytes each
Values:     100 bytes each (50 bytes after compression)
Entries:    1000000
RawSize:    110.6 MB (estimated)
FileSize:   62.9 MB (estimated)

Baseline:
fillseq      :       3.589 micros/op;   30.8 MB/s
fillsync     :    4165.299 micros/op;    0.0 MB/s (1000 ops)
fillrandom   :       5.864 micros/op;   18.9 MB/s
overwrite    :       7.830 micros/op;   14.1 MB/s
readrandom   :       5.534 micros/op; (1000000 of 1000000 found)
readrandom   :       4.292 micros/op; (1000000 of 1000000 found)
readseq      :       0.312 micros/op;  354.1 MB/s
readreverse  :       0.501 micros/op;  220.8 MB/s
compact      :  886211.000 micros/op;
readrandom   :       3.518 micros/op; (1000000 of 1000000 found)
readseq      :       0.251 micros/op;  441.2 MB/s
readreverse  :       0.456 micros/op;  242.4 MB/s
fill100K     :    1329.723 micros/op;   71.7 MB/s (1000 ops)
crc32c       :       1.976 micros/op; 1976.7 MB/s (4K per op)
snappycomp   :       4.705 micros/op;  830.2 MB/s (output: 55.1%)
snappyuncomp :       0.958 micros/op; 4079.1 MB/s
acquireload  :       0.727 micros/op; (each op is 1000 loads)

New:
fillseq      :       3.129 micros/op;   35.4 MB/s
fillsync     :    2748.099 micros/op;    0.0 MB/s (1000 ops)
fillrandom   :       5.394 micros/op;   20.5 MB/s
overwrite    :       7.253 micros/op;   15.3 MB/s
readrandom   :       5.655 micros/op; (1000000 of 1000000 found)
readrandom   :       4.425 micros/op; (1000000 of 1000000 found)
readseq      :       0.298 micros/op;  371.3 MB/s
readreverse  :       0.508 micros/op;  217.9 MB/s
compact      :  885842.000 micros/op;
readrandom   :       3.545 micros/op; (1000000 of 1000000 found)
readseq      :       0.252 micros/op;  438.2 MB/s
readreverse  :       0.425 micros/op;  260.2 MB/s
fill100K     :    1418.347 micros/op;   67.2 MB/s (1000 ops)
crc32c       :       1.987 micros/op; 1966.0 MB/s (4K per op)
snappycomp   :       4.767 micros/op;  819.4 MB/s (output: 55.1%)
snappyuncomp :       0.916 micros/op; 4264.9 MB/s
acquireload  :       0.665 micros/op; (each op is 1000 loads)

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=194002392

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=192365747

This CL switches the public headers to C++11 default and deleted constructors, and adds override to the relevant leveldb::EnvWrapper methods. This should be a good test for C++11 compiler support.

Once this CL settles, the rest of the codebase can be safely modernized to C++11.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=189873212

Fixes https://github.com/google/leveldb/issues/466

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=189425354

External linkage is the default for function declarations in C++.

This also fixes ClangTidy errors generated by removing the "extern"
keyword as described above.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=188730416

This removes the use of the non-portable headers <sys/types.h> and <unistd.h> in c_test.c.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=188503102

When the max file size option was added in CL 134391640 the C API
was not modified to support this.

This change was contributed by GitHub user @olt and fixes issue #439.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=173466388

Env's that filtered out dot files ("." and "..") would return an
empty vector of children causing DestroyDB to do nothing. This fixes
https://github.com/google/leveldb/issues/215

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=172501335

gcc defaults to exporting all symbols, but other linkers do not. Adding
the LEVELDB_EXPORT macro allows a project to set LEVELDB_SHARED_LIBRARY
when building/linking with leveldb as a shared library.

This is to allow leveldb to be created as a shared library on all
platforms support by Chrome and enables a fix for
https://bugs.chromium.org/p/chromium/issues/detail?id=764810.

This also has the benefit of reducing the shared library size from
418863 to 380367 bytes (64-bit Linux).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171037148

BTRFS reorders rename and write operations, so it is possible that a filesystem crash and recovery results in a situation where the file pointed to by CURRENT does not exist. DB::Open currently reports an I/O error in this case. Reporting database corruption is a better hint to the caller, which can attempt to recover the database or erase it and start over.

This issue is not merely theoretical. It was reported as having showed up in the wild at https://github.com/google/leveldb/issues/195 and at https://crbug.com/738961. Also, asides from the BTRFS case described above, incorrect data in CURRENT seems like a possible corruption case that should be handled gracefully.

The Env API changes here can be considered backwards compatible, because an implementation that returns Status::IOError instead of Status::NotFound will still get the same functionality as before.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=161432630

This can be used to report metrics on LevelDB usage.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=156934930

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148937577

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=134391640

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=129930720

The approximate RAM usage of the database is calculated from the memory
allocated for write buffers and the block cache. This is to give an
estimate of memory usage to leveldb clients.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=104222307

Prune() drops on-memory read cache of the database, so that the client can
relief its memory shortage.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=101335710

All other Status::Code enum values have an Is**() method with the one
exception of InvalidArgument.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=97166441

(Based on a suggestion by cmumford.)

"open" benchmark on my workstation speeds up significantly since we
can now avoid three fdatasync calls and a compaction per open:

  Before: ~80000 microseconds
  After:    ~130 microseconds

Details:

(1) Added Options::reuse_logs (currently defaults to false) to control
new behavior.  The intention is to change the default to true after some
baking.

(2) Added Env::NewAppendableFile() whose default implementation returns
a not-supported error.

(3) VersionSet::Recovery attempts to reuse the MANIFEST from which
it is recovering.

(4) DBImpl recovery attempts to reuse the last log file and memtable.

(5) db_test.cc now tests a new configuration that sets reuse_logs to true.

(6) fault_injection_test also tests a reuse_logs==true config.

(7) Added a new recovery_test.

Changes are:

* Update version number to 1.18
* Replace the basic fprintf call with a call to fwrite in order to
  work around the apparent compiler optimization/rewrite failure that we are
  seeing with the new toolchain/iOS SDKs provided with Xcode6 and iOS8.
* Fix ALL the header guards.
* Createed a README.md with the LevelDB project description.
* A new CONTRIBUTING file.
* Don't implicitly convert uint64_t to size_t or int.  Either preserve it as
  uint64_t, or explicitly cast. This fixes MSVC warnings about possible value
  truncation when compiling this code in Chromium.
* Added a DumpFile() library function that encapsulates the guts of the
  "leveldbutil dump" command. This will allow clients to dump
  data to their log files instead of stdout. It will also allow clients to
  supply their own environment.
* leveldb: Remove unused function 'ConsumeChar'.
* leveldbutil: Remove unused member variables from WriteBatchItemPrinter.
* OpenBSD, NetBSD and DragonflyBSD have _LITTLE_ENDIAN, so define
  PLATFORM_IS_LITTLE_ENDIAN like on FreeBSD. This fixes:
   * issue #143
   * issue #198
   * issue #249
* Switch from <cstdatomic> to <atomic>. The former never made it into the
  standard and doesn't exist in modern gcc versions at all.  The later contains
  everything that leveldb was using from the former.
  This problem was noticed when porting to Portable Native Client where no memory
  barrier is defined.  The fact that <cstdatomic> is missing normally goes
  unnoticed since memory barriers are defined for most architectures.
* Make Hash() treat its input as unsigned.  Before this change LevelDB files
  from platforms with different signedness of char were not compatible. This
  change fixes: issue #243
* Verify checksums of index/meta/filter blocks when paranoid_checks set.
* Invoke all tools for iOS with xcrun. (This was causing problems with the new
  XCode 5.1.1 image on pulse.)
* include <sys/stat.h> only once, and fix the following linter warning:
  "Found C system header after C++ system header"
* When encountering a corrupted table file, return Status::Corruption instead of
  Status::InvalidArgument.
* Support cygwin as build platform, patch is from https://code.google.com/p/leveldb/issues/detail?id=188
* Fix typo, merge patch from https://code.google.com/p/leveldb/issues/detail?id=159
* Fix typos and comments, and address the following two issues:
  * issue #166
  * issue #241
* Add missing db synchronize after "fillseq" in the benchmark.
* Removed unused variable in SeekRandom: value (issue #201)

- Cleanup: delete unused IntSetToString

  It was added in http://cr/19491949 (and was referenced at the time).
  The last reference was removed in http://cr/19507363.

  This fixes warning/error with pre-release crosstoolv18:
    'std::string leveldb::{anonymous}::IntSetToString(const std::set<long unsigned int>&)' defined but not used [-Werror=unused-function]

- Added arm64 and and armv7s to IOS build as suggested on leveldb mailing list.

- Changed local variable type from int to size_t

  This eliminates compiler warning/error and resolves
  https://code.google.com/p/leveldb/issues/detail?id=140

- Make Log::Reader not report a corruption when the last record in a
  log file is truncated.
- Fix issue 224: variable created but not utilized.
- Remove comment that referenced a removed feature.

- switched from mmap based writing to simpler stdio based writing. Has a
  minor impact (0.5 microseconds) on microbenchmarks for asynchronous
  writes. Synchronous writes speed up from 30ms to 10ms on linux/ext4.
  Should be much more reliable on diverse platforms.
- compaction errors now immediately put the database into a read-only
  mode (until it is re-opened). As a downside, a disk going out of
  space and then space being created will require a re-open to recover
  from, whereas previously that would happen automatically. On the
  plus side, many corruption possibilities go away.
- force the DB to enter an error-state so that all future writes fail
  when a synchronous log write succeeds but the sync fails.
- repair now regenerates sstables that exhibit problems
- fix issue 218 - Use native memory barriers on OSX
- fix issue 212 - QNX build is broken
- fix build on iOS with xcode 5
- make tests compile and pass on windows

Fix issues 200, 201

Also,
* Fix link to bigtable paper in docs.
* New sstables will have the file extension .ldb. .sst files will
continue to be recognized.
* When building for iOS, use xcrun to execute the compiler. This may
affect issue 177.

Fix issues 77, 87, 182, 190.

Additionally, fix the bug described in
https://groups.google.com/d/msg/leveldb/yL6h1mAOc20/vLU64RylIdMJ
where a large contiguous keyspace of deleted data was not getting
compacted.

Also fix a bug where options.max_open_files was not getting clamped
properly.

Non-functional changes only:
1) Add Kevin Regan to the AUTHORS file
2) Add standard license header to issues/issue178_test.cc

Fixes issues
161
174
178

As well as the issue reported by edouarda14@gmail.com about
MissingSSTFile unit test failing on windows.

Fixes issues
147 - thanks feniksgordonfreeman
153
156
166

Additionally,
* Remove calls to exit(1).
* Fix unused-variable warnings from clang.
* Fix possible overflow error related to num_restart value >= (2^32/4).
* Add leveldbutil to .gitignore.
* Add better log messages when Write is stalled on a compaction.

https://groups.google.com/d/msg/leveldb/Kc9JxuIUu5A/9P0N9RL4ar8J

Details:
* Fix shared library building.
* Reorganize linking commands so flags like --as-needed can be passed.
* C binding exports version numbers.
* Fix small typos in documention.

Highlights
----------
Mmap at most 1000 files on Posix to improve performance for large databases.
Support for more architectures (thanks to Alexander K.)

Building and porting
--------------------
HP/UX support (issue 126)
AtomicPointer for ia64 (issue 123)
Sparc v9 support (issue 124)
Atomic ops for powerpc
Use -fno-builtin-memcmp only when using g++
Simplify IOS build rules (issue 114)
Use CXXFLAGS instead of CFLAGS when invoking C++ compiler (issue 118)
Fix snappy shared library problem (issue 94)
Fix shared library installation path regression
Endian-ness detection tweak for FreeBSD

Bug fixes
---------
Stop ignoring FLAGS_open_files in db_bench
Make bloom test behavior agnostic to endian-ness

Performance
-----------
Limit number of mmapped files to 1000 to improve perf for large dbs
Do not delay for 1 second on shutdown path (issue 125)

Misc
----
Make InMemoryEnv return a no-op logger
C binding now has a wrapper for free (issue 117)
Add thread-safety annotations
Added an in-process lock table (issue 120)
Make RandomAccessFile and SequentialFile non-copyable

In particular, we add a new FilterPolicy class.  An instance
of this class can be supplied in Options when opening a
database.  If supplied, the instance is used to generate
summaries of keys (e.g., a bloom filter) which are placed in
sstables.  These summaries are consulted by DB::Get() so we
can avoid reading sstable blocks that are guaranteed to not
contain the key we are looking for.

This change provides one implementation of FilterPolicy
based on bloom filters.

Other changes:
- Updated version number to 1.4.
- Some build tweaks.
- C binding for CompactRange.
- A few more benchmarks: deleteseq, deleterandom, readmissing, seekrandom.
- Minor .gitignore update.

- Replace raw slice comparison with a call to user comparator.
  Added test for custom comparators.

- Fix end of namespace comments.

- Fixed bug in picking inputs for a level-0 compaction.

  When finding overlapping files, the covered range may expand
  as files are added to the input set.  We now correctly expand
  the range when this happens instead of continuing to use the
  old range.  For example, suppose L0 contains files with the
  following ranges:

      F1: a .. d
      F2:    c .. g
      F3:       f .. j

  and the initial compaction target is F3.  We used to search
  for range f..j which yielded {F2,F3}.  However we now expand
  the range as soon as another file is added.  In this case,
  when F2 is added, we expand the range to c..j and restart the
  search.  That picks up file F1 as well.

  This change fixes a bug related to deleted keys showing up
  incorrectly after a compaction as described in Issue 44.

(Sync with upstream @25072954)

- Added DB::CompactRange() method.

  Changed manual compaction code so it breaks up compactions of
  big ranges into smaller compactions.

  Changed the code that pushes the output of memtable compactions
  to higher levels to obey the grandparent constraint: i.e., we
  must never have a single file in level L that overlaps too
  much data in level L+1 (to avoid very expensive L-1 compactions).

  Added code to pretty-print internal keys.

- Fixed bug where we would not detect overlap with files in
  level-0 because we were incorrectly using binary search
  on an array of files with overlapping ranges.

  Added "leveldb.sstables" property that can be used to dump
  all of the sstables and ranges that make up the db state.

- Removing post_write_snapshot support.  Email to leveldb mailing
  list brought up no users, just confusion from one person about
  what it meant.

- Fixing static_cast char to unsigned on BIG_ENDIAN platforms.

  Fixes	Issue 35 and Issue 36.

- Comment clarification to address leveldb Issue 37.

- Change license in posix_logger.h to match other files.

- A build problem where uint32 was used instead of uint32_t.

Sync with upstream @24408625

Fix GCC -Wshadow warnings in LevelDB's public header files,
reported by Dustin.

Add in-memory Env implementation (helpers/memenv/*).
This enables users to create LevelDB databases in-memory.

Initialize ShardedLRUCache::last_id_ to zero.
This fixes a Valgrind warning.

(Also delete port/sha1_* which were removed upstream some time ago.)

- Added a C binding for LevelDB.
  May be useful as a stable ABI that can be used by 
  programs that keep leveldb in a shared library, 
  or for JNI API.

- Replaced SQLite's readseq benchmark to a more efficient version. 
  SQLite readseq speeds increased by about a factor of 2x 
  from the previous version. Also updated benchmark page to
  reflect readseq speed up.



git-svn-id: https://leveldb.googlecode.com/svn/trunk@46 62dab493-f737-651d-591e-8d6aee1b9529

- Removed one copy of an uncompressed block contents changing
  the signature of Snappy_Uncompress() so it uncompresses into a
  flat array instead of a std::string.
        
  Speeds up readrandom ~10%.

- Instead of a combination of Env/WritableFile, we now have a
  Logger interface that can be easily overridden applications
  that want to supply their own logging.

- Separated out the gcc and Sun Studio parts of atomic_pointer.h
  so we can use 'asm', 'volatile' keywords for Sun Studio.




git-svn-id: https://leveldb.googlecode.com/svn/trunk@39 62dab493-f737-651d-591e-8d6aee1b9529

- LevelDB patch for Sun Studio
  Based on a patch submitted by Theo Schlossnagle - thanks!
  This fixes Issue 17.

- Fix a couple of test related memory leaks.



git-svn-id: https://leveldb.googlecode.com/svn/trunk@38 62dab493-f737-651d-591e-8d6aee1b9529

Slight tweak to the no-overlap optimization: only push to
level 2 to reduce the amount of wasted space when the same
small key range is being repeatedly overwritten.

Fix for Issue 18: Avoid failure on Windows by avoiding
deletion of lock file until the end of DestroyDB().

Fix for Issue 19: Disregard sequence numbers when checking for 
overlap in sstable ranges. This fixes issue 19: when writing 
the same key over and over again, we would generate a sequence 
of sstables that were never merged together since their sequence
numbers were disjoint.

Don't ignore map/unmap error checks.

Miscellaneous fixes for small problems Sanjay found while diagnosing
issue/9 and issue/16 (corruption_testr failures).
- log::Reader reports the record type when it finds an unexpected type.
- log::Reader no longer reports an error when it encounters an expected
  zero record regardless of the setting of the "checksum" flag.
- Added a missing forward declaration.
- Documented a side-effects of larger write buffer sizes
  (longer recovery time).



git-svn-id: https://leveldb.googlecode.com/svn/trunk@37 62dab493-f737-651d-591e-8d6aee1b9529

Check the NEWS file for details of what changed.

git-svn-id: https://leveldb.googlecode.com/svn/trunk@28 62dab493-f737-651d-591e-8d6aee1b9529

* env_chromium.cc should not export symbols.
* Fix MSVC warnings.
* Removed large value support.
* Fix broken reference to documentation file

git-svn-id: https://leveldb.googlecode.com/svn/trunk@24 62dab493-f737-651d-591e-8d6aee1b9529

git-svn-id: https://leveldb.googlecode.com/svn/trunk@23 62dab493-f737-651d-591e-8d6aee1b9529