Env::DeleteFile.

PiperOrigin-RevId: 283607548

The SQLite-specific schema feature is documented at
https://www.sqlite.org/withoutrowid.html and
https://www.sqlite.org/rowidtable.html.

By default, SQLite stores each table in a B-tree keyed by an integer,
called the ROWID. Any index, including the PRIMARY KEY index, is a
separate B-tree mapping index keys to ROWIDs. Tables without ROWIDs are
stored in a B-tree keyed by the primary key. Additional indexes (the
PRIMARY KEY index is implicitly built into the table) are stored as
B-trees mapping index keys to row primary keys.

This CL introduces a boolean --use-rowids flag to db_bench_sqlite. When
the flag is false (default), the schema of the test table includes
WITHOUT ROWID. The test table uses a primary key, so adding WITHOUT
ROWID to the schema reduces the number of B-trees used by the benchmark
from 2 to 1. This brings SQLite's disk usage closer to LevelDB.

When WITHOUT ROWID is used, SQLite fares better (than today) on
benchmarks with small (16-byte) keys, and worse on benchmarks with large
(100kb) keys.

Baseline results:
fillseq      :      21.310 micros/op;    5.2 MB/s
fillseqsync  :     146.377 micros/op;    0.8 MB/s (10000 ops)
fillseqbatch :       2.065 micros/op;   53.6 MB/s
fillrandom   :      34.767 micros/op;    3.2 MB/s
fillrandsync :     159.943 micros/op;    0.7 MB/s (10000 ops)
fillrandbatch :      15.055 micros/op;    7.3 MB/s
overwrite    :      43.660 micros/op;    2.5 MB/s
overwritebatch :      27.691 micros/op;    4.0 MB/s
readrandom   :      12.725 micros/op;
readseq      :       2.602 micros/op;   36.7 MB/s
fillrand100K :     606.333 micros/op;  157.3 MB/s (1000 ops)
fillseq100K  :     657.457 micros/op;  145.1 MB/s (1000 ops)
readseq      :      46.523 micros/op; 2049.9 MB/s
readrand100K :      54.943 micros/op;

Results after this CL:
fillseq      :      16.231 micros/op;    6.8 MB/s
fillseqsync  :     147.460 micros/op;    0.8 MB/s (10000 ops)
fillseqbatch :       2.294 micros/op;   48.2 MB/s
fillrandom   :      27.871 micros/op;    4.0 MB/s
fillrandsync :     141.979 micros/op;    0.8 MB/s (10000 ops)
fillrandbatch :      16.087 micros/op;    6.9 MB/s
overwrite    :      26.829 micros/op;    4.1 MB/s
overwritebatch :      19.014 micros/op;    5.8 MB/s
readrandom   :      11.657 micros/op;
readseq      :       0.155 micros/op;  615.0 MB/s
fillrand100K :     816.812 micros/op;  116.8 MB/s (1000 ops)
fillseq100K  :     754.689 micros/op;  126.4 MB/s (1000 ops)
readseq      :      47.112 micros/op; 2024.3 MB/s
readrand100K :     287.679 micros/op;

Results after this CL, with --use-rowids=1
fillseq      :      20.655 micros/op;    5.4 MB/s
fillseqsync  :     146.408 micros/op;    0.8 MB/s (10000 ops)
fillseqbatch :       2.045 micros/op;   54.1 MB/s
fillrandom   :      34.080 micros/op;    3.2 MB/s
fillrandsync :     154.582 micros/op;    0.7 MB/s (10000 ops)
fillrandbatch :      14.404 micros/op;    7.7 MB/s
overwrite    :      42.928 micros/op;    2.6 MB/s
overwritebatch :      27.829 micros/op;    4.0 MB/s
readrandom   :      12.835 micros/op;
readseq      :       2.483 micros/op;   38.4 MB/s
fillrand100K :     603.265 micros/op;  158.1 MB/s (1000 ops)
fillseq100K  :     662.473 micros/op;  144.0 MB/s (1000 ops)
readseq      :      45.478 micros/op; 2097.0 MB/s
readrand100K :      54.439 micros/op;
PiperOrigin-RevId: 283407101

Fixup for adding the third_party/googletest submodule.

PiperOrigin-RevId: 282373286

PiperOrigin-RevId: 281815695

PiperOrigin-RevId: 279785825

Inspired by https://github.com/google/snappy/pull/85

PiperOrigin-RevId: 279649967

PiperOrigin-RevId: 279238007

Inspired by https://github.com/google/crc32c/pull/32

PiperOrigin-RevId: 278718726

Using CMAKE_INSTALL_INCLUDEDIR before including GNUINstallDirs results
in a broken installation when CMAKE_INSTALL_PREFIX is a non-standard
directory.

Inspired from https://github.com/google/crc32c/pull/39

PiperOrigin-RevId: 278427974

PiperOrigin-RevId: 278300591

The local variable `updates` in DBImpl::Write was hiding the
`updates` parameter. Renamed to avoid this conflict.

PiperOrigin-RevId: 277089971

Added unreached return at the end of Version::Get::State::Match
to stop this _incorrect_ warning:

    version_set.cc:376:5: warning: control reaches end of
    non-void function [-Wreturn-type]

This warning was being emitted when building with clang 6.0.1-10
and also emitted by lgtm.com when statically analyzing leveldb even
though all SaverState enumeration values were handled.

PiperOrigin-RevId: 272455474

Previously used check_cxx_source_compiles to attempt a
build to determine support for clang thread safety checks.

This change is to support static analysis of the leveldb source by
lgtm.com (using Semmle). It failed to build with the following error:

```
[2019-07-04 22:29:58] [build] c++: error: unrecognized command line option ‘-Wthread-safety’; did you mean ‘-fthread-jumps’?
[2019-07-04 22:30:02] [build] make[2]: *** [CMakeFiles/leveldb.dir/build.make:66: CMakeFiles/leveldb.dir/db/builder.cc.o] Error 1
```

PiperOrigin-RevId: 272275528

* Use Ubuntu 18.04 and LLVM 9 on Travis.
* Fix bash conditionals: [ a == b ] should be [ a = b ].

PiperOrigin-RevId: 271898719

PiperOrigin-RevId: 268265314

PiperOrigin-RevId: 266001777

A recent change (4cb80b7ddc) to DBImpl::DeleteObsoleteFiles
unlocked DBImpl::mutex_ while deleting files to allow for
greater concurrency. This change improves on the prior in
a few areas:

1. The table is evicted from the table cache before unlocking
   the mutex. This should only improve performance.
2. This implementation is slightly simpler, but at the cost of
   a bit more memory usage.
3. A comment adding more detail as to why the mutex is being
   unlocked and why it is safe to do so.

PiperOrigin-RevId: 253111645

The documentation (README.md and index.md) referred to the
public headers using an incorrect path - fixing.

PiperOrigin-RevId: 252922925

This change was submitted in https://github.com/google/leveldb/pull/575
by @prajwalchalla.

This fixes issue #523.

PiperOrigin-RevId: 252912613

Fixes #695.

PiperOrigin-RevId: 252895299

PiperOrigin-RevId: 252842234

PiperOrigin-RevId: 250702492

PiperOrigin-RevId: 250383036

PiperOrigin-RevId: 250309603

PiperOrigin-RevId: 249899128

Fixes issue #684.

PiperOrigin-RevId: 249531001

mutex_ was already acquired before accessing DBImpl::versions_ in all
but one place: DBImpl::GetApproximateSizes. This change requires mutex_
to be held before accessing versions_.

PiperOrigin-RevId: 248390814

Converted `for (;<condition>;)` to `while (<condition>)`.

PiperOrigin-RevId: 247950510

PiperOrigin-RevId: 247491163

PiperOrigin-RevId: 247483339

PiperOrigin-RevId: 247424040

Switching from snprintf to std::ostringstream eliminates
cast warning for (unsigned long long).

PiperOrigin-RevId: 247326681

PiperOrigin-RevId: 247261470

The previous attempt of having EnvPosix use O_CLOEXEC (close-on-exec()) when opening file descriptors added tests that relied on procfs, which is Linux-specific. These tests failed on macOS. Unfortunately, the test failures were not caught due to a (since fixed) error in our Travis CI configuration.

This CL re-structures the tests to only rely on POSIX features. Since there is no POSIX-compliant way to get a file name/path out of a file descriptor, this CL breaks up the O_CLOEXEC test into multiple tests, where each Env method that creates an FD gets its own test. This is intended to make it easier to find and fix errors in Env implementations.

This CL also fixes the implementation of NewLogger() to use O_CLOEXEC on macOS. The current implementation passes "we" to fopen(), but the macOS standard C library does not implement the "e" flag yet.

PiperOrigin-RevId: 247088953

Two minor corrections to correct the 900f7d37eb commit
to conform to the Google C++ style guide.

PiperOrigin-RevId: 246907647

PiperOrigin-RevId: 246903086

1) Inline EncodeFixed{32,64}(). They emit single machine instructions on 64-bit processors.
2) Remove size narrowing compiler warnings from DecodeFixed{32,64}().
3) Add comments explaining the current state of optimizations in compilers we care about.
4) Change C-style includes, like <stdint.h>, to C++ style, like <cstdint>.
5) memcpy -> std::memcpy.

The optimization comments are based on https://godbolt.org/z/RdIqS1. The missed optimization opportunities in clang have been reported as https://bugs.llvm.org/show_bug.cgi?id=41761

The change does not have significant impact on benchmarks. Results below.

LevelDB:    version 1.22
Date:       Mon May  6 10:42:18 2019
CPU:        72 * Intel(R) Xeon(R) Gold 6154 CPU @ 3.00GHz
CPUCache:   25344 KB
Keys:       16 bytes each
Values:     100 bytes each (50 bytes after compression)
Entries:    1000000
RawSize:    110.6 MB (estimated)
FileSize:   62.9 MB (estimated)

With change
------------------------------------------------
fillseq      :       2.327 micros/op;   47.5 MB/s
fillsync     :    4185.526 micros/op;    0.0 MB/s (1000 ops)
fillrandom   :       3.662 micros/op;   30.2 MB/s
overwrite    :       4.261 micros/op;   26.0 MB/s
readrandom   :       4.239 micros/op; (1000000 of 1000000 found)
readrandom   :       3.649 micros/op; (1000000 of 1000000 found)
readseq      :       0.174 micros/op;  636.7 MB/s
readreverse  :       0.271 micros/op;  408.7 MB/s
compact      :  570495.000 micros/op;
readrandom   :       2.735 micros/op; (1000000 of 1000000 found)
readseq      :       0.118 micros/op;  937.3 MB/s
readreverse  :       0.190 micros/op;  583.7 MB/s
fill100K     :     860.164 micros/op;  110.9 MB/s (1000 ops)
crc32c       :       1.131 micros/op; 3455.2 MB/s (4K per op)
snappycomp   :       3.034 micros/op; 1287.5 MB/s (output: 55.1%)
snappyuncomp :       0.544 micros/op; 7176.0 MB/s

Baseline
------------------------------------------------
fillseq      :       2.365 micros/op;   46.8 MB/s
fillsync     :    4240.165 micros/op;    0.0 MB/s (1000 ops)
fillrandom   :       3.244 micros/op;   34.1 MB/s
overwrite    :       4.153 micros/op;   26.6 MB/s
readrandom   :       4.698 micros/op; (1000000 of 1000000 found)
readrandom   :       4.065 micros/op; (1000000 of 1000000 found)
readseq      :       0.192 micros/op;  576.3 MB/s
readreverse  :       0.286 micros/op;  386.7 MB/s
compact      :  635979.000 micros/op;
readrandom   :       3.264 micros/op; (1000000 of 1000000 found)
readseq      :       0.169 micros/op;  652.8 MB/s
readreverse  :       0.213 micros/op;  519.5 MB/s
fill100K     :    1055.367 micros/op;   90.4 MB/s (1000 ops)
crc32c       :       1.353 micros/op; 2887.3 MB/s (4K per op)
snappycomp   :       3.036 micros/op; 1286.7 MB/s (output: 55.1%)
snappyuncomp :       0.540 micros/op; 7238.6 MB/s
PiperOrigin-RevId: 246856811

Avoids a use before initialization error. This fixes issue #676.

PiperOrigin-RevId: 246855204