monero-project/monero
monero-project/monero
1
0
Fork 0
mirror of https://github.com/monero-project/monero.git synced 2024-12-23 03:59:33 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

update lmdb64

Browse source
This commit is contained in:
Riccardo Spagni 2015-04-14 21:07:39 +02:00
parent e325c5f04d
commit 41f0a8fe4d
No known key found for this signature in database
GPG key ID: 55432DF31CCD4FCD
22 changed files with 201 additions and 365 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

15
external/db_drivers/liblmdb64/CHANGES vendored
View file

@ -1,5 +1,20 @@
LMDB 0.9 Change Log LMDB 0.9 Change Log
LMDB 0.9.15 Release Engineering
Fix txn init (ITS#7961,#7987)
Fix MDB_PREV_DUP (ITS#7955,#7671)
Fix compact of empty env (ITS#7956)
Fix mdb_load with large values (ITS#8066)
Added workaround for fdatasync bug in ext3fs
Build
Don't use -fPIC for static lib
Update .gitignore (ITS#7952,#7953)
Cleanup for "make test" (ITS#7841)
Misc. Android/Windows cleanup
Documentation
Fix MDB_APPEND doc
Clarify mdb_dbi_open doc
LMDB 0.9.14 Release (2014/09/20) LMDB 0.9.14 Release (2014/09/20)
Fix to support 64K page size (ITS#7713) Fix to support 64K page size (ITS#7713)
Fix to persist decreased as well as increased mapsizes (ITS#7789) Fix to persist decreased as well as increased mapsizes (ITS#7789)

2
external/db_drivers/liblmdb64/COPYRIGHT vendored
View file

@ -1,4 +1,4 @@
Copyright 2011-2014 Howard Chu, Symas Corp. Copyright 2011-2015 Howard Chu, Symas Corp.
All rights reserved. All rights reserved.
Redistribution and use in source and binary forms, with or without Redistribution and use in source and binary forms, with or without

31
external/db_drivers/liblmdb64/lmdb.h vendored
View file

@ -49,15 +49,11 @@
* stale locks can block further operation. * stale locks can block further operation.
* *
* Fix: Check for stale readers periodically, using the * Fix: Check for stale readers periodically, using the
* #mdb_reader_check function or the \ref mdb_stat_1 "mdb_stat" tool. * #mdb_reader_check function or the \ref mdb_stat_1 "mdb_stat" tool. Or just
* Stale writers will be cleared automatically on most systems: * make all programs using the database close it; the lockfile
* - Windows - automatic * is always reset on first open of the environment.
* - BSD, systems using SysV semaphores - automatic
* - Linux, systems using POSIX mutexes with Robust option - automatic
* Otherwise just make all programs using the database close it;
* the lockfile is always reset on first open of the environment.
* *
* - On BSD systems or others configured with MDB_USE_SYSV_SEM, * - On BSD systems or others configured with MDB_USE_POSIX_SEM,
* startup can fail due to semaphores owned by another userid. * startup can fail due to semaphores owned by another userid.
* *
* Fix: Open and close the database as the user which owns the * Fix: Open and close the database as the user which owns the
@ -110,9 +106,6 @@
* for stale readers is performed or the lockfile is reset, * for stale readers is performed or the lockfile is reset,
* since the process may not remove it from the lockfile. * since the process may not remove it from the lockfile.
* *
* This does not apply to write transactions if the system clears
* stale writers, see above.
*
* - If you do that anyway, do a periodic check for stale readers. Or * - If you do that anyway, do a periodic check for stale readers. Or
* close the environment once in a while, so the lockfile can get reset. * close the environment once in a while, so the lockfile can get reset.
* *
@ -126,7 +119,7 @@
* *
* @author Howard Chu, Symas Corporation. * @author Howard Chu, Symas Corporation.
* *
* @copyright Copyright 2011-2014 Howard Chu, Symas Corp. All rights reserved. * @copyright Copyright 2011-2015 Howard Chu, Symas Corp. All rights reserved.
* *
* Redistribution and use in source and binary forms, with or without * Redistribution and use in source and binary forms, with or without
* modification, are permitted only as authorized by the OpenLDAP * modification, are permitted only as authorized by the OpenLDAP
@ -398,7 +391,7 @@ typedef enum MDB_cursor_op {
#define MDB_PAGE_NOTFOUND (-30797) #define MDB_PAGE_NOTFOUND (-30797)
/** Located page was wrong type */ /** Located page was wrong type */
#define MDB_CORRUPTED (-30796) #define MDB_CORRUPTED (-30796)
/** Update of meta page failed or environment had fatal error */ /** Update of meta page failed, probably I/O error */
#define MDB_PANIC (-30795) #define MDB_PANIC (-30795)
/** Environment version mismatch */ /** Environment version mismatch */
#define MDB_VERSION_MISMATCH (-30794) #define MDB_VERSION_MISMATCH (-30794)
@ -736,7 +729,6 @@ void mdb_env_close(MDB_env *env);
* This may be used to set some flags in addition to those from * This may be used to set some flags in addition to those from
* #mdb_env_open(), or to unset these flags. If several threads * #mdb_env_open(), or to unset these flags. If several threads
* change the flags at the same time, the result is undefined. * change the flags at the same time, the result is undefined.
* Most flags cannot be changed after #mdb_env_open().
* @param[in] env An environment handle returned by #mdb_env_create() * @param[in] env An environment handle returned by #mdb_env_create()
* @param[in] flags The flags to change, bitwise OR'ed together * @param[in] flags The flags to change, bitwise OR'ed together
* @param[in] onoff A non-zero value sets the flags, zero clears them. * @param[in] onoff A non-zero value sets the flags, zero clears them.
@ -954,17 +946,6 @@ int mdb_txn_begin(MDB_env *env, MDB_txn *parent, unsigned int flags, MDB_txn **
*/ */
MDB_env *mdb_txn_env(MDB_txn *txn); MDB_env *mdb_txn_env(MDB_txn *txn);
/** @brief Return the transaction's ID.
*
* This returns the identifier associated with this transaction. For a
* read-only transaction, this corresponds to the snapshot being read;
* concurrent readers will frequently have the same transaction ID.
*
* @param[in] txn A transaction handle returned by #mdb_txn_begin()
* @return A transaction ID, valid if input is an active transaction.
*/
size_t mdb_txn_id(MDB_txn *txn);
/** @brief Commit all the operations of a transaction into the database. /** @brief Commit all the operations of a transaction into the database.
* *
* The transaction handle is freed. It and its cursors must not be used * The transaction handle is freed. It and its cursors must not be used

466
external/db_drivers/liblmdb64/mdb.c vendored
View file

@ -5,7 +5,7 @@
* BerkeleyDB API, but much simplified. * BerkeleyDB API, but much simplified.
*/ */
/* /*
* Copyright 2011-2014 Howard Chu, Symas Corp. * Copyright 2011-2015 Howard Chu, Symas Corp.
* All rights reserved. * All rights reserved.
* *
* Redistribution and use in source and binary forms, with or without * Redistribution and use in source and binary forms, with or without
@ -110,7 +110,7 @@ extern int cacheflush(char *addr, int nbytes, int cache);
#endif #endif
#if defined(__APPLE__) || defined (BSD) #if defined(__APPLE__) || defined (BSD)
# define MDB_USE_SYSV_SEM 1 # define MDB_USE_POSIX_SEM 1
# define MDB_FDATASYNC fsync # define MDB_FDATASYNC fsync
#elif defined(ANDROID) #elif defined(ANDROID)
# define MDB_FDATASYNC fsync # define MDB_FDATASYNC fsync
@ -118,18 +118,11 @@ extern int cacheflush(char *addr, int nbytes, int cache);
#ifndef _WIN32 #ifndef _WIN32
#include <pthread.h> #include <pthread.h>
#ifdef MDB_USE_SYSV_SEM #ifdef MDB_USE_POSIX_SEM
#include <sys/ipc.h> # define MDB_USE_HASH 1
#include <sys/sem.h> #include <semaphore.h>
#ifdef _SEM_SEMUN_UNDEFINED #endif
union semun { #endif
int val;
struct semid_ds *buf;
unsigned short *array;
};
#endif /* _SEM_SEMUN_UNDEFINED */
#endif /* MDB_USE_SYSV_SEM */
#endif /* !_WIN32 */
#ifdef USE_VALGRIND #ifdef USE_VALGRIND
#include <valgrind/memcheck.h> #include <valgrind/memcheck.h>
@ -206,10 +199,6 @@ union semun {
/** Features under development */ /** Features under development */
#ifndef MDB_DEVEL #ifndef MDB_DEVEL
#define MDB_DEVEL 0 #define MDB_DEVEL 0
#endif
#if defined(_WIN32) || defined(MDB_USE_SYSV_SEM) || defined(EOWNERDEAD)
#define MDB_ROBUST_SUPPORTED 1
#endif #endif
/** Wrapper around __func__, which is a C99 feature */ /** Wrapper around __func__, which is a C99 feature */
@ -222,16 +211,6 @@ union semun {
# define mdb_func_ "<mdb_unknown>" # define mdb_func_ "<mdb_unknown>"
#endif #endif
/* Internal error codes, not exposed outside liblmdb */
#define MDB_NO_ROOT (MDB_LAST_ERRCODE + 10)
#ifdef _WIN32
#define MDB_OWNERDEAD ((int) WAIT_ABANDONED)
#elif defined MDB_USE_SYSV_SEM
#define MDB_OWNERDEAD (MDB_LAST_ERRCODE + 11)
#else
#define MDB_OWNERDEAD EOWNERDEAD
#endif
#ifdef _WIN32 #ifdef _WIN32
#define MDB_USE_HASH 1 #define MDB_USE_HASH 1
#define MDB_PIDLOCK 0 #define MDB_PIDLOCK 0
@ -239,7 +218,6 @@ union semun {
#define pthread_t HANDLE #define pthread_t HANDLE
#define pthread_mutex_t HANDLE #define pthread_mutex_t HANDLE
#define pthread_cond_t HANDLE #define pthread_cond_t HANDLE
typedef HANDLE mdb_mutex_t;
#define pthread_key_t DWORD #define pthread_key_t DWORD
#define pthread_self() GetCurrentThreadId() #define pthread_self() GetCurrentThreadId()
#define pthread_key_create(x,y) \ #define pthread_key_create(x,y) \
@ -253,10 +231,10 @@ typedef HANDLE mdb_mutex_t;
#define pthread_cond_wait(cond,mutex) do{SignalObjectAndWait(*mutex, *cond, INFINITE, FALSE); WaitForSingleObject(*mutex, INFINITE);}while(0) #define pthread_cond_wait(cond,mutex) do{SignalObjectAndWait(*mutex, *cond, INFINITE, FALSE); WaitForSingleObject(*mutex, INFINITE);}while(0)
#define THREAD_CREATE(thr,start,arg) thr=CreateThread(NULL,0,start,arg,0,NULL) #define THREAD_CREATE(thr,start,arg) thr=CreateThread(NULL,0,start,arg,0,NULL)
#define THREAD_FINISH(thr) WaitForSingleObject(thr, INFINITE) #define THREAD_FINISH(thr) WaitForSingleObject(thr, INFINITE)
#define MDB_MUTEX(env, rw) ((env)->me_##rw##mutex) #define LOCK_MUTEX_R(env) pthread_mutex_lock(&(env)->me_rmutex)
#define LOCK_MUTEX0(mutex) WaitForSingleObject(mutex, INFINITE) #define UNLOCK_MUTEX_R(env) pthread_mutex_unlock(&(env)->me_rmutex)
#define UNLOCK_MUTEX(mutex) ReleaseMutex(mutex) #define LOCK_MUTEX_W(env) pthread_mutex_lock(&(env)->me_wmutex)
#define mdb_mutex_consistent(mutex) 0 #define UNLOCK_MUTEX_W(env) pthread_mutex_unlock(&(env)->me_wmutex)
#define getpid() GetCurrentProcessId() #define getpid() GetCurrentProcessId()
#define MDB_FDATASYNC(fd) (!FlushFileBuffers(fd)) #define MDB_FDATASYNC(fd) (!FlushFileBuffers(fd))
#define MDB_MSYNC(addr,len,flags) (!FlushViewOfFile(addr,len)) #define MDB_MSYNC(addr,len,flags) (!FlushViewOfFile(addr,len))
@ -279,59 +257,38 @@ typedef HANDLE mdb_mutex_t;
/** For MDB_LOCK_FORMAT: True if readers take a pid lock in the lockfile */ /** For MDB_LOCK_FORMAT: True if readers take a pid lock in the lockfile */
#define MDB_PIDLOCK 1 #define MDB_PIDLOCK 1
#ifdef MDB_USE_SYSV_SEM #ifdef MDB_USE_POSIX_SEM
typedef struct mdb_mutex { #define LOCK_MUTEX_R(env) mdb_sem_wait((env)->me_rmutex)
int semid; #define UNLOCK_MUTEX_R(env) sem_post((env)->me_rmutex)
int semnum; #define LOCK_MUTEX_W(env) mdb_sem_wait((env)->me_wmutex)
int *locked; #define UNLOCK_MUTEX_W(env) sem_post((env)->me_wmutex)
} mdb_mutex_t;
#define MDB_MUTEX(env, rw) (&(env)->me_##rw##mutex)
#define LOCK_MUTEX0(mutex) mdb_sem_wait(mutex)
#define UNLOCK_MUTEX(mutex) do { \
struct sembuf sb = { 0, 1, SEM_UNDO }; \
sb.sem_num = (mutex)->semnum; \
*(mutex)->locked = 0; \
semop((mutex)->semid, &sb, 1); \
} while(0)
static int static int
mdb_sem_wait(mdb_mutex_t *sem) mdb_sem_wait(sem_t *sem)
{ {
int rc, *locked = sem->locked; int rc;
struct sembuf sb = { 0, -1, SEM_UNDO }; while ((rc = sem_wait(sem)) && (rc = errno) == EINTR) ;
sb.sem_num = sem->semnum;
do {
if (!semop(sem->semid, &sb, 1)) {
rc = *locked ? MDB_OWNERDEAD : MDB_SUCCESS;
*locked = 1;
break;
}
} while ((rc = errno) == EINTR);
return rc; return rc;
} }
#define mdb_mutex_consistent(mutex) 0
#else #else
/** Pointer/HANDLE type of shared mutex/semaphore. /** Lock the reader mutex.
*/ */
typedef pthread_mutex_t mdb_mutex_t; #define LOCK_MUTEX_R(env) pthread_mutex_lock(&(env)->me_txns->mti_mutex)
/** Mutex for the reader table (rw = r) or write transaction (rw = w). /** Unlock the reader mutex.
*/ */
#define MDB_MUTEX(env, rw) (&(env)->me_txns->mti_##rw##mutex) #define UNLOCK_MUTEX_R(env) pthread_mutex_unlock(&(env)->me_txns->mti_mutex)
/** Lock the reader or writer mutex.
* Returns 0 or a code to give #mdb_mutex_failed(), as in #LOCK_MUTEX(). /** Lock the writer mutex.
* Only a single write transaction is allowed at a time. Other writers
* will block waiting for this mutex.
*/ */
#define LOCK_MUTEX0(mutex) pthread_mutex_lock(mutex) #define LOCK_MUTEX_W(env) pthread_mutex_lock(&(env)->me_txns->mti_wmutex)
/** Unlock the reader or writer mutex. /** Unlock the writer mutex.
*/ */
#define UNLOCK_MUTEX(mutex) pthread_mutex_unlock(mutex) #define UNLOCK_MUTEX_W(env) pthread_mutex_unlock(&(env)->me_txns->mti_wmutex)
/** Mark mutex-protected data as repaired, after death of previous owner. #endif /* MDB_USE_POSIX_SEM */
*/
#define mdb_mutex_consistent(mutex) pthread_mutex_consistent(mutex)
#endif /* MDB_USE_SYSV_SEM */
/** Get the error code for the last failed system function. /** Get the error code for the last failed system function.
*/ */
@ -356,35 +313,14 @@ typedef pthread_mutex_t mdb_mutex_t;
#define GET_PAGESIZE(x) ((x) = sysconf(_SC_PAGE_SIZE)) #define GET_PAGESIZE(x) ((x) = sysconf(_SC_PAGE_SIZE))
#endif #endif
#if defined(_WIN32) #if defined(_WIN32) || defined(MDB_USE_POSIX_SEM)
#define MNAME_LEN 32 #define MNAME_LEN 32
#elif defined(MDB_USE_SYSV_SEM)
#define MNAME_LEN (sizeof(int))
#else #else
#define MNAME_LEN (sizeof(pthread_mutex_t)) #define MNAME_LEN (sizeof(pthread_mutex_t))
#endif #endif
#ifdef MDB_USE_SYSV_SEM
#define SYSV_SEM_FLAG 1 /**< SysV sems in lockfile format */
#else
#define SYSV_SEM_FLAG 0
#endif
/** @} */ /** @} */
#ifdef MDB_ROBUST_SUPPORTED
/** Lock mutex, handle any error, set rc = result.
* Return 0 on success, nonzero (not rc) on error.
*/
#define LOCK_MUTEX(rc, env, mutex) \
(((rc) = LOCK_MUTEX0(mutex)) && \
((rc) = mdb_mutex_failed(env, mutex, rc)))
static int mdb_mutex_failed(MDB_env *env, mdb_mutex_t *mutex, int rc);
#else
#define LOCK_MUTEX(rc, env, mutex) ((rc) = LOCK_MUTEX0(mutex))
#define mdb_mutex_failed(env, mutex, rc) (rc)
#endif
#ifndef _WIN32 #ifndef _WIN32
/** A flag for opening a file and requesting synchronous data writes. /** A flag for opening a file and requesting synchronous data writes.
* This is only used when writing a meta page. It's not strictly needed; * This is only used when writing a meta page. It's not strictly needed;
@ -508,7 +444,7 @@ static txnid_t mdb_debug_start;
/** The version number for a database's datafile format. */ /** The version number for a database's datafile format. */
#define MDB_DATA_VERSION ((MDB_DEVEL) ? 999 : 1) #define MDB_DATA_VERSION ((MDB_DEVEL) ? 999 : 1)
/** The version number for a database's lockfile format. */ /** The version number for a database's lockfile format. */
#define MDB_LOCK_VERSION ((MDB_DEVEL) ? 999 : 1) #define MDB_LOCK_VERSION 1
/** @brief The max size of a key we can write, or 0 for dynamic max. /** @brief The max size of a key we can write, or 0 for dynamic max.
* *
@ -692,16 +628,13 @@ typedef struct MDB_txbody {
uint32_t mtb_magic; uint32_t mtb_magic;
/** Format of this lock file. Must be set to #MDB_LOCK_FORMAT. */ /** Format of this lock file. Must be set to #MDB_LOCK_FORMAT. */
uint32_t mtb_format; uint32_t mtb_format;
#if defined(_WIN32) #if defined(_WIN32) || defined(MDB_USE_POSIX_SEM)
char mtb_rmname[MNAME_LEN]; char mtb_rmname[MNAME_LEN];
#elif defined(MDB_USE_SYSV_SEM)
int mtb_semid;
int mtb_rlocked;
#else #else
/** Mutex protecting access to this table. /** Mutex protecting access to this table.
* This is the #MDB_MUTEX(env,r) reader table lock. * This is the reader lock that #LOCK_MUTEX_R acquires.
*/ */
pthread_mutex_t mtb_rmutex; pthread_mutex_t mtb_mutex;
#endif #endif
/** The ID of the last transaction committed to the database. /** The ID of the last transaction committed to the database.
* This is recorded here only for convenience; the value can always * This is recorded here only for convenience; the value can always
@ -721,23 +654,16 @@ typedef struct MDB_txninfo {
MDB_txbody mtb; MDB_txbody mtb;
#define mti_magic mt1.mtb.mtb_magic #define mti_magic mt1.mtb.mtb_magic
#define mti_format mt1.mtb.mtb_format #define mti_format mt1.mtb.mtb_format
#define mti_rmutex mt1.mtb.mtb_rmutex #define mti_mutex mt1.mtb.mtb_mutex
#define mti_rmname mt1.mtb.mtb_rmname #define mti_rmname mt1.mtb.mtb_rmname
#define mti_txnid mt1.mtb.mtb_txnid #define mti_txnid mt1.mtb.mtb_txnid
#define mti_numreaders mt1.mtb.mtb_numreaders #define mti_numreaders mt1.mtb.mtb_numreaders
#ifdef MDB_USE_SYSV_SEM
#define mti_semid mt1.mtb.mtb_semid
#define mti_rlocked mt1.mtb.mtb_rlocked
#endif
char pad[(sizeof(MDB_txbody)+CACHELINE-1) & ~(CACHELINE-1)]; char pad[(sizeof(MDB_txbody)+CACHELINE-1) & ~(CACHELINE-1)];
} mt1; } mt1;
union { union {
#if defined(_WIN32) #if defined(_WIN32) || defined(MDB_USE_POSIX_SEM)
char mt2_wmname[MNAME_LEN]; char mt2_wmname[MNAME_LEN];
#define mti_wmname mt2.mt2_wmname #define mti_wmname mt2.mt2_wmname
#elif defined MDB_USE_SYSV_SEM
int mt2_wlocked;
#define mti_wlocked mt2.mt2_wlocked
#else #else
pthread_mutex_t mt2_wmutex; pthread_mutex_t mt2_wmutex;
#define mti_wmutex mt2.mt2_wmutex #define mti_wmutex mt2.mt2_wmutex
@ -752,7 +678,6 @@ typedef struct MDB_txninfo {
((uint32_t) \ ((uint32_t) \
((MDB_LOCK_VERSION) \ ((MDB_LOCK_VERSION) \
/* Flags which describe functionality */ \ /* Flags which describe functionality */ \
+ (SYSV_SEM_FLAG << 18) \
+ (((MDB_PIDLOCK) != 0) << 16))) + (((MDB_PIDLOCK) != 0) << 16)))
/** @} */ /** @} */
@ -1222,11 +1147,11 @@ struct MDB_env {
int me_live_reader; /**< have liveness lock in reader table */ int me_live_reader; /**< have liveness lock in reader table */
#ifdef _WIN32 #ifdef _WIN32
int me_pidquery; /**< Used in OpenProcess */ int me_pidquery; /**< Used in OpenProcess */
#endif HANDLE me_rmutex; /* Windows mutexes don't reside in shared mem */
#if defined(_WIN32) || defined(MDB_USE_SYSV_SEM) HANDLE me_wmutex;
/* Windows mutexes/SysV semaphores do not reside in shared mem */ #elif defined(MDB_USE_POSIX_SEM)
mdb_mutex_t me_rmutex; sem_t *me_rmutex; /* Shared mutexes are not supported */
mdb_mutex_t me_wmutex; sem_t *me_wmutex;
#endif #endif
void *me_userctx; /**< User-settable context */ void *me_userctx; /**< User-settable context */
MDB_assert_func *me_assert_func; /**< Callback for assertion failures */ MDB_assert_func *me_assert_func; /**< Callback for assertion failures */
@ -1278,7 +1203,7 @@ static int mdb_page_split(MDB_cursor *mc, MDB_val *newkey, MDB_val *newdata,
static int mdb_env_read_header(MDB_env *env, MDB_meta *meta); static int mdb_env_read_header(MDB_env *env, MDB_meta *meta);
static int mdb_env_pick_meta(const MDB_env *env); static int mdb_env_pick_meta(const MDB_env *env);
static int mdb_env_write_meta(MDB_txn *txn); static int mdb_env_write_meta(MDB_txn *txn);
#if !(defined(_WIN32) || defined(MDB_USE_SYSV_SEM)) /* Drop unused excl arg */ #if !(defined(_WIN32) || defined(MDB_USE_POSIX_SEM)) /* Drop unused excl arg */
# define mdb_env_close0(env, excl) mdb_env_close1(env) # define mdb_env_close0(env, excl) mdb_env_close1(env)
#endif #endif
static void mdb_env_close0(MDB_env *env, int excl); static void mdb_env_close0(MDB_env *env, int excl);
@ -1315,7 +1240,6 @@ static void mdb_xcursor_init1(MDB_cursor *mc, MDB_node *node);
static int mdb_drop0(MDB_cursor *mc, int subs); static int mdb_drop0(MDB_cursor *mc, int subs);
static void mdb_default_cmp(MDB_txn *txn, MDB_dbi dbi); static void mdb_default_cmp(MDB_txn *txn, MDB_dbi dbi);
static int mdb_reader_check0(MDB_env *env, int rlocked, int *dead);
/** @cond */ /** @cond */
static MDB_cmp_func mdb_cmp_memn, mdb_cmp_memnr, mdb_cmp_int, mdb_cmp_cint, mdb_cmp_long; static MDB_cmp_func mdb_cmp_memn, mdb_cmp_memnr, mdb_cmp_int, mdb_cmp_cint, mdb_cmp_long;
@ -1343,7 +1267,7 @@ static char *const mdb_errstr[] = {
"MDB_NOTFOUND: No matching key/data pair found", "MDB_NOTFOUND: No matching key/data pair found",
"MDB_PAGE_NOTFOUND: Requested page not found", "MDB_PAGE_NOTFOUND: Requested page not found",
"MDB_CORRUPTED: Located page was wrong type", "MDB_CORRUPTED: Located page was wrong type",
"MDB_PANIC: Update of meta page failed or environment had fatal error", "MDB_PANIC: Update of meta page failed",
"MDB_VERSION_MISMATCH: Database environment version mismatch", "MDB_VERSION_MISMATCH: Database environment version mismatch",
"MDB_INVALID: File is not an LMDB file", "MDB_INVALID: File is not an LMDB file",
"MDB_MAP_FULL: Environment mapsize limit reached", "MDB_MAP_FULL: Environment mapsize limit reached",
@ -2583,7 +2507,6 @@ mdb_txn_renew0(MDB_txn *txn)
} else { } else {
MDB_PID_T pid = env->me_pid; MDB_PID_T pid = env->me_pid;
MDB_THR_T tid = pthread_self(); MDB_THR_T tid = pthread_self();
mdb_mutex_t *rmutex = MDB_MUTEX(env, r);
if (!env->me_live_reader) { if (!env->me_live_reader) {
rc = mdb_reader_pid(env, Pidset, pid); rc = mdb_reader_pid(env, Pidset, pid);
@ -2592,26 +2515,24 @@ mdb_txn_renew0(MDB_txn *txn)
env->me_live_reader = 1; env->me_live_reader = 1;
} }
if (LOCK_MUTEX(rc, env, rmutex)) LOCK_MUTEX_R(env);
return rc;
nr = ti->mti_numreaders; nr = ti->mti_numreaders;
for (i=0; i<nr; i++) for (i=0; i<nr; i++)
if (ti->mti_readers[i].mr_pid == 0) if (ti->mti_readers[i].mr_pid == 0)
break; break;
if (i == env->me_maxreaders) { if (i == env->me_maxreaders) {
UNLOCK_MUTEX(rmutex); UNLOCK_MUTEX_R(env);
return MDB_READERS_FULL; return MDB_READERS_FULL;
} }
r = &ti->mti_readers[i]; ti->mti_readers[i].mr_pid = pid;
r->mr_txnid = (txnid_t)-1; ti->mti_readers[i].mr_tid = tid;
r->mr_tid = tid;
r->mr_pid = pid; /* should be written last, see ITS#7971. */
if (i == nr) if (i == nr)
ti->mti_numreaders = ++nr; ti->mti_numreaders = ++nr;
/* Save numreaders for un-mutexed mdb_env_close() */ /* Save numreaders for un-mutexed mdb_env_close() */
env->me_numreaders = nr; env->me_numreaders = nr;
UNLOCK_MUTEX(rmutex); UNLOCK_MUTEX_R(env);
r = &ti->mti_readers[i];
new_notls = (env->me_flags & MDB_NOTLS); new_notls = (env->me_flags & MDB_NOTLS);
if (!new_notls && (rc=pthread_setspecific(env->me_txkey, r))) { if (!new_notls && (rc=pthread_setspecific(env->me_txkey, r))) {
r->mr_pid = 0; r->mr_pid = 0;
@ -2627,8 +2548,8 @@ mdb_txn_renew0(MDB_txn *txn)
} }
} else { } else {
if (ti) { if (ti) {
if (LOCK_MUTEX(rc, env, MDB_MUTEX(env, w))) LOCK_MUTEX_W(env);
return rc;
txn->mt_txnid = ti->mti_txnid; txn->mt_txnid = ti->mti_txnid;
meta = env->me_metas[txn->mt_txnid & 1]; meta = env->me_metas[txn->mt_txnid & 1];
} else { } else {
@ -2822,13 +2743,6 @@ mdb_txn_env(MDB_txn *txn)
return txn->mt_env; return txn->mt_env;
} }
size_t
mdb_txn_id(MDB_txn *txn)
{
if(!txn) return 0;
return txn->mt_txnid;
}
/** Export or close DBI handles opened in this txn. */ /** Export or close DBI handles opened in this txn. */
static void static void
mdb_dbis_update(MDB_txn *txn, int keep) mdb_dbis_update(MDB_txn *txn, int keep)
@ -2901,7 +2815,7 @@ mdb_txn_reset0(MDB_txn *txn, const char *act)
env->me_txn = NULL; env->me_txn = NULL;
/* The writer mutex was locked in mdb_txn_begin. */ /* The writer mutex was locked in mdb_txn_begin. */
if (env->me_txns) if (env->me_txns)
UNLOCK_MUTEX(MDB_MUTEX(env, w)); UNLOCK_MUTEX_W(env);
} else { } else {
txn->mt_parent->mt_child = NULL; txn->mt_parent->mt_child = NULL;
env->me_pgstate = ((MDB_ntxn *)txn)->mnt_pgstate; env->me_pgstate = ((MDB_ntxn *)txn)->mnt_pgstate;
@ -3494,7 +3408,7 @@ done:
mdb_dbis_update(txn, 1); mdb_dbis_update(txn, 1);
if (env->me_txns) if (env->me_txns)
UNLOCK_MUTEX(MDB_MUTEX(env, w)); UNLOCK_MUTEX_W(env);
if (txn != env->me_txn0) if (txn != env->me_txn0)
free(txn); free(txn);
@ -3569,7 +3483,6 @@ mdb_env_read_header(MDB_env *env, MDB_meta *meta)
return 0; return 0;
} }
/** Fill in most of the zeroed #MDB_meta for an empty database environment */
static void ESECT static void ESECT
mdb_env_init_meta0(MDB_env *env, MDB_meta *meta) mdb_env_init_meta0(MDB_env *env, MDB_meta *meta)
{ {
@ -3586,7 +3499,7 @@ mdb_env_init_meta0(MDB_env *env, MDB_meta *meta)
/** Write the environment parameters of a freshly created DB environment. /** Write the environment parameters of a freshly created DB environment.
* @param[in] env the environment handle * @param[in] env the environment handle
* @param[in] meta the #MDB_meta to write * @param[out] meta address of where to store the meta information
* @return 0 on success, non-zero on failure. * @return 0 on success, non-zero on failure.
*/ */
static int ESECT static int ESECT
@ -3613,6 +3526,8 @@ mdb_env_init_meta(MDB_env *env, MDB_meta *meta)
psize = env->me_psize; psize = env->me_psize;
mdb_env_init_meta0(env, meta);
p = calloc(2, psize); p = calloc(2, psize);
p->mp_pgno = 0; p->mp_pgno = 0;
p->mp_flags = P_META; p->mp_flags = P_META;
@ -3670,10 +3585,6 @@ mdb_env_write_meta(MDB_txn *txn)
mp->mm_dbs[0] = txn->mt_dbs[0]; mp->mm_dbs[0] = txn->mt_dbs[0];
mp->mm_dbs[1] = txn->mt_dbs[1]; mp->mm_dbs[1] = txn->mt_dbs[1];
mp->mm_last_pg = txn->mt_next_pgno - 1; mp->mm_last_pg = txn->mt_next_pgno - 1;
#if !(defined(_MSC_VER) || defined(__i386__) || defined(__x86_64__))
/* LY: issue a memory barrier, if not x86. ITS#7969 */
__sync_synchronize();
#endif
mp->mm_txnid = txn->mt_txnid; mp->mm_txnid = txn->mt_txnid;
if (!(env->me_flags & (MDB_NOMETASYNC|MDB_NOSYNC))) { if (!(env->me_flags & (MDB_NOMETASYNC|MDB_NOSYNC))) {
unsigned meta_size = env->me_psize; unsigned meta_size = env->me_psize;
@ -3783,9 +3694,9 @@ mdb_env_create(MDB_env **env)
e->me_fd = INVALID_HANDLE_VALUE; e->me_fd = INVALID_HANDLE_VALUE;
e->me_lfd = INVALID_HANDLE_VALUE; e->me_lfd = INVALID_HANDLE_VALUE;
e->me_mfd = INVALID_HANDLE_VALUE; e->me_mfd = INVALID_HANDLE_VALUE;
#ifdef MDB_USE_SYSV_SEM #ifdef MDB_USE_POSIX_SEM
e->me_rmutex.semid = -1; e->me_rmutex = SEM_FAILED;
e->me_wmutex.semid = -1; e->me_wmutex = SEM_FAILED;
#endif #endif
e->me_pid = getpid(); e->me_pid = getpid();
GET_PAGESIZE(e->me_os_psize); GET_PAGESIZE(e->me_os_psize);
@ -3886,16 +3797,16 @@ mdb_env_set_mapsize(MDB_env *env, size_t size)
*/ */
if (env->me_map) { if (env->me_map) {
int rc; int rc;
MDB_meta *meta;
void *old; void *old;
if (env->me_txn) if (env->me_txn)
return EINVAL; return EINVAL;
meta = env->me_metas[mdb_env_pick_meta(env)];
if (!size) if (!size)
size = meta->mm_mapsize; size = env->me_metas[mdb_env_pick_meta(env)]->mm_mapsize;
{ else if (size < env->me_mapsize) {
/* Silently round up to minimum if the size is too small */ /* If the configured size is smaller, make sure it's
size_t minsize = (meta->mm_last_pg + 1) * env->me_psize; * still big enough. Silently round up to minimum if not.
*/
size_t minsize = (env->me_metas[mdb_env_pick_meta(env)]->mm_last_pg + 1) * env->me_psize;
if (size < minsize) if (size < minsize)
size = minsize; size = minsize;
} }
@ -3982,7 +3893,6 @@ mdb_env_open2(MDB_env *env)
else else
env->me_pidquery = PROCESS_QUERY_INFORMATION; env->me_pidquery = PROCESS_QUERY_INFORMATION;
#endif /* _WIN32 */ #endif /* _WIN32 */
#ifdef BROKEN_FDATASYNC #ifdef BROKEN_FDATASYNC
/* ext3/ext4 fdatasync is broken on some older Linux kernels. /* ext3/ext4 fdatasync is broken on some older Linux kernels.
* https://lkml.org/lkml/2012/9/3/83 * https://lkml.org/lkml/2012/9/3/83
@ -4031,6 +3941,8 @@ mdb_env_open2(MDB_env *env)
} }
#endif #endif
memset(&meta, 0, sizeof(meta));
if ((i = mdb_env_read_header(env, &meta)) != 0) { if ((i = mdb_env_read_header(env, &meta)) != 0) {
if (i != ENOENT) if (i != ENOENT)
return i; return i;
@ -4039,40 +3951,24 @@ mdb_env_open2(MDB_env *env)
env->me_psize = env->me_os_psize; env->me_psize = env->me_os_psize;
if (env->me_psize > MAX_PAGESIZE) if (env->me_psize > MAX_PAGESIZE)
env->me_psize = MAX_PAGESIZE; env->me_psize = MAX_PAGESIZE;
memset(&meta, 0, sizeof(meta));
mdb_env_init_meta0(env, &meta);
meta.mm_mapsize = DEFAULT_MAPSIZE;
} else { } else {
env->me_psize = meta.mm_psize; env->me_psize = meta.mm_psize;
} }
/* Was a mapsize configured? */ /* Was a mapsize configured? */
if (!env->me_mapsize) { if (!env->me_mapsize) {
env->me_mapsize = meta.mm_mapsize; /* If this is a new environment, take the default,
} * else use the size recorded in the existing env.
{ */
/* Make sure mapsize >= committed data size. Even when using env->me_mapsize = newenv ? DEFAULT_MAPSIZE : meta.mm_mapsize;
* mm_mapsize, which could be broken in old files (ITS#7789). } else if (env->me_mapsize < meta.mm_mapsize) {
/* If the configured size is smaller, make sure it's
* still big enough. Silently round up to minimum if not.
*/ */
size_t minsize = (meta.mm_last_pg + 1) * meta.mm_psize; size_t minsize = (meta.mm_last_pg + 1) * meta.mm_psize;
if (env->me_mapsize < minsize) if (env->me_mapsize < minsize)
env->me_mapsize = minsize; env->me_mapsize = minsize;
} }
meta.mm_mapsize = env->me_mapsize;
if (newenv && !(flags & MDB_FIXEDMAP)) {
/* mdb_env_map() may grow the datafile. Write the metapages
* first, so the file will be valid if initialization fails.
* Except with FIXEDMAP, since we do not yet know mm_address.
* We could fill in mm_address later, but then a different
* program might end up doing that - one with a memory layout
* and map address which does not suit the main program.
*/
rc = mdb_env_init_meta(env, &meta);
if (rc)
return rc;
newenv = 0;
}
rc = mdb_env_map(env, (flags & MDB_FIXEDMAP) ? meta.mm_address : NULL); rc = mdb_env_map(env, (flags & MDB_FIXEDMAP) ? meta.mm_address : NULL);
if (rc) if (rc)
@ -4256,8 +4152,8 @@ mdb_env_excl_lock(MDB_env *env, int *excl)
if (!rc) { if (!rc) {
*excl = 1; *excl = 1;
} else } else
# ifdef MDB_USE_SYSV_SEM # ifdef MDB_USE_POSIX_SEM
if (*excl < 0) /* always true when !MDB_USE_SYSV_SEM */ if (*excl < 0) /* always true when !MDB_USE_POSIX_SEM */
# endif # endif
{ {
lock_info.l_type = F_RDLCK; lock_info.l_type = F_RDLCK;
@ -4382,10 +4278,6 @@ mdb_env_setup_locks(MDB_env *env, char *lpath, int mode, int *excl)
int fdflags; int fdflags;
# define MDB_CLOEXEC 0 # define MDB_CLOEXEC 0
#endif #endif
#endif
#ifdef MDB_USE_SYSV_SEM
int semid;
union semun semu;
#endif #endif
int rc; int rc;
off_t size, rsize; off_t size, rsize;
@ -4499,28 +4391,50 @@ mdb_env_setup_locks(MDB_env *env, char *lpath, int mode, int *excl)
if (!env->me_rmutex) goto fail_errno; if (!env->me_rmutex) goto fail_errno;
env->me_wmutex = CreateMutex(&mdb_all_sa, FALSE, env->me_txns->mti_wmname); env->me_wmutex = CreateMutex(&mdb_all_sa, FALSE, env->me_txns->mti_wmname);
if (!env->me_wmutex) goto fail_errno; if (!env->me_wmutex) goto fail_errno;
#elif defined(MDB_USE_SYSV_SEM) #elif defined(MDB_USE_POSIX_SEM)
unsigned short vals[2] = {1, 1}; struct stat stbuf;
semid = semget(IPC_PRIVATE, 2, mode); struct {
if (semid < 0) dev_t dev;
goto fail_errno; ino_t ino;
semu.array = vals; } idbuf;
if (semctl(semid, 0, SETALL, semu) < 0) MDB_val val;
goto fail_errno; char encbuf[11];
env->me_txns->mti_semid = semid;
#else /* MDB_USE_SYSV_SEM */ #if defined(__NetBSD__)
#define MDB_SHORT_SEMNAMES 1 /* limited to 14 chars */
#endif
if (fstat(env->me_lfd, &stbuf)) goto fail_errno;
idbuf.dev = stbuf.st_dev;
idbuf.ino = stbuf.st_ino;
val.mv_data = &idbuf;
val.mv_size = sizeof(idbuf);
mdb_hash_enc(&val, encbuf);
#ifdef MDB_SHORT_SEMNAMES
encbuf[9] = '\0'; /* drop name from 15 chars to 14 chars */
#endif
sprintf(env->me_txns->mti_rmname, "/MDBr%s", encbuf);
sprintf(env->me_txns->mti_wmname, "/MDBw%s", encbuf);
/* Clean up after a previous run, if needed: Try to
* remove both semaphores before doing anything else.
*/
sem_unlink(env->me_txns->mti_rmname);
sem_unlink(env->me_txns->mti_wmname);
env->me_rmutex = sem_open(env->me_txns->mti_rmname,
O_CREAT|O_EXCL, mode, 1);
if (env->me_rmutex == SEM_FAILED) goto fail_errno;
env->me_wmutex = sem_open(env->me_txns->mti_wmname,
O_CREAT|O_EXCL, mode, 1);
if (env->me_wmutex == SEM_FAILED) goto fail_errno;
#else /* MDB_USE_POSIX_SEM */
pthread_mutexattr_t mattr; pthread_mutexattr_t mattr;
if ((rc = pthread_mutexattr_init(&mattr)) if ((rc = pthread_mutexattr_init(&mattr))
|| (rc = pthread_mutexattr_setpshared(&mattr, PTHREAD_PROCESS_SHARED)) || (rc = pthread_mutexattr_setpshared(&mattr, PTHREAD_PROCESS_SHARED))
#ifdef MDB_ROBUST_SUPPORTED || (rc = pthread_mutex_init(&env->me_txns->mti_mutex, &mattr))
|| (rc = pthread_mutexattr_setrobust(&mattr, PTHREAD_MUTEX_ROBUST))
#endif
|| (rc = pthread_mutex_init(&env->me_txns->mti_rmutex, &mattr))
|| (rc = pthread_mutex_init(&env->me_txns->mti_wmutex, &mattr))) || (rc = pthread_mutex_init(&env->me_txns->mti_wmutex, &mattr)))
goto fail; goto fail;
pthread_mutexattr_destroy(&mattr); pthread_mutexattr_destroy(&mattr);
#endif /* _WIN32 || MDB_USE_SYSV_SEM */ #endif /* _WIN32 || MDB_USE_POSIX_SEM */
env->me_txns->mti_magic = MDB_MAGIC; env->me_txns->mti_magic = MDB_MAGIC;
env->me_txns->mti_format = MDB_LOCK_FORMAT; env->me_txns->mti_format = MDB_LOCK_FORMAT;
@ -4528,9 +4442,6 @@ mdb_env_setup_locks(MDB_env *env, char *lpath, int mode, int *excl)
env->me_txns->mti_numreaders = 0; env->me_txns->mti_numreaders = 0;
} else { } else {
#ifdef MDB_USE_SYSV_SEM
struct semid_ds buf;
#endif
if (env->me_txns->mti_magic != MDB_MAGIC) { if (env->me_txns->mti_magic != MDB_MAGIC) {
DPUTS("lock region has invalid magic"); DPUTS("lock region has invalid magic");
rc = MDB_INVALID; rc = MDB_INVALID;
@ -4551,26 +4462,13 @@ mdb_env_setup_locks(MDB_env *env, char *lpath, int mode, int *excl)
if (!env->me_rmutex) goto fail_errno; if (!env->me_rmutex) goto fail_errno;
env->me_wmutex = OpenMutex(SYNCHRONIZE, FALSE, env->me_txns->mti_wmname); env->me_wmutex = OpenMutex(SYNCHRONIZE, FALSE, env->me_txns->mti_wmname);
if (!env->me_wmutex) goto fail_errno; if (!env->me_wmutex) goto fail_errno;
#elif defined(MDB_USE_SYSV_SEM) #elif defined(MDB_USE_POSIX_SEM)
semid = env->me_txns->mti_semid; env->me_rmutex = sem_open(env->me_txns->mti_rmname, 0);
semu.buf = &buf; if (env->me_rmutex == SEM_FAILED) goto fail_errno;
/* check for read access */ env->me_wmutex = sem_open(env->me_txns->mti_wmname, 0);
if (semctl(semid, 0, IPC_STAT, semu) < 0) if (env->me_wmutex == SEM_FAILED) goto fail_errno;
goto fail_errno;
/* check for write access */
if (semctl(semid, 0, IPC_SET, semu) < 0)
goto fail_errno;
#endif #endif
} }
#ifdef MDB_USE_SYSV_SEM
env->me_rmutex.semid = semid;
env->me_wmutex.semid = semid;
env->me_rmutex.semnum = 0;
env->me_wmutex.semnum = 1;
env->me_rmutex.locked = &env->me_txns->mti_rlocked;
env->me_wmutex.locked = &env->me_txns->mti_wlocked;
#endif
return MDB_SUCCESS; return MDB_SUCCESS;
fail_errno: fail_errno:
@ -4590,8 +4488,8 @@ fail:
* environment and re-opening it with the new flags. * environment and re-opening it with the new flags.
*/ */
#define CHANGEABLE (MDB_NOSYNC|MDB_NOMETASYNC|MDB_MAPASYNC|MDB_NOMEMINIT) #define CHANGEABLE (MDB_NOSYNC|MDB_NOMETASYNC|MDB_MAPASYNC|MDB_NOMEMINIT)
#define CHANGELESS (MDB_FIXEDMAP|MDB_NOSUBDIR|MDB_RDONLY| \ #define CHANGELESS (MDB_FIXEDMAP|MDB_NOSUBDIR|MDB_RDONLY|MDB_WRITEMAP| \
MDB_WRITEMAP|MDB_NOTLS|MDB_NOLOCK|MDB_NORDAHEAD) MDB_NOTLS|MDB_NOLOCK|MDB_NORDAHEAD)
#if VALID_FLAGS & PERSISTENT_FLAGS & (CHANGEABLE|CHANGELESS) #if VALID_FLAGS & PERSISTENT_FLAGS & (CHANGEABLE|CHANGELESS)
# error "Persistent DB flags & env flags overlap, but both go in mm_flags" # error "Persistent DB flags & env flags overlap, but both go in mm_flags"
@ -4802,15 +4700,20 @@ mdb_env_close0(MDB_env *env, int excl)
/* Windows automatically destroys the mutexes when /* Windows automatically destroys the mutexes when
* the last handle closes. * the last handle closes.
*/ */
#elif defined(MDB_USE_SYSV_SEM) #elif defined(MDB_USE_POSIX_SEM)
if (env->me_rmutex.semid != -1) { if (env->me_rmutex != SEM_FAILED) {
sem_close(env->me_rmutex);
if (env->me_wmutex != SEM_FAILED)
sem_close(env->me_wmutex);
/* If we have the filelock: If we are the /* If we have the filelock: If we are the
* only remaining user, clean up semaphores. * only remaining user, clean up semaphores.
*/ */
if (excl == 0) if (excl == 0)
mdb_env_excl_lock(env, &excl); mdb_env_excl_lock(env, &excl);
if (excl > 0) if (excl > 0) {
semctl(env->me_rmutex.semid, 0, IPC_RMID); sem_unlink(env->me_txns->mti_rmname);
sem_unlink(env->me_txns->mti_wmname);
}
} }
#endif #endif
munmap((void *)env->me_txns, (env->me_maxreaders-1)*sizeof(MDB_reader)+sizeof(MDB_txninfo)); munmap((void *)env->me_txns, (env->me_maxreaders-1)*sizeof(MDB_reader)+sizeof(MDB_txninfo));
@ -4830,6 +4733,7 @@ mdb_env_close0(MDB_env *env, int excl)
env->me_flags &= ~(MDB_ENV_ACTIVE|MDB_ENV_TXKEY); env->me_flags &= ~(MDB_ENV_ACTIVE|MDB_ENV_TXKEY);
} }
void ESECT void ESECT
mdb_env_close(MDB_env *env) mdb_env_close(MDB_env *env)
{ {
@ -6164,6 +6068,7 @@ int
mdb_cursor_put(MDB_cursor *mc, MDB_val *key, MDB_val *data, mdb_cursor_put(MDB_cursor *mc, MDB_val *key, MDB_val *data,
unsigned int flags) unsigned int flags)
{ {
enum { MDB_NO_ROOT = MDB_LAST_ERRCODE+10 }; /* internal code */
MDB_env *env; MDB_env *env;
MDB_node *leaf = NULL; MDB_node *leaf = NULL;
MDB_page *fp, *mp; MDB_page *fp, *mp;
@ -8822,7 +8727,6 @@ static int ESECT
mdb_env_copyfd0(MDB_env *env, HANDLE fd) mdb_env_copyfd0(MDB_env *env, HANDLE fd)
{ {
MDB_txn *txn = NULL; MDB_txn *txn = NULL;
mdb_mutex_t *wmutex = NULL;
int rc; int rc;
size_t wsize; size_t wsize;
char *ptr; char *ptr;
@ -8847,13 +8751,11 @@ mdb_env_copyfd0(MDB_env *env, HANDLE fd)
mdb_txn_reset0(txn, "reset-stage1"); mdb_txn_reset0(txn, "reset-stage1");
/* Temporarily block writers until we snapshot the meta pages */ /* Temporarily block writers until we snapshot the meta pages */
wmutex = MDB_MUTEX(env, w); LOCK_MUTEX_W(env);
if (LOCK_MUTEX(rc, env, wmutex))
goto leave;
rc = mdb_txn_renew0(txn); rc = mdb_txn_renew0(txn);
if (rc) { if (rc) {
UNLOCK_MUTEX(wmutex); UNLOCK_MUTEX_W(env);
goto leave; goto leave;
} }
} }
@ -8877,8 +8779,8 @@ mdb_env_copyfd0(MDB_env *env, HANDLE fd)
break; break;
} }
} }
if (wmutex) if (env->me_txns)
UNLOCK_MUTEX(wmutex); UNLOCK_MUTEX_W(env);
if (rc) if (rc)
goto leave; goto leave;
@ -9001,7 +8903,7 @@ mdb_env_copy(MDB_env *env, const char *path)
int ESECT int ESECT
mdb_env_set_flags(MDB_env *env, unsigned int flag, int onoff) mdb_env_set_flags(MDB_env *env, unsigned int flag, int onoff)
{ {
if (flag & (env->me_map ? ~CHANGEABLE : ~(CHANGEABLE|CHANGELESS))) if ((flag & CHANGEABLE) != flag)
return EINVAL; return EINVAL;
if (onoff) if (onoff)
env->me_flags |= flag; env->me_flags |= flag;
@ -9555,22 +9457,17 @@ mdb_pid_insert(MDB_PID_T *ids, MDB_PID_T pid)
int ESECT int ESECT
mdb_reader_check(MDB_env *env, int *dead) mdb_reader_check(MDB_env *env, int *dead)
{ {
unsigned int i, j, rdrs;
MDB_reader *mr;
MDB_PID_T *pids, pid;
int count = 0;
if (!env) if (!env)
return EINVAL; return EINVAL;
if (dead) if (dead)
*dead = 0; *dead = 0;
return env->me_txns ? mdb_reader_check0(env, 0, dead) : MDB_SUCCESS; if (!env->me_txns)
} return MDB_SUCCESS;
/** As #mdb_reader_check(). rlocked = <caller locked the reader mutex>. */
static int mdb_reader_check0(MDB_env *env, int rlocked, int *dead)
{
mdb_mutex_t *rmutex = rlocked ? NULL : MDB_MUTEX(env, r);
unsigned int i, j, rdrs;
MDB_reader *mr;
MDB_PID_T *pids, pid;
int rc = MDB_SUCCESS, count = 0;
rdrs = env->me_txns->mti_numreaders; rdrs = env->me_txns->mti_numreaders;
pids = malloc((rdrs+1) * sizeof(MDB_PID_T)); pids = malloc((rdrs+1) * sizeof(MDB_PID_T));
if (!pids) if (!pids)
@ -9578,32 +9475,22 @@ static int mdb_reader_check0(MDB_env *env, int rlocked, int *dead)
pids[0] = 0; pids[0] = 0;
mr = env->me_txns->mti_readers; mr = env->me_txns->mti_readers;
for (i=0; i<rdrs; i++) { for (i=0; i<rdrs; i++) {
if (mr[i].mr_pid && mr[i].mr_pid != env->me_pid) {
pid = mr[i].mr_pid; pid = mr[i].mr_pid;
if (pid && pid != env->me_pid) {
if (mdb_pid_insert(pids, pid) == 0) { if (mdb_pid_insert(pids, pid) == 0) {
if (!mdb_reader_pid(env, Pidcheck, pid)) { if (!mdb_reader_pid(env, Pidcheck, pid)) {
/* Stale reader found */ LOCK_MUTEX_R(env);
j = i;
if (rmutex) {
if ((rc = LOCK_MUTEX0(rmutex)) != 0) {
if ((rc = mdb_mutex_failed(env, rmutex, rc)))
break;
rdrs = 0; /* the above checked all readers */
} else {
/* Recheck, a new process may have reused pid */ /* Recheck, a new process may have reused pid */
if (mdb_reader_pid(env, Pidcheck, pid)) if (!mdb_reader_pid(env, Pidcheck, pid)) {
j = rdrs; for (j=i; j<rdrs; j++)
}
}
for (; j<rdrs; j++)
if (mr[j].mr_pid == pid) { if (mr[j].mr_pid == pid) {
DPRINTF(("clear stale reader pid %u txn %"Z"d", DPRINTF(("clear stale reader pid %u txn %"Z"d",
(unsigned) pid, mr[j].mr_txnid)); (unsigned) pid, mr[j].mr_txnid));
mr[j].mr_pid = 0; mr[j].mr_pid = 0;
count++; count++;
} }
if (rmutex) }
UNLOCK_MUTEX(rmutex); UNLOCK_MUTEX_R(env);
} }
} }
} }
@ -9611,55 +9498,6 @@ static int mdb_reader_check0(MDB_env *env, int rlocked, int *dead)
free(pids); free(pids);
if (dead) if (dead)
*dead = count; *dead = count;
return rc; return MDB_SUCCESS;
} }
#ifdef MDB_ROBUST_SUPPORTED
/** Handle #LOCK_MUTEX0() failure.
* Try to repair the lock file if the mutex owner died.
* @param[in] env the environment handle
* @param[in] mutex LOCK_MUTEX0() mutex
* @param[in] rc LOCK_MUTEX0() error (nonzero)
* @return 0 on success with the mutex locked, or an error code on failure.
*/
static int mdb_mutex_failed(MDB_env *env, mdb_mutex_t *mutex, int rc)
{
int toggle, rlocked, rc2;
if (rc == MDB_OWNERDEAD) {
/* We own the mutex. Clean up after dead previous owner. */
rc = MDB_SUCCESS;
rlocked = (mutex == MDB_MUTEX(env, r));
if (!rlocked) {
/* Keep mti_txnid updated, otherwise next writer can
* overwrite data which latest meta page refers to.
*/
toggle = mdb_env_pick_meta(env);
env->me_txns->mti_txnid = env->me_metas[toggle]->mm_txnid;
/* env is hosed if the dead thread was ours */
if (env->me_txn) {
env->me_flags |= MDB_FATAL_ERROR;
env->me_txn = NULL;
rc = MDB_PANIC;
}
}
DPRINTF(("%cmutex owner died, %s", (rlocked ? 'r' : 'w'),
(rc ? "this process' env is hosed" : "recovering")));
rc2 = mdb_reader_check0(env, rlocked, NULL);
if (rc2 == 0)
rc2 = mdb_mutex_consistent(mutex);
if (rc || (rc = rc2)) {
DPRINTF(("LOCK_MUTEX recovery failed, %s", mdb_strerror(rc)));
UNLOCK_MUTEX(mutex);
}
} else {
#ifdef _WIN32
rc = ErrCode();
#endif
DPRINTF(("LOCK_MUTEX failed, %s", mdb_strerror(rc)));
}
return rc;
}
#endif /* MDB_ROBUST_SUPPORTED */
/** @} */ /** @} */

2
external/db_drivers/liblmdb64/mdb_copy.1 vendored
View file

@ -1,5 +1,5 @@
.TH MDB_COPY 1 "2014/06/20" "LMDB 0.9.14" .TH MDB_COPY 1 "2014/06/20" "LMDB 0.9.14"
.\" Copyright 2012-2014 Howard Chu, Symas Corp. All Rights Reserved. .\" Copyright 2012-2015 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE. .\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.SH NAME .SH NAME
mdb_copy \- LMDB environment copy tool mdb_copy \- LMDB environment copy tool

2
external/db_drivers/liblmdb64/mdb_copy.c vendored
View file

@ -1,6 +1,6 @@
/* mdb_copy.c - memory-mapped database backup tool */ /* mdb_copy.c - memory-mapped database backup tool */
/* /*
* Copyright 2012 Howard Chu, Symas Corp. * Copyright 2012-2015 Howard Chu, Symas Corp.
* All rights reserved. * All rights reserved.
* *
* Redistribution and use in source and binary forms, with or without * Redistribution and use in source and binary forms, with or without

2
external/db_drivers/liblmdb64/mdb_dump.1 vendored
View file

@ -1,5 +1,5 @@
.TH MDB_DUMP 1 "2014/06/20" "LMDB 0.9.14" .TH MDB_DUMP 1 "2014/06/20" "LMDB 0.9.14"
.\" Copyright 2014 Howard Chu, Symas Corp. All Rights Reserved. .\" Copyright 2014-2015 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE. .\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.SH NAME .SH NAME
mdb_dump \- LMDB environment export tool mdb_dump \- LMDB environment export tool

2
external/db_drivers/liblmdb64/mdb_dump.c vendored
View file

@ -1,6 +1,6 @@
/* mdb_dump.c - memory-mapped database dump tool */ /* mdb_dump.c - memory-mapped database dump tool */
/* /*
* Copyright 2011-2014 Howard Chu, Symas Corp. * Copyright 2011-2015 Howard Chu, Symas Corp.
* All rights reserved. * All rights reserved.
* *
* Redistribution and use in source and binary forms, with or without * Redistribution and use in source and binary forms, with or without

2
external/db_drivers/liblmdb64/mdb_load.1 vendored
View file

@ -1,5 +1,5 @@
.TH MDB_LOAD 1 "2014/06/20" "LMDB 0.9.14" .TH MDB_LOAD 1 "2014/06/20" "LMDB 0.9.14"
.\" Copyright 2014 Howard Chu, Symas Corp. All Rights Reserved. .\" Copyright 2014-2015 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE. .\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.SH NAME .SH NAME
mdb_load \- LMDB environment import tool mdb_load \- LMDB environment import tool

12
external/db_drivers/liblmdb64/mdb_load.c vendored
View file

@ -1,6 +1,6 @@
/* mdb_load.c - memory-mapped database load tool */ /* mdb_load.c - memory-mapped database load tool */
/* /*
* Copyright 2011-2014 Howard Chu, Symas Corp. * Copyright 2011-2015 Howard Chu, Symas Corp.
* All rights reserved. * All rights reserved.
* *
* Redistribution and use in source and binary forms, with or without * Redistribution and use in source and binary forms, with or without
@ -176,7 +176,7 @@ static int unhex(unsigned char *c2)
static int readline(MDB_val *out, MDB_val *buf) static int readline(MDB_val *out, MDB_val *buf)
{ {
unsigned char *c1, *c2, *end; unsigned char *c1, *c2, *end;
size_t len; size_t len, l2;
int c; int c;
if (!(mode & NOHDR)) { if (!(mode & NOHDR)) {
@ -206,6 +206,7 @@ badend:
c1 = buf->mv_data; c1 = buf->mv_data;
len = strlen((char *)c1); len = strlen((char *)c1);
l2 = len;
/* Is buffer too short? */ /* Is buffer too short? */
while (c1[len-1] != '\n') { while (c1[len-1] != '\n') {
@ -217,17 +218,18 @@ badend:
return EOF; return EOF;
} }
c1 = buf->mv_data; c1 = buf->mv_data;
c1 += buf->mv_size; c1 += l2;
if (fgets((char *)c1, buf->mv_size, stdin) == NULL) { if (fgets((char *)c1, buf->mv_size+1, stdin) == NULL) {
Eof = 1; Eof = 1;
badend(); badend();
return EOF; return EOF;
} }
buf->mv_size *= 2; buf->mv_size *= 2;
len = strlen((char *)c1); len = strlen((char *)c1);
l2 += len;
} }
c1 = c2 = buf->mv_data; c1 = c2 = buf->mv_data;
len = strlen((char *)c1); len = l2;
c1[--len] = '\0'; c1[--len] = '\0';
end = c1 + len; end = c1 + len;

2
external/db_drivers/liblmdb64/mdb_stat.1 vendored
View file

@ -1,5 +1,5 @@
.TH MDB_STAT 1 "2014/06/20" "LMDB 0.9.14" .TH MDB_STAT 1 "2014/06/20" "LMDB 0.9.14"
.\" Copyright 2012-2014 Howard Chu, Symas Corp. All Rights Reserved. .\" Copyright 2012-2015 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE. .\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.SH NAME .SH NAME
mdb_stat \- LMDB environment status tool mdb_stat \- LMDB environment status tool

2
external/db_drivers/liblmdb64/mdb_stat.c vendored
View file

@ -1,6 +1,6 @@
/* mdb_stat.c - memory-mapped database status tool */ /* mdb_stat.c - memory-mapped database status tool */
/* /*
* Copyright 2011-2013 Howard Chu, Symas Corp. * Copyright 2011-2015 Howard Chu, Symas Corp.
* All rights reserved. * All rights reserved.
* *
* Redistribution and use in source and binary forms, with or without * Redistribution and use in source and binary forms, with or without

2
external/db_drivers/liblmdb64/midl.c vendored
View file

@ -3,7 +3,7 @@
/* $OpenLDAP$ */ /* $OpenLDAP$ */
/* This work is part of OpenLDAP Software <http://www.openldap.org/>. /* This work is part of OpenLDAP Software <http://www.openldap.org/>.
* *
* Copyright 2000-2014 The OpenLDAP Foundation. * Copyright 2000-2015 The OpenLDAP Foundation.
* All rights reserved. * All rights reserved.
* *
* Redistribution and use in source and binary forms, with or without * Redistribution and use in source and binary forms, with or without

2
external/db_drivers/liblmdb64/midl.h vendored
View file

@ -11,7 +11,7 @@
/* $OpenLDAP$ */ /* $OpenLDAP$ */
/* This work is part of OpenLDAP Software <http://www.openldap.org/>. /* This work is part of OpenLDAP Software <http://www.openldap.org/>.
* *
* Copyright 2000-2014 The OpenLDAP Foundation. * Copyright 2000-2015 The OpenLDAP Foundation.
* All rights reserved. * All rights reserved.
* *
* Redistribution and use in source and binary forms, with or without * Redistribution and use in source and binary forms, with or without

2
external/db_drivers/liblmdb64/mtest.c vendored
View file

@ -1,6 +1,6 @@
/* mtest.c - memory-mapped database tester/toy */ /* mtest.c - memory-mapped database tester/toy */
/* /*
* Copyright 2011-2014 Howard Chu, Symas Corp. * Copyright 2011-2015 Howard Chu, Symas Corp.
* All rights reserved. * All rights reserved.
* *
* Redistribution and use in source and binary forms, with or without * Redistribution and use in source and binary forms, with or without

2
external/db_drivers/liblmdb64/mtest2.c vendored
View file

@ -1,6 +1,6 @@
/* mtest2.c - memory-mapped database tester/toy */ /* mtest2.c - memory-mapped database tester/toy */
/* /*
* Copyright 2011-2014 Howard Chu, Symas Corp. * Copyright 2011-2015 Howard Chu, Symas Corp.
* All rights reserved. * All rights reserved.
* *
* Redistribution and use in source and binary forms, with or without * Redistribution and use in source and binary forms, with or without

2
external/db_drivers/liblmdb64/mtest3.c vendored
View file

@ -1,6 +1,6 @@
/* mtest3.c - memory-mapped database tester/toy */ /* mtest3.c - memory-mapped database tester/toy */
/* /*
* Copyright 2011-2014 Howard Chu, Symas Corp. * Copyright 2011-2015 Howard Chu, Symas Corp.
* All rights reserved. * All rights reserved.
* *
* Redistribution and use in source and binary forms, with or without * Redistribution and use in source and binary forms, with or without

2
external/db_drivers/liblmdb64/mtest4.c vendored
View file

@ -1,6 +1,6 @@
/* mtest4.c - memory-mapped database tester/toy */ /* mtest4.c - memory-mapped database tester/toy */
/* /*
* Copyright 2011-2014 Howard Chu, Symas Corp. * Copyright 2011-2015 Howard Chu, Symas Corp.
* All rights reserved. * All rights reserved.
* *
* Redistribution and use in source and binary forms, with or without * Redistribution and use in source and binary forms, with or without

2
external/db_drivers/liblmdb64/mtest5.c vendored
View file

@ -1,6 +1,6 @@
/* mtest5.c - memory-mapped database tester/toy */ /* mtest5.c - memory-mapped database tester/toy */
/* /*
* Copyright 2011-2014 Howard Chu, Symas Corp. * Copyright 2011-2015 Howard Chu, Symas Corp.
* All rights reserved. * All rights reserved.
* *
* Redistribution and use in source and binary forms, with or without * Redistribution and use in source and binary forms, with or without

2
external/db_drivers/liblmdb64/mtest6.c vendored
View file

@ -1,6 +1,6 @@
/* mtest6.c - memory-mapped database tester/toy */ /* mtest6.c - memory-mapped database tester/toy */
/* /*
* Copyright 2011-2014 Howard Chu, Symas Corp. * Copyright 2011-2015 Howard Chu, Symas Corp.
* All rights reserved. * All rights reserved.
* *
* Redistribution and use in source and binary forms, with or without * Redistribution and use in source and binary forms, with or without

2
external/db_drivers/liblmdb64/sample-bdb.txt vendored
View file

@ -3,7 +3,7 @@
* Do a line-by-line comparison of this and sample-mdb.txt * Do a line-by-line comparison of this and sample-mdb.txt
*/ */
/* /*
* Copyright 2012 Howard Chu, Symas Corp. * Copyright 2012-2015 Howard Chu, Symas Corp.
* All rights reserved. * All rights reserved.
* *
* Redistribution and use in source and binary forms, with or without * Redistribution and use in source and binary forms, with or without

2
external/db_drivers/liblmdb64/sample-mdb.txt vendored
View file

@ -3,7 +3,7 @@
* Do a line-by-line comparison of this and sample-bdb.txt * Do a line-by-line comparison of this and sample-bdb.txt
*/ */
/* /*
* Copyright 2012 Howard Chu, Symas Corp. * Copyright 2012-2015 Howard Chu, Symas Corp.
* All rights reserved. * All rights reserved.
* *
* Redistribution and use in source and binary forms, with or without * Redistribution and use in source and binary forms, with or without