Commit graph

1251 commits

Author SHA1 Message Date
luigi1111
e2b7e55be6 get_payments short ID
Add support for short/integrated/encrypted IDs to get_payments RPC
2016-03-05 13:30:48 -06:00
warptangent
0485d17274
blockchain_export: Support BerkeleyDB
TEST:

blockchain_export -h

This should show "berkeley" as an available option to --database.

With an existing BerkeleyDB database, run:

blockchain_export --database berkeley
2016-03-05 06:46:11 -08:00
warptangent
41487e32be
blockchain_export: Move DB implementation includes 2016-03-05 06:46:09 -08:00
warptangent
bdec7cbfb3
BlockchainLMDB: Use DB error helper consistently 2016-03-03 21:27:13 -08:00
warptangent
c5932eb5ea BlockchainLMDB: Add DB error to exception 2016-03-03 20:15:37 -08:00
warptangent
a49c355cae
Blockchain: Omit verbose time stats messages by default
This is already the default for the daemon, but by checking a command
line argument and calling a Blockchain member function setter.

Initialize the variable to false so it's not dependent on an external
command-line argument check. This allows utilities like
blockchain_import to have a reasonable default without code changes.
2016-03-03 20:14:48 -08:00
Howard Chu
ee7a8b87f9 Get rid of lmdb_cur
We don't need it now with per-txn cursors.
2016-03-03 19:06:27 +00:00
Howard Chu
d57ec7595c Fix language 2016-03-03 00:34:19 +00:00
Howard Chu
f5affbef26 More for df239428c0
Let ARMv7 work again
2016-03-03 00:13:13 +00:00
warptangent
2c823c1589
Make partial revert consistent
If user-defined comparator is used, subdb shouldn't be opened with
MDB_INTEGERKEY.

TODO: Again, this will be added back with future schema updates.
2016-03-01 02:47:16 -08:00
warptangent
df239428c0
Revert part of 7db89ed2ee
For now, so existing databases work.

TODO: add these back with future schema updates.
2016-02-29 18:03:41 -08:00
warptangent
c9c4060dc6
BlockchainLMDB: Add height and db error to exception 2016-02-29 17:52:54 -08:00
Riccardo Spagni
bf09285443
Merge pull request #688
c2a1fee simplewallet: prompt for private keys when generating wallets (moneromooo-monero)
4513b4c simplewallet: add a new --restore-from-keys option (moneromooo-monero)
2016-02-29 17:30:29 -05:00
Riccardo Spagni
c3c4e28061
Merge pull request #687
5e3557d move g_test_dbg_lock_sleep from a global to a function level static (moneromooo-monero)
2016-02-29 17:30:10 -05:00
Riccardo Spagni
d60bf4ee36
Merge pull request #686
7db89ed ARMv7: fix unaligned accesses (Howard Chu)
5a07cef Wrap some more actions in a larger read txn (Howard Chu)
8cc7a36 read txn/cursor stuff (Howard Chu)
86a7f2b core: check whether an update is needed straight away (moneromooo-monero)
ea5fa5e core: print "update needed" hard fork notifications in red (moneromooo-monero)
2016-02-29 17:29:26 -05:00
moneromooo-monero
c2a1fee428
simplewallet: prompt for private keys when generating wallets
instead of supplying them on the command line. It's more
unwieldy, but better protects against information leaks.
2016-02-27 11:53:11 +00:00
Howard Chu
7db89ed2ee ARMv7: fix unaligned accesses
And cleanup some key comparators
2016-02-25 13:57:00 +00:00
Howard Chu
5a07cefe7b Wrap some more actions in a larger read txn 2016-02-23 20:47:15 +00:00
Howard Chu
8cc7a36f0b read txn/cursor stuff
Could wrap more later.
2016-02-23 20:47:15 +00:00
moneromooo-monero
86a7f2b1e7 core: check whether an update is needed straight away 2016-02-23 20:29:17 +00:00
moneromooo-monero
ea5fa5e9b6 core: print "update needed" hard fork notifications in red 2016-02-23 20:29:17 +00:00
moneromooo-monero
283894d8ff
dns_utils: const compatibility with older libunbound
https://www.nlnetlabs.nl/bugs-script/show_bug.cgi?id=515
2016-02-23 19:02:05 +00:00
moneromooo-monero
4513b4cd2b
simplewallet: add a new --restore-from-keys option
It is similar in use to --restore-from-view-key, but also expects
a spend private key.

Requested by luigi1112, and useful to restore MyMonero wallets.
2016-02-22 22:10:55 +00:00
moneromooo-monero
5e3557d2c3
move g_test_dbg_lock_sleep from a global to a function level static
This avoids the need to define that variable in every program
which uses epee.
2016-02-22 19:34:09 +00:00
moneromooo-monero
0eb4d054f6
db_bdb: add missing include for std::condition_variable 2016-02-20 15:38:43 +00:00
Riccardo Spagni
81a93cd171
Merge pull request #680
360f72f simplewallet: wording change for single tx confirmation (moneromooo-monero)
2016-02-20 17:11:31 +02:00
Riccardo Spagni
c3baa82ac8
Merge pull request #678
11d555c Fix crash in std::map for connections_map (Howard Chu)
014f886 std::condvar is broken on Win32 with gcc/g++ 4.8 too (Howard Chu)
7c86c59 Use boost::thread instead of std::thread (Howard Chu)
2016-02-20 17:10:51 +02:00
Riccardo Spagni
a903d9c3f6
Merge pull request #677
57e75fa BlockchainBDB: Check if hard fork subdbs need reset (warptangent)
47f6cf8 BlockchainBDB: Support blockchain_import --drop-hard-fork command (warptangent)
2016-02-20 17:09:49 +02:00
moneromooo-monero
360f72fa90
simplewallet: wording change for single tx confirmation 2016-02-20 11:17:28 +00:00
Howard Chu
014f8868f2 std::condvar is broken on Win32 with gcc/g++ 4.8 too
Use boost...
2016-02-18 21:31:21 +00:00
Howard Chu
7c86c5997d Use boost::thread instead of std::thread
std::thread crashes on (at least) ARMv6 g++ 4.8/4.9
2016-02-18 21:30:10 +00:00
warptangent
57e75fa33f
BlockchainBDB: Check if hard fork subdbs need reset
See f7e337e625 for LMDB equivalent.
2016-02-18 06:38:56 -08:00
warptangent
47f6cf8bea
BlockchainBDB: Support blockchain_import --drop-hard-fork command
See c657e772c4 for LMDB equivalent.
2016-02-18 06:37:45 -08:00
warptangent
d258f6e63a
blockchain_import: Pass ARCH_WIDTH macro if 32-bit or not.
This also avoids warnings.
2016-02-18 06:14:35 -08:00
Riccardo Spagni
1889c0e81c
Merge pull request #675
27d4e50 core: check whether an update is needed straight away (moneromooo-monero)
8892173 core: print "update needed" hard fork notifications in red (moneromooo-monero)
2016-02-18 10:01:37 +02:00
Riccardo Spagni
43fe8c4933
Merge pull request #674
05e7ac0 blockchain_import: Check bit width for more than just WIN32 (warptangent)
2016-02-18 10:01:01 +02:00
moneromooo-monero
27d4e50dfa
core: check whether an update is needed straight away 2016-02-17 22:57:38 +00:00
moneromooo-monero
889217381f
core: print "update needed" hard fork notifications in red 2016-02-17 22:57:14 +00:00
Howard Chu
9218cad640 Fix cffc411c90
Don't include bdb header unless defined(BERKELEY_DB)
2016-02-17 20:41:34 +00:00
warptangent
05e7ac0360
blockchain_import: Check bit width for more than just WIN32
Pass the CMake bit width setting to compile flags for blockchain_import
and blockchain_converter.

For LMDB on 32-bit, hyc has found that batch size of 100 appears to be a
good default.
2016-02-17 12:37:15 -08:00
Howard Chu
02abe3590d Use MDB_PREV_MULTIPLE
in get_global_output_indices
2016-02-17 16:14:50 +00:00
Howard Chu
7a4755d3a4 Fixup after lmdb master resync 2016-02-17 16:14:38 +00:00
Riccardo Spagni
c069814773
Merge pull request #672
3b13a74 Shutup about VERSION 0 (Howard Chu)
1537477 Use cursor in get_output_key (Howard Chu)
f2faf8c Use MDB_APPEND mode where possible (Howard Chu)
090b548 Use cursors in write txns (Howard Chu)
ed08d21 Keep a running blocksize count (Howard Chu)
0fc9334 Win32 import batchsize tweaks (Howard Chu)
2016-02-17 17:10:07 +02:00
Riccardo Spagni
6636b0a6da
Merge pull request #671
8f09b71 wallet: clear missing data on rescan_bc (moneromooo-monero)
2016-02-17 17:09:48 +02:00
Riccardo Spagni
3225a7d28d
Merge pull request #666
27f76e2 blockchain_import: Build string for db type list (warptangent)
1aa8a9d blockchain_import: Add mode argument representing multiple DB flags (warptangent)
cffc411 blockchain_import: Support BerkeleyDB (warptangent)
19c1aaa blockchain_import: Add database type argument (warptangent)
0fedce0 blockchain_import: Make LMDB-specific names general (warptangent)
2016-02-17 17:09:21 +02:00
Howard Chu
3b13a7473e Shutup about VERSION 0 2016-02-17 05:06:19 +00:00
Howard Chu
1537477c9f Use cursor in get_output_key 2016-02-17 04:05:29 +00:00
Howard Chu
f2faf8cdd9 Use MDB_APPEND mode where possible
When keys are contiguous and monotonically increasing, this gets
denser page utilization (doesn't leave padding in page splits).
Can't be used for keys that are inserted in random order (e.g. hashes)
In total this only saves around 1.5% of space compared to original
DB code. The previous patch accounted for 0.8% savings on its own;
the blocks tables just aren't that big.
2016-02-17 04:05:29 +00:00
Howard Chu
090b548c3b Use cursors in write txns
Saves a bit of seek overhead. LMDB frees them automatically
in txn_(commit|abort) so they need no cleanup.
2016-02-17 04:05:29 +00:00
Howard Chu
ed08d2152e Keep a running blocksize count
Used in batch size estimation, avoids rereading already processed
blocks during import
2016-02-17 04:05:28 +00:00
Howard Chu
0fc93345ea Win32 import batchsize tweaks
Reduce frequency of resizes: bump minimum increase from 128MB to 512MB
Use a bigger safety margin at small batch sizes
2016-02-17 04:05:28 +00:00
moneromooo-monero
8f09b71b9b
wallet: clear missing data on rescan_bc 2016-02-15 20:51:30 +00:00
warptangent
27f76e2ee4
blockchain_import: Build string for db type list 2016-02-14 16:25:35 -08:00
warptangent
1aa8a9d26f
blockchain_import: Add mode argument representing multiple DB flags
These modes match those optionally provided as part of the daemon's
--db-type argument.

Argument after the # is interpreted as a composite mode if there's only
one (no comma separated arguments).

Sample usage:

blockchain_import --database lmdb#fastest

blockchain_import --database berkeley#fastest

Multiple specific DB flags are still supported, e.g.

blockchain_import --database lmdb#nosync,nordahead

blockchain_import --database berkeley#txn_nosync
2016-02-14 16:25:35 -08:00
Howard Chu
d8f9bb380c Keep a running blocksize count
Used in batch size estimation, avoids rereading already processed
blocks during import
2016-02-15 00:22:09 +00:00
warptangent
cffc411c90
blockchain_import: Support BerkeleyDB 2016-02-14 16:02:38 -08:00
warptangent
19c1aaa9f6
blockchain_import: Add database type argument
Default to LMDB.
2016-02-14 16:02:37 -08:00
warptangent
0fedce00d5
blockchain_import: Make LMDB-specific names general
Also update terms to better distinguish blockchain management/engine
from database type.
2016-02-14 16:02:37 -08:00
Howard Chu
bf22109d16 Win32 import batchsize tweaks
Reduce frequency of resizes: bump minimum increase from 128MB to 512MB
Use a bigger safety margin at small batch sizes
2016-02-14 19:31:52 +00:00
Riccardo Spagni
63e2d4195b
Merge pull request #657
1995923 BlockchainLMDB: Deal with DB exceptions at block level with particularity (warptangent)
c16cc20 BlockchainLMDB: Add sanity check for inconsistent state (warptangent)
9118d0a BlockchainLMDB: Call destructor on allocated txn if setup fails (warptangent)
f5581c3 BlockchainLMDB: Replace remaining txn pointer NULLs with nullptr (warptangent)
2016-02-13 19:54:47 +02:00
Riccardo Spagni
354d478286
Merge pull request #656
9832d18 cmake: Include OpenSSL libraries in static linking (warptangent)
c2f2437 cmake: Remove unused variable (warptangent)
7205210 cmake: Fix unbound config compile settings (warptangent)
3e72d97 cmake: Fix overly broad use of dynamic link settings (warptangent)
2016-02-13 19:51:56 +02:00
Riccardo Spagni
999992a64b
Merge pull request #652
c7e6b77 crypto: only check MONERO_USE_SOFTWARE_AES once (moneromooo-monero)
74aef21 crypto: use software AES based on the MONERO_USE_SOFTWARE_AES env var (moneromooo-monero)
2016-02-13 19:51:30 +02:00
warptangent
1995923559
BlockchainLMDB: Deal with DB exceptions at block level with particularity
Add another DB error exception type to distinguish failed txn setup from
general use of txn.

This keeps the error handling flow the same as before the block-level
txn setup changes that moved control up a layer to BlockchainDB.
2016-02-13 05:12:39 -08:00
warptangent
c16cc204d3
BlockchainLMDB: Add sanity check for inconsistent state
This hasn't been known to occur in block-level txn abort, but throw
exception if it does.
2016-02-13 04:17:37 -08:00
warptangent
9118d0a44c
BlockchainLMDB: Call destructor on allocated txn if setup fails 2016-02-13 04:17:37 -08:00
warptangent
f5581c3536
BlockchainLMDB: Replace remaining txn pointer NULLs with nullptr
For consistency.
2016-02-13 04:17:36 -08:00
warptangent
3e72d97ca7
cmake: Fix overly broad use of dynamic link settings
Remove LINK_SEARCH_START_STATIC and LINK_SEARCH_END_STATIC. This is more
appropriate when the compiler flag -static is used.

This had been causing CMake to omit the linker flags necesssary to
distinguish static and dynamic library linking. CMake had assumed static
linking for the target, causing it to omit explicit static link flags.
That is problematic without the -static compile flag being set.

With a library located in system directories, like libboost_date_time,
the full static path (.a), though found correctly by CMake, was treated
by the linker as a dynamic library. This is because
target_link_libraries() transforms the full path to -l<libname> if it's
in a system directory.  Without -static or explicit linker flags, the
dynamic library (.so) is linked.

Removing the above two properties removes the assumption of static. So
-Wl;-Bstatic is inserted where needed. This causes -l<libname> to
properly refer to the static library instead of dynamic.
2016-02-12 14:49:43 -08:00
moneromooo-monero
c7e6b77395
crypto: only check MONERO_USE_SOFTWARE_AES once 2016-02-09 09:38:17 +00:00
moneromooo-monero
74aef213fe
crypto: use software AES based on the MONERO_USE_SOFTWARE_AES env var
Setting to no or 0 also works. If set, any other value enables it.

Useful for running with valgrind in cases where it fails at
properly implementing AES-NI.
2016-02-09 09:30:34 +00:00
moneromooo-monero
759383c52d
hardfork: add a default fork entry for v1 if none exist
To avoid special cases
2016-02-08 20:58:06 +00:00
moneromooo-monero
c7f82ec769
blockchain: initialize m_hardfork to NULL
It can now be set by some other code, and is thus tested
2016-02-08 20:57:20 +00:00
moneromooo-monero
8649b9f1ef
blockchain_db: pass hard fork object as a simple pointer 2016-02-08 20:56:31 +00:00
moneromooo-monero
a333c42cde
core_tests: add tests for hard fork behaviors (MRL-0004)
We also replace the --fakechain option with an optional structure
containing details about configuration for the core/blockchain,
for test purposes. This seems more future friendly.
2016-02-08 18:52:00 +00:00
moneromooo-monero
9ef57946a0
blockchain_utilities: remove unused num_blocks variable 2016-02-08 18:26:03 +00:00
moneromooo-monero
6c8e5c3459
blockchain: reset hardfork object when resetting blockchain
Not doing so will prevent the new genesis block from being
reset if a switch past v1 had occured already.
2016-02-08 18:22:03 +00:00
warptangent
e298b14a55
Blockchain: Update comments on removing block 2016-02-08 09:28:25 -08:00
warptangent
b368e29f67
blockchain_import: Get hard fork version from HardFork
Replace temporary assignment that used hardcoded values.
2016-02-08 09:28:21 -08:00
warptangent
e02577f594
Move HardFork DB update to BlockchainDB::add_block()
Ensures the database is consistent.

Also simplifes blockchain_import in that verify mode off has less to
work around.
2016-02-08 09:28:19 -08:00
warptangent
3800875406
Make HardFork object available to BlockchainDB and derived DB implementations
This will later allow the HardFork object's DB update functions to be
called when the DB transaction that persists across block add/remove is
open.
2016-02-08 09:28:17 -08:00
warptangent
fd46c96dce
BlockchainDB/LMDB: Refactor block-scope DB txn handling for add block
Move block-scope txn start and stop from BlockchainLMDB to BlockchainDB.
2016-02-08 09:28:15 -08:00
warptangent
f3a6000094
BlockchainDB/LMDB/BDB: Extract DB txn functions for block add/remove 2016-02-08 09:28:14 -08:00
warptangent
f47d5b0fe3
BlockchainLMDB: Allow two HardFork functions to update DB during block add
Note that this doesn't yet cause them to be called during block add.
2016-02-08 09:28:12 -08:00
warptangent
15ee0bef4b
BlockchainLMDB: extract txn macros used during block add/remove 2016-02-08 09:28:10 -08:00
warptangent
9d62c161a3
blockchain_import: Add hard fork data for each block when verify mode is off 2016-02-08 08:50:59 -08:00
warptangent
7006b2ebe1
blockchain_utilities: Have fake_core create HardFork object
The HardFork object is needed for the importer to update the hard fork
data when verify mode is off.
2016-02-08 08:50:53 -08:00
warptangent
8f863e742d
Blockchain: Optionally pass in HardFork object 2016-02-08 08:50:51 -08:00
warptangent
f7e337e625
fake_core: Check if hard fork subdbs need reset at start 2016-02-08 08:50:48 -08:00
warptangent
c657e772c4
blockchain_import: Add --drop-hard-fork command 2016-02-08 08:50:47 -08:00
warptangent
412243901d
blockchain_import: Add exception to log output 2016-02-08 08:50:27 -08:00
Riccardo Spagni
1cfe0bdbef
Merge pull request #648
f6abc53 simplewallet: check watch-only view key matches standard address (moneromooo-monero)
2016-02-08 18:34:40 +02:00
Riccardo Spagni
0c4222158f
Merge pull request #646
c7b96b9 wallet: check a key image isn't already present when adding one (moneromooo-monero)
2016-02-08 18:34:13 +02:00
Riccardo Spagni
a1a5b810bc
Merge pull request #645
d9f4b39 daemon: remove leftover debug log (moneromooo-monero)
2016-02-08 18:33:30 +02:00
Riccardo Spagni
de78962e1c
Merge pull request #643
05bfb51 core: move the db lock to the data directory (moneromooo-monero)
2016-02-08 18:32:08 +02:00
Riccardo Spagni
cea6f6fb9c
Merge pull request #641
7658ac0 blockchain: revert handle_get_objects adding block id on tx not found (moneromooo-monero)
3a0f4d8 berkeleydb: fix delete/free mismatch (moneromooo-monero)
1642be2 minor bugfixes and refactoring (Thomas Winget)
098dcf2 unit_tests: fix mnemonics unit test testing invalid seeds (moneromooo-monero)
2016-02-08 18:31:42 +02:00
moneromooo-monero
f6abc5324f
simplewallet: check watch-only view key matches standard address 2016-02-05 22:41:45 +00:00
moneromooo-monero
d9f4b395e8
daemon: remove leftover debug log 2016-02-05 16:36:28 +00:00
moneromooo-monero
c7b96b91ed
wallet: check a key image isn't already present when adding one
If it is, it points to reuse of a tx key, which isn't meant to happen.
If it does, a key image collision means that only one of those
outputs is spendable, so the wallet selects the larger amount,
unless that output was spent already.

This causes a discrepancy betewen reported received inputs and
payment total.

Since tx keys are 256 bits, this should never happen except if
done on purpose, or if a sender uses a bad PRNG.
2016-02-05 16:24:23 +00:00
moneromooo-monero
05bfb51d3e
core: move the db lock to the data directory
Locking just one db turns out to not have been a good idea, since
the pool and p2p state fdles have to be used anyway.

Also ensure the directory exists before tring to lock.
2016-02-04 18:09:45 +00:00
moneromooo-monero
7658ac0f45
blockchain: revert handle_get_objects adding block id on tx not found
This differs from the original CN code, and there seems to be
no reason to include the block itself, if it was found
2016-02-03 21:08:03 +00:00
moneromooo-monero
3a0f4d8a57
berkeleydb: fix delete/free mismatch
Despite being C++, the stats object is allocated by the underlying
C layer using malloc(3).
2016-02-03 08:49:11 +00:00