Commit graph

1795 commits

Author SHA1 Message Date
Howard Chu 118dd69dd5 Use DUPFIXED for block_info and output_txs
Saves another ~150MB or so on the full blockchain
2016-04-05 20:55:16 +01:00
Howard Chu 6225716f3c More outputs consolidation
Also bumped DB VERSION to 1
Another significant speedup and space savings:
Get rid of global_output_indices, remove indirection from output to keys

This is the change warptangent described on irc but never got to finish.
2016-04-05 20:55:12 +01:00
Howard Chu 7c5abdc3a3 Use DUPFIXED for output_keys
Saves another 90MB on 200000 block import.
Had to bring back compare_uint64 for this, but it's safe since
this table is always 64-bit aligned.
2016-04-05 20:54:42 +01:00
Howard Chu 8e9d8e3364 Use DUPFIXED for tx_indices
Small space savings, no measurable speedup
2016-04-05 20:54:39 +01:00
Howard Chu a12f9365f8 Use DUPFIXED for block_heights
Only a small savings...
2016-04-05 20:54:07 +01:00
Howard Chu 38c2277d6f Use DUPFIXED for spent_keys 2016-04-05 20:54:07 +01:00
warptangent bf769c32ba Add back changes from revert.
m_tx_outputs doesn't need to be changed, as it's no longer dup list.
2016-04-05 20:54:07 +01:00
warptangent 46b991b362 Use MDB_APPEND mode with two tx subdbs
This is possible on those using a tx index as a key.
2016-04-05 20:54:07 +01:00
warptangent 9aadedb1d0 Schema update: tx_indices - consolidate the tx subdbs from 5 to 3 2016-04-05 20:54:06 +01:00
warptangent a2f518aa01 Schema update: tx_indices - yet less indirection 2016-04-05 20:54:06 +01:00
warptangent 8d12a8df2c Schema update: tx_indices - improve further with less indirection 2016-04-05 20:54:06 +01:00
warptangent 7c013f66e9 Add batch warning for further review 2016-04-05 20:54:06 +01:00
warptangent ae0854a431 Schema update: tx_indices 2016-04-05 20:54:06 +01:00
Howard Chu 8d252a4214 Consolidated block info 2016-04-05 20:53:59 +01:00
warptangent 132c666f67 Update schema for "tx_outputs" to use array containing amount output indices
This speeds up wallet refresh by directly retrieving a tx's amount output indices.

It removes the indirection and walking the amount output duplicate list
for every amount in each requested tx.

"tx_outputs" is used by:
Amount output indices are needed for wallet refresh.
Global output indices are needed for removing a tx.

Both amount output indices and global output indices are now stored in
an array of 64-bit unsigned ints:

tx_outputs[<tx_hash>] -> [ <a1_oi, a1_gi, a2_oi, a2_gi, ...> ]

Previously it was:
tx_outputs[<tx_hash>] -> duplicate list of <a1_gi, a2_gi, a3_gi, ...>

The amount output list had to be walked for every amount in order to
find each amount's output index, by comparing the amount's global output
index with each one in the duplicate list until a match was found.

See also d045dfa7ce
2016-04-05 20:30:50 +01:00
warptangent 309f8f3d44 blockchain_utilities: Update documentation 2016-04-05 20:30:50 +01:00
luigi1111 7ac1db42c2 get_payments short ID
Add support for short/integrated/encrypted IDs to get_payments RPC
2016-04-05 20:30:50 +01:00
Riccardo Spagni 38fed83b65
Merge pull request #703
dfca2ae unit_tests: fix hard fork unit test compilation (warptangent)
2016-03-05 23:12:32 +02:00
Riccardo Spagni 93605eb05f
Merge pull request #702
0485d17 blockchain_export: Support BerkeleyDB (warptangent)
41487e3 blockchain_export: Move DB implementation includes (warptangent)
2016-03-05 23:12:05 +02:00
Riccardo Spagni 32dc08d19a
Merge pull request #700
bdec7cb BlockchainLMDB: Use DB error helper consistently (warptangent)
c5932eb BlockchainLMDB: Add DB error to exception (warptangent)
a49c355 Blockchain: Omit verbose time stats messages by default (warptangent)
2016-03-05 23:10:38 +02:00
Riccardo Spagni c3af15702f
Merge pull request #698
ee7a8b8 Get rid of lmdb_cur (Howard Chu)
2016-03-05 23:06:26 +02:00
Riccardo Spagni 14c1e214ad
Merge pull request #696
d57ec75 Fix language (Howard Chu)
2016-03-05 23:06:04 +02:00
Riccardo Spagni 80aa57a6cd
Merge pull request #695
f5affbe More for df239428c0 (Howard Chu)
2016-03-05 23:05:29 +02:00
warptangent dfca2ae1b5
unit_tests: fix hard fork unit test compilation
Match changed BlockchainDB function declaration.
2016-03-05 10:01:10 -08:00
warptangent 0485d17274
blockchain_export: Support BerkeleyDB
TEST:

blockchain_export -h

This should show "berkeley" as an available option to --database.

With an existing BerkeleyDB database, run:

blockchain_export --database berkeley
2016-03-05 06:46:11 -08:00
warptangent 41487e32be
blockchain_export: Move DB implementation includes 2016-03-05 06:46:09 -08:00
warptangent bdec7cbfb3
BlockchainLMDB: Use DB error helper consistently 2016-03-03 21:27:13 -08:00
warptangent c5932eb5ea BlockchainLMDB: Add DB error to exception 2016-03-03 20:15:37 -08:00
warptangent a49c355cae
Blockchain: Omit verbose time stats messages by default
This is already the default for the daemon, but by checking a command
line argument and calling a Blockchain member function setter.

Initialize the variable to false so it's not dependent on an external
command-line argument check. This allows utilities like
blockchain_import to have a reasonable default without code changes.
2016-03-03 20:14:48 -08:00
Howard Chu ee7a8b87f9 Get rid of lmdb_cur
We don't need it now with per-txn cursors.
2016-03-03 19:06:27 +00:00
Howard Chu d57ec7595c Fix language 2016-03-03 00:34:19 +00:00
Howard Chu f5affbef26 More for df239428c0
Let ARMv7 work again
2016-03-03 00:13:13 +00:00
Riccardo Spagni a7310031f4
Merge pull request #693
2c823c1 Make partial revert consistent (warptangent)
2016-03-01 07:40:00 -05:00
warptangent 2c823c1589
Make partial revert consistent
If user-defined comparator is used, subdb shouldn't be opened with
MDB_INTEGERKEY.

TODO: Again, this will be added back with future schema updates.
2016-03-01 02:47:16 -08:00
Riccardo Spagni e0493e8429
Merge pull request #692
df23942 Revert part of 7db89ed2ee (warptangent)
c9c4060 BlockchainLMDB: Add height and db error to exception (warptangent)
2016-02-29 21:09:51 -05:00
warptangent df239428c0
Revert part of 7db89ed2ee
For now, so existing databases work.

TODO: add these back with future schema updates.
2016-02-29 18:03:41 -08:00
warptangent c9c4060dc6
BlockchainLMDB: Add height and db error to exception 2016-02-29 17:52:54 -08:00
Riccardo Spagni 95b3b09e11
Merge pull request #689
5ab33ca unit_tests: update test openalias address (moneromooo-monero)
2016-02-29 17:31:07 -05:00
Riccardo Spagni bf09285443
Merge pull request #688
c2a1fee simplewallet: prompt for private keys when generating wallets (moneromooo-monero)
4513b4c simplewallet: add a new --restore-from-keys option (moneromooo-monero)
2016-02-29 17:30:29 -05:00
Riccardo Spagni c3c4e28061
Merge pull request #687
5e3557d move g_test_dbg_lock_sleep from a global to a function level static (moneromooo-monero)
2016-02-29 17:30:10 -05:00
Riccardo Spagni d60bf4ee36
Merge pull request #686
7db89ed ARMv7: fix unaligned accesses (Howard Chu)
5a07cef Wrap some more actions in a larger read txn (Howard Chu)
8cc7a36 read txn/cursor stuff (Howard Chu)
86a7f2b core: check whether an update is needed straight away (moneromooo-monero)
ea5fa5e core: print "update needed" hard fork notifications in red (moneromooo-monero)
2016-02-29 17:29:26 -05:00
Riccardo Spagni 838f7aba7d
Merge pull request #684
283894d dns_utils: const compatibility with older libunbound (moneromooo-monero)
2016-02-29 17:28:48 -05:00
moneromooo-monero c2a1fee428
simplewallet: prompt for private keys when generating wallets
instead of supplying them on the command line. It's more
unwieldy, but better protects against information leaks.
2016-02-27 11:53:11 +00:00
Howard Chu 7db89ed2ee ARMv7: fix unaligned accesses
And cleanup some key comparators
2016-02-25 13:57:00 +00:00
moneromooo-monero 5ab33cabf6
unit_tests: update test openalias address
It was recently changed
2016-02-23 23:12:55 +00:00
Howard Chu 5a07cefe7b Wrap some more actions in a larger read txn 2016-02-23 20:47:15 +00:00
Howard Chu 8cc7a36f0b read txn/cursor stuff
Could wrap more later.
2016-02-23 20:47:15 +00:00
moneromooo-monero 86a7f2b1e7 core: check whether an update is needed straight away 2016-02-23 20:29:17 +00:00
moneromooo-monero ea5fa5e9b6 core: print "update needed" hard fork notifications in red 2016-02-23 20:29:17 +00:00
moneromooo-monero 283894d8ff
dns_utils: const compatibility with older libunbound
https://www.nlnetlabs.nl/bugs-script/show_bug.cgi?id=515
2016-02-23 19:02:05 +00:00