Commit graph

537 commits

Author SHA1 Message Date
Thomas Winget
7b14d4a17f
Steps toward multiple dbs available -- working
There will need to be some more refactoring for these changes to be
considered complete/correct, but for now it's working.

new daemon cli argument "--db-type", works for LMDB and BerkeleyDB.

A good deal of refactoring is also present in this commit, namely
Blockchain no longer instantiates BlockchainDB, but rather is passed a
pointer to an already-instantiated BlockchainDB on init().
2015-03-25 12:09:44 -04:00
Thomas Winget
874f48bc82
update berkeleydb branch to blockchain branch 2015-03-25 07:30:23 -04:00
Thomas Winget
5c0bc0050c
Merge upstream updates into blockchain branch 2015-03-25 05:56:36 -04:00
Thomas Winget
8855a32044
Merge upstream to daemonize changes
Preparation for PR
2015-03-24 02:47:15 -04:00
warptangent
dbdcf11778
blockchain_converter: Add support for resume from last block
Add option "--resume <on|off>" where default is on.
2015-03-23 18:52:55 -07:00
warptangent
488080326c
blockchain_import: lengthen string for line clear 2015-03-22 15:45:37 -07:00
warptangent
7476d2e253
blockchain_export: show progress during export 2015-03-22 15:45:37 -07:00
warptangent
4bedd68d2c
Update Blockchain::get_db() to return reference instead of pointer
Where this method is used, a BlockchainDB object is always expected, so
a pointer is unnecessary and less safe.
2015-03-22 15:45:36 -07:00
warptangent
0386e9925b
Add README for blockchain converter, importer, and exporter utilities 2015-03-22 15:45:25 -07:00
Thomas Winget
f78bb00943
Hopefully fixes build on Windows for real this time 2015-03-20 16:56:55 -04:00
Thomas Winget
2b0583b2c6
Hopefully fixes build on Windows 2015-03-20 16:40:54 -04:00
Thomas Winget
ead7fad552
BerkeleyDB implementation of BlockchainDB seems to be working! 2015-03-17 22:12:09 -04:00
Thomas Winget
8e3347f310
Pull blockchain changes into berkeleydb branch 2015-03-17 19:52:53 -04:00
Thomas Winget
e146027acd
BlockchainBDB passes unit tests 2015-03-17 17:18:45 -04:00
warptangent
ffadb6571a
blockchain_export: Add compile-time support for BlockchainDB
This allows an LMDB database to be used as the blockchain to export.

Adjust SOURCE_DB in src/blockchain_converter/blockchain_export.h
depending on needs. Defaults to DB_MEMORY.

DB_MEMORY is a sensible default for users migrating to LMDB, as it
allows the exporter to use the in-memory blockchain while the other
binaries work with LMDB, without recompiling anything.
2015-03-16 11:57:26 -07:00
Thomas Winget
43477b7dac
BerkeleyDB Blockchain building, not working yet
Everything except actually *using* BlockchainBDB is wired up, but the db
itself is not yet working.  Some error about user mem not large enough.
I think I know what this error means, but I can't determine the cause.

Notes: BerkeleyDB does not allow 0-indexing in its recno type databases,
  so block numbers *in the database* will be 1-indexed.  Modifications
  to indexing have been made as needed.
2015-03-16 09:14:51 -04:00
Thomas Winget
cade0da8f1
CMake wiring, minor cleanup, minor test addition
Make Cmake things aware of BerkeleyDB and BlockchainBDB

Make the BlockchainDB unit tests aware of BlockchainBDB
2015-03-16 04:17:53 -04:00
Thomas Winget
1bc89398b4
BerkeleyDB BlockchainDB impl copy/paste/modify
LMDB implementation code copy/paste/modified into the Berkeley DB
implementation.  Need to test if it builds, then if it works, and so on,
but the code is all there.
2015-03-16 04:17:48 -04:00
Thomas Winget
5112dc37d7
Try to not pollute cryptonote namespace 2015-03-16 04:17:44 -04:00
Thomas Winget
edef0bb771
Initial commit of BDB BlockchainDB implementation
Basically verbatim copy of LMDB implementation, but with the guts ripped
out and includes changed, etc.
2015-03-16 04:17:32 -04:00
warptangent
f6cbfb623b
Add blockchain_export utility
Based on work by tomerkon.
See https://github.com/tomerkon/bitmonero
  src/cryptonote_core/bootfilesaver.{h,cpp}
  src/bootfilegen/bootfilegen.cpp
2015-03-16 00:26:59 -07:00
warptangent
275cbd4348
Add support for database open with flags
Add support to:
  - BlockchainDB, BlockchainLMDB
  - blockchain_import utility to open LMDB database with one or more
    LMDB flags.

Sample use:
  $ blockchain_import --database lmdb#nosync
  $ blockchain_import --database lmdb#nosync,nometasync
2015-03-16 00:26:59 -07:00
warptangent
cb862cb81a
Add mdb_flags variable to LMDB database open 2015-03-16 00:26:58 -07:00
warptangent
260cc56fae
Add blockchain_import utility
This imports to the blockchain database from an exported blockchain
file.

It can be used to bootstrap a new database or to add blocks to an
existing one.

Supports:
  - both the in-memory and LMDB implementations
  - optional: batching, verification, testnet

See help for usage.

Based on work by tomerkon.
See https://github.com/tomerkon
  src/cryptonote_core/bootfileloader.{h,cpp}
2015-03-16 00:26:51 -07:00
warptangent
ca75b4789c
Blockchain: add get_db() accessor, needed for blockchain_import
This handling may be changed in the future.
2015-03-15 13:22:52 -07:00
warptangent
a3dd9d10f3
blockchain_converter: Add support for batch transactions
Add log level support.

Add testnet support.

Add command-line options:
  --help
  --data-dir
  --testnet-data-dir
  --testnet
  --log-level
  --batch
  --batch-size
  --block-number

See help for usage. Run at log level 1 to see profiling stats.
2015-03-15 13:22:47 -07:00
warptangent
acb5d291b8
Update and relocate comment that applies class wide 2015-03-15 13:22:32 -07:00
Thomas Winget
eee3ee7073
BlockchainDB implementations have names now
In order to make things more general, BlockchainDB now has get_db_name()
which should return a string with the "name" of that type of db.
This "name" will be the subfolder name that holds that db type's files
within the monero folder.

Small bugfix: blockchain_converter was not correctly appending this in
the prior hard-coded-string implementation of the subfolder data
directory concept.
2015-03-13 21:39:27 -04:00
Thomas Winget
5eab480cb1
Moved BlockchainDB into its own src/ subfolder
Ostensibly janitorial work, but should be more relevant later down the
line.  Things that depend on core cryptonote things (i.e.
    cryptonote_core) don't necessarily depend on BlockchainDB and thus
have no need to have BlockchainDB baked in with them.
2015-03-06 15:20:45 -05:00
smooth
7e72e943db update checkpoint 2015-03-06 02:38:06 -08:00
smooth
41a95e7b11 add comment about avoiding overflow 2015-03-05 00:45:54 -08:00
smooth
754a785ee1 minimum subsidy for mining incentives, remove unused LEGACY_FEE define 2015-03-04 23:35:04 -08:00
Riccardo Spagni
c01069f352
fixed English word list issue: 'launchpad' should be 'ourselves' 2015-03-04 15:31:24 +02:00
Riccardo Spagni
10e4132e22
fixed English word list issue: 'incline' should have remained 'inline' 2015-03-04 15:29:20 +02:00
Thomas Winget
9dab105e2e
DNS checkpoint loading for testnet should now be correct 2015-03-01 03:36:46 -05:00
Thomas Winget
52f9629bd7
sending commands to forked daemon works on testnet now 2015-03-01 02:35:33 -05:00
Thomas Winget
76289d0e3b
Fix tests building -- function signatures changed 2015-03-01 02:21:19 -05:00
Thomas Winget
db53e1956a
revert stop_daemon method to use correct exit
This was changed because sometimes the daemon does not complete its exit
routine with this method, but as it does correctly wind most things down
even if it gets stuck I've changed it back.
2015-02-24 00:54:50 -05:00
Thomas Winget
96cbecffd7
RPC calls for background daemon added in
The RPC calls the daemon executable uses to talk to the running daemon
instance have mostly been added back in.  Rate limiting has not been
added in upstream, but is on its way in a separate effort, so those
calls are still NOPed out.
2015-02-24 00:44:36 -05:00
Thomas Winget
9193d6fb5b
Daemonize changes pulled in -- daemon builds
many RPC functions added by the daemonize changes
(and related changes on the upstream dev branch that were not merged)
were commented out (apart from return).  Other than that, this *should*
work...at any rate, it builds, and that's something.
2015-02-24 00:05:19 -05:00
Thomas Winget
0ad0784f46
Changed log level of debug message -- too spammy 2015-02-23 18:28:20 -05:00
warptangent
6485dacc2f
BlockchainLMDB: Add profiling to tx_exists() 2015-02-23 00:33:39 -08:00
warptangent
83fb6d8d07
BlockchainLMDB: Add batch transaction support to tx_exists() 2015-02-23 00:33:38 -08:00
warptangent
8529c0ea9a
BlockchainDB, BlockchainLMDB: Add profiling for DB commits 2015-02-23 00:33:38 -08:00
warptangent
7a66b8bbcf
BlockchainDB: Add virtual function declarations for batch transactions 2015-02-23 00:33:38 -08:00
warptangent
b7a2d84919
BlockchainLMDB: Add check for open database to two functions 2015-02-23 00:33:38 -08:00
warptangent
58ecc58be1
BlockchainLMDB: Add support for batch transactions 2015-02-23 00:33:37 -08:00
warptangent
8909d7d82e
Improve block and tx processing efficiency by less repeat hashing
BlockchainLMDB::add_block()
  BlockchainLMDB::add_transaction_data()

  BlockchainDB::add_transaction()
2015-02-23 00:33:37 -08:00
warptangent
3676ac5841
Add profiling to block and tx processing 2015-02-23 00:33:37 -08:00
warptangent
ce71abd0fe
Move LMDB storage to subfolder 2015-02-23 00:33:37 -08:00