blitzwing/cryptpad
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
10899 commits 72 branches 132 tags 541 MiB
Commit graph

386 commits

Author SHA1 Message Date
ansuz
90899aa2f4 start reviewing and removing XXX notes 2020-03-24 13:03:49 -04:00
yflory
fe14399e67 Merge branch 'whiteboard' into soon 2020-03-24 16:15:13 +01:00
ansuz
2ef0af29e5 guard against index worker rpc responses for a txid with no supplied callback 2020-03-23 22:03:59 -04:00
ansuz
ee8d5c9c36 fix the premiumUploadSize validation to check the correct attribute 2020-03-23 16:13:48 -04:00
yflory
dc0b26da0b Merge branch 'soon' into staging 2020-03-23 12:33:17 +01:00
Alexander
a190af079d
Add shutdown() to non-disk logger 
The script `evict-inactive.js` calls `Log.shutdown()` at the end, which currently fails if logging to disk has been disabled via `logPath`.
 2020-03-21 14:43:25 +01:00
ansuz
ed63d98ab0 Merge branch 'scaling-index' into soon 2020-03-20 18:57:51 -04:00
ansuz
590e249500 resolve merge conflict 2020-03-20 13:07:35 -04:00
ansuz
31c7cecaf9 add a few checks for safety 2020-03-20 10:54:01 -04:00
ansuz
cea9705bbe Merge branch 'aggressive-stream-closing' into staging 2020-03-20 10:50:17 -04:00
ansuz
faa133aab8 remove an XXX 2020-03-20 10:22:48 -04:00
ansuz
d386e223e4 simplify open/close of writeStreams 2020-03-19 17:33:22 -04:00
ansuz
32cd0f3c4d increase timeout value from 15s to 45. ought to be enough for anybody 2020-03-19 16:13:24 -04:00
ansuz
05a4e86cdb close streams whenever we finish using them. time out if necessary 2020-03-19 16:11:24 -04:00
ansuz
4522ffa18a compute indexes in child processes 2020-03-19 10:46:18 -04:00
ansuz
f8ad649b45 [style] bail out early to avoid nesting 2020-03-18 10:30:42 -04:00
ansuz
019e5e708b wrap workers in a function scope and add a validateMessage method to HK's Env 2020-03-17 17:23:35 -04:00
ansuz
5467e1ffac replace ad-hoc response handler with Util.response 2020-03-17 16:52:41 -04:00
ansuz
9e85a1411e abstract the logic around worker choice out of message handling 2020-03-17 16:06:01 -04:00
ansuz
631ea54b49 lint compliance 2020-03-17 15:40:06 -04:00
ansuz
67dd4a608c Merge remote-tracking branch 'origin/scaling' into scaling2 2020-03-17 15:36:58 -04:00
ansuz
b2fcde87d8 implement an admin command to update the default storage limit without a restart 2020-03-17 10:11:52 -04:00
yflory
923616aef0 Use each process in order instead of using a random one 2020-03-17 15:10:23 +01:00
ansuz
f345998956 fix undefined reference 2020-03-17 10:10:20 -04:00
yflory
fb0eb1b20c Use more subprocesses 2020-03-17 14:18:41 +01:00
yflory
0d636dabc9 Check signature for history keeper in a different process 2020-03-17 13:29:53 +01:00
ansuz
30f17040ac close streams when possible, simplify some code, enforce asynchrony 2020-03-16 16:13:38 -04:00
ansuz
2a9780abcf lint compliance 2020-03-16 12:29:26 -04:00
yflory
578042154e Better reconnect after invalid lastKnownHash 2020-03-13 17:39:16 +01:00
ansuz
2d47e7e2cf implement an admin rpc to fetch ulimit data 2020-03-12 12:09:44 -04:00
ansuz
81c7416095 lint compliance 2020-03-12 11:40:46 -04:00
ansuz
f644c24362 add an admin-rpc to count the number of open files 2020-03-12 11:35:16 -04:00
ansuz
1230057970 disable IS_PAD_PINNED 2020-03-12 11:31:08 -04:00
ansuz
e1a0daac9c possible server fixes 2020-03-12 11:23:43 -04:00
ansuz
bc034d95a3 enforce asynchrony in 'gethistoryOffset' 2020-03-12 11:13:23 -04:00
ansuz
d4038344ba stop iterating over tasks if you can't read the root directory 2020-03-11 16:34:00 -04:00
ansuz
4e8ebac3c1 guard against the possible non-existence of metadata for a channel 2020-03-09 17:42:10 -04:00
ansuz
b32f3e34af fix local worker debugging on firefox 2020-03-06 11:43:34 -05:00
ansuz
6d8cdca7a3 add 'rejected' field to metadata when getMetadata requests are blocked 2020-03-05 12:09:18 -05:00
ansuz
9d8bb43d03 finally untangle metadata and index caches 2020-03-04 12:56:33 -05:00
ansuz
32d769447a begin standardizing our method of streaming lines from files 2020-03-04 11:38:14 -05:00
ansuz
35eca2c5d2 pull file streaming out into its own file, leave a few notes 2020-03-04 11:38:14 -05:00
ansuz
ce6cf6f727 revert some changes that couldn't have possibly worked 2020-03-04 09:59:17 -05:00
ansuz
b126e4456b simplify quota logic 2020-03-03 16:55:40 -05:00
ansuz
53ed247bc2 expose maxUploadSize and premiumUploadSize via /api/config 2020-03-03 16:47:44 -05:00
ansuz
170aa6d47e clean up a few tasks related to allow lists 2020-03-03 15:52:49 -05:00
ansuz
2f00ff3278 simplify check-in deactivation 2020-02-28 14:24:53 -05:00
ansuz
03ff9bd0d7 start deprecating usage of myDomain 2020-02-28 11:45:41 -05:00
ansuz
de6b9e2228 apply some defaults when loading the config 2020-02-28 10:50:30 -05:00
ansuz
94d8e7f2c2 provide a consistent type for 'customLimits' 2020-02-28 10:49:20 -05:00
ansuz
3cf09924ae fix default CSP headers 2020-02-28 10:46:44 -05:00
ansuz
8d509fd6d5 Merge branch 'staging' into small-config 2020-02-27 15:25:39 -05:00
ansuz
c26560e3f9 commit stashed admin-rpc refactor 2020-02-27 15:00:31 -05:00
ansuz
02fc343727 Merge branch 'staging' into small-config 2020-02-27 13:54:15 -05:00
ansuz
3df47a1415 continued WIP config reorganization 2020-02-27 13:24:19 -05:00
ansuz
294a444603 WIP removing defaults from the example config file 2020-02-27 13:09:12 -05:00
ansuz
59ad80d7f1 support larger upload sizes for users with premium accounts 2020-02-26 13:09:11 -05:00
ansuz
3f2ebb773b merge master back to staging 2020-02-25 11:20:20 -05:00
ansuz
542150b775 merge server components from staging 2020-02-25 10:09:35 -05:00
ansuz
f951951077 merge communities-allow-list and lint compliance 2020-02-25 09:48:41 -05:00
ansuz
f0e5fc7614 merge some missing code from communities-allow-list 2020-02-25 09:09:16 -05:00
ansuz
ad08fe0c08 silence a few more routine websocket errors 2020-02-25 08:01:25 -05:00
ansuz
021a6b361f remove a bogus XXX note 2020-02-21 14:33:28 -05:00
ansuz
033bd361f9 add backwards compatibility with older servers 2020-02-21 13:58:21 -05:00
ansuz
cac114bb52 merge staging into pending allow-list work 2020-02-21 13:52:10 -05:00
ansuz
386add7c58 fix an invalid use of dropChannel 2020-02-21 13:46:11 -05:00
ansuz
33c12c27a1 ignore successive metadata errors with the same message 2020-02-21 13:38:56 -05:00
ansuz
7ee8778210 kick unauthorized users from restricted channels 
and update dependencies
 2020-02-21 13:05:24 -05:00
ansuz
fe0e02a421 restrict access to metadata 2020-02-21 10:31:10 -05:00
ansuz
791aad53f2 WIP allow list changes 2020-02-21 08:18:05 -05:00
ansuz
597f417ad6 WIP block history requests that bypass JOIN restriction (where appropriate) 2020-02-19 15:32:15 -05:00
ansuz
79bc8830ef disable access restriction if all owners are removed 2020-02-19 14:39:28 -05:00
ansuz
f579c9b059 semi-functional allow-list implementation in historyKeeper 2020-02-19 14:22:12 -05:00
ansuz
0edcdcb1b8 oups! fix a missing callback that's been here for a while... 2020-02-18 16:15:05 -05:00
ansuz
27e57e7af0 oups! fix a missing callback that's been here for a while... 2020-02-18 16:07:14 -05:00
ansuz
eac3e4cbcd sketch out some historyKeeper functionality related to access lists 2020-02-18 16:05:12 -05:00
ansuz
156c37942d implement new metadata commands related to allow lists and mailboxes 2020-02-18 14:16:07 -05:00
ansuz
a280cc85d2 Merge branch 'staging' into communities-allow-list 2020-02-17 12:04:41 -05:00
ansuz
626a031115 resolve conflict with master/staging 2020-02-17 12:04:01 -05:00
ansuz
ff73e96cb8 reimplement the trim history fix from staging 2020-02-17 12:02:49 -05:00
ansuz
a4be6185de merge staging and do a little lint compliance 2020-02-17 11:54:27 -05:00
ansuz
f478ae725d Merge branch 'staging' into communities-allow-list 2020-02-17 11:48:56 -05:00
ansuz
79f1280cf3 Merge branch 'soon' into staging 2020-02-17 11:42:08 -05:00
ansuz
b56367414b don't overwrite cached indices when new users join a channel 2020-02-17 11:31:00 -05:00
ansuz
253ea0d336 Merge branch 'staging' into communities-allow-list 2020-02-17 10:38:40 -05:00
ansuz
e8949168ec lint compliance 2020-02-17 10:30:44 -05:00
ansuz
5dff6535ed add a simple guard against unparsed messages when trimming history 2020-02-17 10:19:51 -05:00
ansuz
8694c17023 sketch out metadata commands for allow lists 2020-02-17 09:26:36 -05:00
ansuz
f86196e40a implement shared environment between historyKeeper and RPC 2020-02-17 09:01:10 -05:00
ansuz
3f606d8c75 remove some duplicated code 2020-02-14 16:45:53 -05:00
ansuz
725d10fc60 nest storage directory inside './lib' 2020-02-14 16:29:30 -05:00
ansuz
38c1700173 Respond to pinning RPCs as soon as possible 
(instead of waiting until you've read an unbounded number
of pin logs while queries back up in memory)

Also replace instances of 'publicKey' with 'safeKey' or 'unsafeKey'
to clearly and correctly indicate their format.
 2020-02-14 13:46:40 -05:00
ansuz
cded52f83f replicate existing pinned.load API correctly 2020-02-14 11:45:51 -05:00
ansuz
65f88617cf add support for an optional handler for each pin log 
great for analyzing which files are most in need of optimization
 2020-02-13 18:16:32 -05:00
ansuz
3dc789cbca replace pinned.js with low-profile streaming implementation 2020-02-13 17:46:10 -05:00
ansuz
609eddc9e1 reimplement the the Pinned.load API 
* recurse into directories on demand
* optimized for lower memory consumption
* less parallelism than before
 2020-02-13 17:41:36 -05:00
ansuz
2345323f0d lint compliance 2020-02-13 17:40:55 -05:00
ansuz
2d6626234b yet another async scheduler, this time supporting flexible runtime control flow based on declarative priority levels 2020-02-13 16:44:11 -05:00
yflory
0f697ac865 Merge branch 'staging' into communities-trim 2020-02-11 11:10:12 +01:00
ansuz
1231cacef1 remove XXX 2020-02-10 11:21:04 -05:00
ansuz
57be324850 remove some XXX notes 2020-02-10 10:01:02 -05:00
ansuz
68a77885d4 suppress some routine server logs 2020-02-10 09:03:16 -05:00
ansuz
7f22da0f57 fix use of incorrect key format in admin-rpc 2020-02-10 08:07:29 -05:00
ansuz
7000be0e70 clean up historykeeper 2020-02-07 18:58:57 -05:00
ansuz
802034616c centralize historykeeper-rpc interaction in rpc methods 2020-02-07 12:53:12 -05:00
yflory
fd095644a2 Merge branch 'staging' into communities-trim 2020-02-06 11:55:07 +01:00
yflory
64c0f3a90d Add missing RPC command 2020-02-06 11:54:46 +01:00
yflory
9a857ea058 Merge branch 'staging' into communities-trim 2020-02-06 10:55:12 +01:00
ansuz
b4172676f3 lint compliance 2020-02-05 18:12:09 -05:00
ansuz
f17d14fd99 simpler limit updates 2020-02-05 18:01:48 -05:00
ansuz
bde17a62a1 continue refactoring rpc 2020-02-05 17:31:44 -05:00
ansuz
fa525817ba fix broken removePins RPC 2020-02-05 13:05:06 -05:00
ansuz
a00a9fd018 don't hang when reading unparseable lines from history 2020-02-05 13:05:05 -05:00
yflory
1d56c08700 Merge branch 'staging' into communities-trim 2020-02-05 18:23:10 +01:00
ansuz
3741cbcbc0 rewrite trimmed history WITH newlines 2020-02-05 12:22:53 -05:00
yflory
4079f1bfdb Fix RPC command 2020-02-05 18:12:10 +01:00
yflory
6655f493e0 Merge branch 'staging' into communities-trim 2020-02-05 17:56:14 +01:00
ansuz
5808c534b5 fix function signature for trimHistory RPC 2020-02-05 11:53:47 -05:00
yflory
c9676e1c91 Merge branch 'staging' into communities-trim 2020-02-05 17:05:45 +01:00
ansuz
9c28941f6c lint compliance 2020-02-05 10:47:43 -05:00
yflory
5cb266838a Fix syntax error 2020-02-05 11:28:27 +01:00
ansuz
0d69620687 tweak the metadata line handler to handle an edge case in trim history 2020-02-04 17:33:51 -05:00
ansuz
f8f3a48e8b use latest chainpad-server 2020-02-04 11:15:44 -05:00
ansuz
46dfa026f0 fix an API change that caused a typeError 2020-02-03 18:47:18 -05:00
ansuz
88be40ede3 standardize some function signatures and factor out a lot of boilerplate 2020-02-03 18:32:21 -05:00
ansuz
65ba85d97b clear historyKeeper cache when we trim a channel 2020-02-03 17:14:42 -05:00
ansuz
43307ffb1a define all server intervals in a map so we can easily clear them all 2020-02-03 17:14:23 -05:00
ansuz
6523974ca2 fix a WRITE_PRIVATE_MESSAGE rpc regression 2020-02-03 15:47:41 -05:00
ansuz
779e817443 stop relying on netflux-server internals 
* create RPC module from inside historyKeeper
* stop passing around netflux-server context
  * update to use newer netflux-server's formal APIs
* manage your own cache of indexes instead of storing things in the netflux context
 2020-02-03 14:20:05 -05:00
ansuz
06c29ef1d1 latest api changes to match the netflux-server refactor 2020-02-03 10:03:43 -05:00
ansuz
b922860339 drop usage of historyKeeper.setConfig 2020-01-27 18:54:16 -05:00
ansuz
80c012f34d prepare to merge history keeper and rpc 2020-01-27 17:57:39 -05:00
ansuz
b093d3f0d2 WIP massive rpc refactor 2020-01-24 14:45:53 -05:00
ansuz
ceb351326c split out some more rpc functionality and fix broken module paths 2020-01-24 13:36:14 -05:00
ansuz
c1f222dd6c move metadata commands from rpc to their own module 2020-01-24 13:19:40 -05:00
ansuz
bb7e8e4512 move login block functionality into its own rpc module 2020-01-24 13:14:26 -05:00
ansuz
c765362744 move more rpc functionality into modules 2020-01-24 13:06:46 -05:00
ansuz
c93b39c094 separate more rpc functionality into pinning and core submodules 2020-01-24 12:43:11 -05:00
ansuz
ba6e3f33bd move admin commands into their own module 2020-01-24 11:25:48 -05:00
ansuz
39b0785406 apply custom limits immediately at startup 2020-01-24 09:57:12 -05:00
ansuz
9cdf54aff2 untested implementation of trimHistory 2020-01-23 17:58:24 -05:00
ansuz
c388641479 drop support for 'retainData' configuration 2020-01-23 15:13:19 -05:00
ansuz
f45de2b52f move some server deps from repo root to lib/ 2020-01-23 15:01:56 -05:00
ansuz
b585dd998d throw in a little asynchrony 2020-01-23 13:27:12 -05:00
ansuz
7072fe4fa4 implement and test a complex constrained scheduler 2020-01-15 13:32:52 -05:00
Adrian Nöthlich
a714e41d49
Log errors in faulty custom config file 
CryptPAD now terminates and does not fallback to default config if the custom config is faulty.
The error in the custom config is logged to the terminal now.

Signed-off-by: Adrian Nöthlich <git@promasu.tech>
 2020-01-04 23:58:01 +01:00
ansuz
1da662687f guarantee asynchronous execution for batch reads and write queues 2019-12-05 13:35:26 -05:00
ansuz
9b74136b66 Merge branch 'invitations' into staging 2019-09-30 10:53:10 +02:00
ansuz
69e61ec2f2 ignore metadata commands which add invalid owners 2019-09-27 14:28:59 +02:00
ansuz
069776e804 use fast implementations of tweetnacl serverside 2019-09-27 13:30:17 +02:00
ansuz
cb6b4a8eb3 always supply a callback when writing messages 2019-09-12 18:04:24 +02:00
ansuz
ab868237a0 replace ad hoc line stream with readline module 2019-09-11 11:01:41 +02:00
ansuz
73854a8b6a implement Util.both 2019-09-11 10:54:44 +02:00
ansuz
a3eff2728e update lib/once.js to match clientside definition 2019-09-09 16:39:54 +02:00
ansuz
29813e485d add a little debugging info to batch-read 2019-09-06 15:03:42 +02:00
ansuz
fe884278f4 ignore non-interesting errors when processing metadata 2019-09-06 13:36:06 +02:00
ansuz
dc078c1ca1 test mailboxes and RPCs from the command line, fix some minor errors 2019-09-06 13:35:36 +02:00
ansuz
d81ff791ad WIP cli tests for rpc 2019-09-05 18:01:21 +02:00
ansuz
9ce7cea9cc abstract historyKeeper's batched reads implementation into a library 2019-09-04 17:22:06 +02:00
ansuz
4902554a61 implement per-channel fifo queues for metadata and channel writes 2019-09-04 11:51:33 +02:00
ansuz
70161dbd73 Merge branch 'staging' of github.com:xwiki-labs/cryptpad into teams-server-features 2019-09-03 17:08:16 +02:00
ansuz
ba18d5c79d WIP CLI client 2019-09-03 16:36:27 +02:00
yflory
4027672678 Store the mailbox of all the owners 2019-09-03 15:11:23 +02:00
yflory
d52e449f87 Fix setMetadata RPC saferphore 2019-09-03 13:29:57 +02:00
yflory
ed5671a548 Don't write metadata updates to the log if it doesn't change the value 2019-09-02 17:16:14 +02:00
yflory
738030e8c0 Add pending_owners commands 2019-08-30 12:07:03 +02:00
yflory
e12e5e1f25 Allow pending_owners to claim ownership 2019-08-30 11:49:15 +02:00
yflory
28b2341f2c Serverside metadata update 2019-08-28 12:40:35 +02:00
ansuz
c5ca82f0e1 throw errors if you try to update expiration times, as this isn't implemented yet 2019-08-21 12:28:19 +02:00
ansuz
0a0c64d240 only increment the metadata line index if you can parse the line 2019-08-21 12:26:46 +02:00
ansuz
1c37ec7aee prevent duplicated owners in metadata amendments 2019-08-21 12:25:17 +02:00
ansuz
ce847700c4 expose the index of the metadata log processor 2019-08-12 15:26:23 +02:00
ansuz
b36f50f2c9 update metadata queries to include edits 2019-06-27 18:21:12 +02:00
ansuz
efd0efede4 implement RESET_OWNERS command for editable metadata 2019-06-27 12:13:29 +02:00
ansuz
7b18bf91ce implement functions for iterating over metadata logs 2019-06-27 11:11:22 +02:00
zimbatm
35dd3abbb5
make the config path configurable 2019-05-15 14:56:07 +02:00
ansuz
ecd9647a2e add shutdown command for the logstore 2019-04-23 13:25:03 +02:00
ansuz
cd793b295d provide a standard api for calculating pin data for a given public key 2019-04-17 16:30:45 +02:00
ansuz
5c9cd5ffc6 standardize pin calculation routine 2019-04-17 14:46:46 +02:00
ansuz
d5a9eacef0 standardize loading of server configuration 2019-04-17 13:48:39 +02:00
ansuz
dde28dfa02 reorder logLevel priority 2019-04-11 11:23:22 +02:00
ansuz
f9eb1fde67 oops 2019-04-11 10:54:00 +02:00
ansuz
0bd14bfcbe fix logging to stdout and document logging configuration 2019-04-09 14:29:02 +02:00
ansuz
b3e0f9e886 fix use before definition 2019-04-09 11:57:33 +02:00
ansuz
4e9273f35c improved logger behaviour and updated default configuration 2019-04-09 11:40:51 +02:00
ansuz
e1e6d51038 new logging api 2019-04-08 18:11:36 +02:00