blitzwing/cryptpad
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
15485 commits 72 branches 132 tags 541 MiB
Commit graph

693 commits

Author SHA1 Message Date
ansuz
7b6c8b83ef stricter websocket CSP and tests to match 2022-02-14 11:45:44 +05:30
ansuz
383684d339 add new, very specific tests for CSP to the checkup page 2022-02-10 16:53:14 +05:30
ansuz
ebcb6b2ab0 log child process ids in the event of an RPC timeout 2022-02-03 15:11:26 +05:30
ansuz
d2852fdba4 perform extra validation for private message rpc 2022-01-26 03:58:29 +05:30
ansuz
29fe4b7223 restrict unnecessarily permissive CSP 2022-01-21 18:01:55 +05:30
ansuz
a93ab05310 handle absolute paths in a few obviously problematic cases 2021-11-22 18:16:35 +05:30
ansuz
b8d6af7891 adjust CSP headers for printing from OnlyOffice 
* allow outer to load resources from the sandbox (for fonts)
* test whether the expected CSP values are present on the checkup page
* simplify the nodejs server a bit
 2021-10-19 14:22:10 +05:30
ansuz
c1f9d11333 delay working on some flagged issues till next release 2021-08-16 18:32:34 +05:30
ansuz
1c08a27f60 fix a newly introduced reference error 2021-08-10 18:48:34 +05:30
ansuz
a33774aa6f properly abort from channel archival if no metadata is returned 
and add some logging when trimming history
 2021-08-10 15:24:23 +05:30
ansuz
854a635c5f remove hardcoded translations and defer non-blocking issues till the next release 2021-07-27 03:40:03 +05:30
ansuz
9bb884cac6 fix a type error caused by (my) invalid API use 2021-07-06 18:21:55 +05:30
ansuz
f13b82bdf6 disable integrated eviction by default 
we'll implement an admin panel checkbox to enable it later
 2021-07-06 16:14:40 +05:30
ansuz
7bdabb5cbc archive blocks before overwriting them 2021-07-06 15:42:37 +05:30
ansuz
1c1dc421a3 fix broken block archival 2021-07-01 16:44:40 +05:30
ansuz
f7f2146fa5 miscellaneous cleanup and notes 2021-06-30 19:09:53 +05:30
ansuz
d27cbb69dc filter email and instance purpose from telemetry unless we have consent 2021-06-30 19:05:37 +05:30
ansuz
76b90d3c8a correct a few more places where the client expected hashes in pin responses 
or where the server incorrectly provided them
 2021-06-30 18:15:55 +05:30
ansuz
8e725f3d7c stop returning the hash of all user pins after pinning 
the client doesn't use it and it's CPU-intensive
 2021-06-30 17:20:03 +05:30
ansuz
7c7acbeae6 delegate block validation to workers 2021-06-28 15:07:48 +05:30
ansuz
ba1a7b37e1 separate validation and storage methods for blocks 2021-06-28 14:39:26 +05:30
ansuz
9806d718d5 implement block archival 2021-06-25 20:53:09 +05:30
ansuz
caece0123e archive pin logs instead of removing them outright 2021-06-25 20:22:16 +05:30
ansuz
3cbf4c9d6f save instancePurpose choice to server from admin panel 2021-06-25 13:04:29 +05:30
ansuz
9027409ce5 serverside components of instancePurpose flag 2021-06-25 12:29:13 +05:30
ansuz
f5e91ef3ef provide installMethod detail in server telemetry 2021-06-22 16:32:33 +05:30
ansuz
bb7c40237b detect new versions in server telemetry responses 2021-06-10 20:56:12 +05:30
ansuz
2fd812ff36 WIP admin translations and update available button 2021-06-09 18:45:02 +05:30
ansuz
98c8d7086f include instance version in Env 2021-06-09 18:43:31 +05:30
ansuz
41db88a360 WIP admin 'network' panel 2021-06-08 20:24:30 +05:30
ansuz
eea0688879 log messages which fail signature validation 2021-05-31 15:53:55 +05:30
ansuz
87c6e3270d make a note about restricting channel creation to registered users 2021-05-20 16:13:28 +05:30
ansuz
62f8fb93b5 take notes of desirable admin RPC commands 2021-05-07 18:07:13 +05:30
yflory
7a682397e2 Add checkup test about Google's Floc 2021-05-07 14:23:15 +02:00
ansuz
eb32651c67 leave some notes about a minor race condition 2021-05-05 12:07:45 +05:30
ansuz
30fc2a5edf Merge branch 'staging' into restricted-registration 2021-05-01 02:04:22 +05:30
ansuz
8c61948d02 implement SET_ADMIN_EMAIL and SET_SUPPORT_MAILBOX decrees 
and update changelog
 2021-04-30 14:48:22 +05:30
ansuz
729d51fb9b update block storage APIs 2021-04-29 13:32:19 +05:30
ansuz
15a8284a30 add some logging and avoid some errors 2021-04-29 07:09:16 +05:30
ansuz
18c73ad282 WIP working password change with restricted registration 2021-04-28 11:16:55 +05:30
ansuz
8896391536 refactor writeLoginBlock to account for more asynchronous flow 2021-04-27 17:13:31 +05:30
ansuz
50045c08d0 WIP restrict registration 2021-04-26 18:31:33 +05:30
ansuz
15d6f573e7 don't send DUPLICATE error 2021-04-12 17:47:45 +05:30
ansuz
75a316e70d adjust or remove comments 2021-04-12 13:39:54 +05:30
yflory
6027123d12 Fix maintenance 2021-04-08 16:10:03 +02:00
yflory
7f44009617 Better validation function in decrees 2021-04-08 13:04:12 +02:00
ansuz
4e3a7fef24 add notes from our review 2021-04-02 17:14:15 +05:30
ansuz
67af5c220c server notes 2021-04-02 09:59:28 +05:30
yflory
8f679c141c Broadcast update 2021-03-30 17:41:12 +02:00
yflory
96892bff48 Merge branch 'staging' into broadcast 2021-03-30 13:53:49 +02:00
yflory
6f1caf3540 Fix team creation 2021-03-26 17:12:27 +01:00
ansuz
75f9459950 leave a FIXME in the server 2021-03-23 09:25:02 +05:30
ansuz
5eddb41d77 add more background info for some protocol semantics 2021-03-22 16:51:11 +05:30
yflory
ed09b92592 Fix deleted pad restored corrupted from cache 2021-03-22 10:39:46 +01:00
yflory
7971a6d267 Major broadcast refactoring 2021-03-16 17:00:23 +01:00
yflory
dbce0fe429 Use 33 characters admin channels 2021-03-16 15:25:28 +01:00
ansuz
5f269fbe53 Merge branch 'soon' into staging 2021-03-15 15:35:42 +05:30
yflory
d15c0461cc Add /api/broadcast and improve message deletion 2021-03-12 12:46:11 +01:00
yflory
a17e732473 Add XXX 2021-03-10 17:40:42 +01:00
yflory
fee8a88169 Instance notification prototype 2021-03-09 18:27:12 +01:00
Dmitry Borodaenko
6cd2131f27 Pass archivePath to Store.create always 2021-03-03 11:31:08 -08:00
ansuz
c9d1996e0d avoid incrementing index size twice for a single message 2021-03-01 18:23:24 +05:30
ansuz
4316264044 fix incomplete unowned file upload 2021-02-19 13:12:35 +05:30
yflory
f5908e5f57 Fix history range 2021-02-18 16:28:40 +01:00
ansuz
baab65fa36 Merge branch 'callback' into staging 2021-02-17 17:47:57 +05:30
ansuz
70ba468211 lint compliance 2021-02-17 17:47:23 +05:30
yflory
f09ef072ac Fix XXX 2021-02-17 12:56:51 +01:00
ansuz
9e1d82f373 move archive deletion out of the main eviction script 2021-02-17 14:17:41 +05:30
ansuz
50bec7386d flag logic that is missing callbacks 2021-02-16 19:25:47 +05:30
yflory
287b80d4b8 Merge branch 'staging' into callback 2021-02-09 15:30:46 +01:00
yflory
ea4c529e39 Remove offset code 2021-02-09 15:29:34 +01:00
ansuz
b8a7defad9 lint compliance 2021-01-31 15:29:55 +05:30
ansuz
4a29d9df00 implement admin GET_USER_TOTAL_SIZE call 2021-01-31 15:28:45 +05:30
ansuz
8990e8def9 provide more context for EUNKNOWN logs 2021-01-26 10:31:29 +05:30
ansuz
43a5d5664f remove XXX 2021-01-21 17:42:41 +05:30
yflory
0fc8bfde4a Use new server format for history keeper time 2021-01-20 15:06:43 +01:00
yflory
fc514fb61d Support a callback on channelMessage 2021-01-19 14:53:38 +01:00
yflory
6c394b37c8 Merge branch 'staging' into eunknown 2021-01-19 10:49:53 +01:00
yflory
e5af30d678 more EUNKNOWN 2021-01-13 10:16:42 +01:00
ansuz
d1eff71584 Merge branch 'soon' into staging 2021-01-08 21:23:18 +05:30
ansuz
929159a0f4 implement server offline mode (no ws) 2021-01-08 14:49:04 +05:30
ansuz
c806b97076 resolve conflicts between rebrand and staging 2020-12-29 17:20:53 +05:30
ansuz
fed04a025c display size of uploaded files upon completion 2020-12-23 17:21:06 +05:30
ansuz
646fdcc43e Merge branch 'cache-metadata' into staging 2020-12-11 20:16:18 +05:30
ansuz
de90949e3e suppress ENOENT on metadata when restoring archived files 2020-12-11 20:12:36 +05:30
ansuz
4c7f7e271e remove an XXX note, log when restoring docs by admin rpc 2020-12-11 17:57:42 +05:30
ansuz
d0df2f5840 log when archiving content via the admin panel 2020-12-11 17:03:48 +05:30
ansuz
66195a17a1 Merge branch 'staging' of github.com:xwiki-labs/cryptpad into cache-metadata 2020-12-08 16:57:13 +05:30
ansuz
e6a4c68863 always delete the throttled cleanup functions once they have run 2020-12-08 16:50:37 +05:30
ansuz
4495d99e49 use the standard method of checking whether metadata is valid 2020-12-08 16:45:57 +05:30
ansuz
6bc103be5f delay cache eviction each time the metadata is requested 2020-12-08 16:43:50 +05:30
ansuz
d4f94f3091 briefly cache server metadata in-memory to avoid repeated reads 2020-12-08 16:34:28 +05:30
yflory
34284fcc0c Fix issues with deprecated cache 2020-12-08 12:03:40 +01:00
ansuz
820f2a0054 Merge branch 'soon' into staging 2020-12-08 11:14:53 +05:30
ansuz
93d56e5e3d profile worker commands by total running time in seconds 2020-12-08 10:44:46 +05:30
ansuz
93c807524f update offset debugging info 2020-12-08 10:24:09 +05:30
yflory
5bbde5fec5 Merge branch 'soon' into cacheRT 2020-12-01 16:40:18 +01:00
ansuz
d4e6d86961 enforce asynchrony in new database offset methods 2020-11-27 08:21:51 +05:30
ansuz
466072d03b read files starting from the oldest known point of relevance when computing indices 2020-11-26 12:15:02 +05:30
yflory
93ddf51564 Merge branch 'download' into cacheRT 2020-11-24 16:46:22 +01:00
ansuz
cd77b5ffc8 Merge branch 'soon' into staging 2020-11-19 13:18:08 +05:30
ansuz
faa84bcbd7 describe arguments for 'completeUpload' RPC 2020-11-19 12:16:02 +05:30
yflory
bd4b44476d Archive and restore UI in the admin panel 2020-11-18 16:49:40 +01:00
ansuz
e185008846 complete uploads in child processes 
also fix a nasty race condition for unowned file uploads
 2020-11-18 18:26:21 +05:30
ansuz
b155f7b291 increase child-process timeout and improve error handling 2020-11-17 16:09:47 +05:30
David Benqué
ead97d8f41 Remove FAQ and references to it 2020-11-16 11:06:48 +00:00
yflory
5946b10d0b No page reload when the cache is corrupted onReady 2020-11-06 15:00:58 +01:00
ansuz
1f1b40b83d move some implictly global state to env.js 2020-11-03 16:48:31 +05:30
ansuz
aa547a7b76 align nodejs http headers with example nginx 2020-11-03 16:48:31 +05:30
ansuz
5cf8ca70e6 don't pin falsey document ids 2020-11-03 16:48:31 +05:30
yflory
c9a3cad78b Fix various issues with restricted pads 2020-11-03 16:48:31 +05:30
yflory
d16cc2472d First prototype of an offline cache 2020-11-03 10:49:13 +01:00
David Benqué
66d434b583 Remove about page from defaults and add XXX for future pages 2020-10-30 14:29:15 +00:00
ansuz
034472d3dd move some implictly global state to env.js 2020-10-27 12:47:15 +05:30
ansuz
0bc2433e02 Merge branch 'accessBug' into staging 2020-10-27 09:09:09 +05:30
ansuz
d95b0954f9 Merge branch 'fix-printing' into staging 2020-10-27 08:13:58 +05:30
ansuz
a2b79d84b8 align nodejs http headers with example nginx 2020-10-27 08:12:23 +05:30
ansuz
f7bd3bdc23 don't pin falsey document ids 2020-10-26 17:34:41 +05:30
yflory
69c26fe8c7 Fix various issues with restricted pads 2020-10-23 15:44:47 +02:00
ansuz
d72e053560 make a note to improve stream timeout error handling 2020-10-22 11:24:58 +05:30
ansuz
67430de7ff Make efforts to avoid closing streams mid-read 
1. Close streams when you're done with them
2. Close streams if they seem to have been abandoned
 2020-10-22 11:17:03 +05:30
ansuz
fbfb25bf29 lint compliance 2020-10-21 21:29:52 +05:30
ansuz
100b417646 guard against several serverside typeErrors 
and warn in cases where they would have occurred
 2020-10-21 18:24:54 +05:30
ansuz
e8428a2a73 prevent a case of multiple callbacks 2020-10-21 18:13:10 +05:30
ansuz
fbc9edd795 rename latest admin commands and implement metadata getter/remover 2020-10-20 17:12:26 +05:30
ansuz
488ec93ece allow expert admins to get and clear cached channel indices 2020-10-20 17:07:55 +05:30
ansuz
faa7ebf399 Fix some bugs with integrated eviction 
1. implement SET_LAST_EVICTION as an admin command, not a decree
2. expect a return value from Env.evictInactive and expose it via Env.evictionReport
 2020-10-15 13:15:39 +05:30
ansuz
ac322c8e82 make the eviction script call back with accumulated statistics 2020-10-14 17:23:53 +05:30
ansuz
a29c5641b8 introduce some artificial delays in the eviction script 
...and reduce the margin of error in its bloom filters
 2020-10-14 17:22:31 +05:30
ansuz
90f046f896 prevent long-running worker tasks from timing out 2020-10-14 17:18:57 +05:30
ansuz
6e57366b7f address a file descriptor leak 2020-10-13 13:14:39 +05:30
ansuz
a2c0d2165b implement a few more admin decrees 2020-10-13 11:36:30 +05:30
ansuz
abd84665ae lint compliance 2020-10-13 11:00:45 +05:30
ansuz
6ec5171518 add support for changing a few more Env parameters at runtime 2020-10-13 10:56:40 +05:30
ansuz
adb988058d stop logging dev/fresh mode 2020-10-12 19:19:57 +05:30
ansuz
f2ec9cbe33 load premium and customLimits to avoid evicting them even if inactive 2020-10-12 19:18:12 +05:30
ansuz
45b063e378 suppress irrelevant errors from Decrees.load 2020-10-12 19:15:48 +05:30
ansuz
854e4c06ff report lastEviction in an admin INSTANCE_STATUS 2020-10-12 17:52:15 +05:30
ansuz
471a9a33ac take note of a minor error with pin RPCs 2020-10-12 17:51:38 +05:30
ansuz
f86409e1ba take note of which Environment variables should be made mutable via decree 2020-10-12 17:50:40 +05:30
ansuz
6d13a785c6 downgrade a non-critical XXX to a FIXME 2020-10-12 17:42:41 +05:30
ansuz
46ebd7b40b remove unsupported storage configuration parameters 2020-10-12 17:40:21 +05:30
ansuz
369c92c01d initialize Env from server and deduplicate several attributes 2020-10-12 17:39:53 +05:30
ansuz
b5d3a10dc2 guard against possible typeErrors from unvalidated config input 2020-10-09 13:28:13 +05:30
ansuz
964aa2bb79 validate custom limits provided via config at launch 2020-10-08 19:07:00 +05:30
ansuz
f4f803ccd9 reuse Keys.canonicalize to accept any pubkey format 2020-10-08 19:07:00 +05:30
ansuz
914139dbfa lint compliance 2020-10-08 19:07:00 +05:30
yflory
51e1b9b043 Fix custom limit not applied when using a raw edPublic 2020-10-08 19:07:00 +05:30
ansuz
145386e944 suppress an inconsequential error 2020-10-08 19:07:00 +05:30
ansuz
aa35e092f7 update an inaccurate comment 2020-10-08 19:07:00 +05:30
ansuz
928b7e3583 implement GET_LIMITS admin rpc 
...and label stored usage limits according to their source
 2020-10-08 19:07:00 +05:30
ansuz
5029281a53 implement RM_QUOTA decree 2020-10-08 19:07:00 +05:30
ansuz
af47b2cab1 implement ADD_QUOTA admin decree 2020-10-08 19:07:00 +05:30
ansuz
998f2267c4 ensure the path to the decree file exists before attempting to write to it 2020-10-08 19:07:00 +05:30
ansuz
c2918ac888 fix inverted conditional 2020-10-08 19:07:00 +05:30
ansuz
b62d4387a6 drop support for SET_DEFAULT_STORAGE_LIMIT 
it's redundant now that we support persistent admin decrees
and it was never exposed via the UI anyway
 2020-10-08 19:07:00 +05:30
ansuz
0c5db31fbc refactor quota logic for external usage 2020-10-08 19:07:00 +05:30
ansuz
bd6234c5bc lint compliance and some notes 2020-10-08 19:07:00 +05:30
ansuz
1781ee2585 implement 'UPDATE_DEFAULT_STORAGE' as a decree 2020-10-08 19:07:00 +05:30
ansuz
7aa7d5978f implement persistence of config changes from admin panel 2020-10-08 19:07:00 +05:30
ansuz
fcfcb3dca6 fix history trim for channels with access lists 2020-10-07 18:44:24 +05:30
ansuz
d509fd8b8f downgrade severity of several XXX notes 2020-10-07 17:09:46 +05:30
ansuz
1ba080739b Merge branch 'X' into staging 2020-10-07 16:28:08 +05:30
ansuz
cfcfe2f65f enable APIs for XLSX export in firefox 2020-10-07 14:47:43 +05:30
yflory
e698241ee9 Merge branch 'historyOO' into staging 2020-10-05 16:41:23 +02:00
ansuz
04d1bc9cd7 archive inactive data in one of the server's workers 2020-09-30 14:45:52 +05:30
ansuz
6ada5fd751 always consider premium accounts to be active 2020-09-29 18:48:23 +05:30
ansuz
5358fab615 more WIP inactive account eviction 2020-09-29 18:04:51 +05:30
ansuz
a9b95865ba Merge branch 'staging' into inactive-accounts 2020-09-28 17:56:30 +05:30
ansuz
d9219f4ecc add a note for later 2020-09-25 14:27:22 +05:30
ansuz
5749b6fd7b add an admin RPC to archive documents 2020-09-25 14:24:26 +05:30
ansuz
d794e0b48f include inactive accounts in the archival script 2020-09-24 19:02:22 +05:30
yflory
25606b92bf lint compliance 2020-09-22 15:27:16 +02:00
yflory
4f147d4fd2 Add support for onlyoffice history 2020-09-18 17:54:57 +02:00
yflory
b31707098a Improve getHistoryRange to support 'toHash' 2020-09-18 12:42:09 +02:00
ansuz
95778cc7c9 respond with ENOENT instead of E_NO_OWNERS when archiving a file that does not exist 2020-07-23 09:30:04 -04:00
ansuz
b04133e6c3 remove some dead code 2020-07-23 09:28:00 -04:00
ansuz
a21c3372c0 avoid a potential type error 2020-07-23 09:27:29 -04:00
yflory
04af753a81 lint compliance 2020-07-01 12:10:12 +02:00
ansuz
03be102ce4 deduplicate some code by reusing the client definition on the server 2020-06-30 14:33:00 -04:00
ansuz
ebcc9a069b add server-side support for the new format of public signing keys 2020-06-25 13:14:26 -04:00
ansuz
48deced859 fix serverside index corruption 2020-06-16 04:08:03 -04:00
ansuz
5b2929a6fc suppress some useless info from some commonly logged errors 2020-05-06 13:50:48 -04:00
ansuz
9f1f01f3b4 disable worker task queueing and address a probable memory leak 2020-05-05 18:06:28 -04:00
ansuz
b0af6b5488 fix an incorrect condition when checking for available server workers 
Also, log when the RPC queue has a backlog and when it is drained
 2020-05-05 10:53:42 -04:00
ansuz
4037270a23 add a FIXME note for a sketchy implementation in history keeper 2020-05-04 15:46:08 -04:00
ansuz
02347d134a queue signature validation per-channel to ensure correct ordering 2020-04-30 18:18:05 -04:00
ansuz
9975c27e84 increase timeout from one minute to three and decrease the number of parallel jobs per worker 2020-04-29 17:24:38 -04:00
ansuz
488f876ad6 lint compliance 2020-04-29 17:23:29 -04:00
ansuz
d35b015c3f log some info on successful file uploads 2020-04-28 17:12:03 -04:00
ansuz
37d3995ac1 unify format of console output with stored logs 2020-04-28 17:05:15 -04:00
ansuz
8a5d012edf add special error handling for a few special serverside errors 2020-04-28 16:57:52 -04:00
ansuz
8bee95d13d improve error reporting in GET_HISTORY 2020-04-28 16:56:11 -04:00
ansuz
e05891184e fix regression in maxWorkers 2020-04-21 12:23:48 -04:00
ansuz
b56c73be6e delegate more work from getOlderHistory to the worker 2020-04-17 17:01:47 -04:00
ansuz
fd169ff39c close streams after 2 minutes instead of 5 
Hopefully this helps to even out our memory and CPU usage.
There's also some improved labels so that IF streams time out
we will know the exact origin of the offending call.
 2020-04-17 15:05:48 -04:00
ansuz
bac8ef3c6e lint compliance 2020-04-17 11:43:54 -04:00
ansuz
4e57e390da write tasks inside of workers 2020-04-16 16:37:24 -04:00
ansuz
ba6faca02e make the number of workers configurable 2020-04-16 13:53:45 -04:00
ansuz
0465f31a45 add a FIXME note 2020-04-16 13:02:27 -04:00
ansuz
04ab7f538a recover from worker faults and unify worker types 2020-04-16 13:02:07 -04:00
ansuz
e0a6852b79 correctly serialize an error log 2020-04-16 10:45:18 -04:00
ansuz
9ed82640bf Merge branch 'worker-queue' into staging 2020-04-15 19:32:42 -04:00
ansuz
e8b1fcf710 solve some cases where crashing workers could result in an invalid state for the queue 2020-04-15 13:59:54 -04:00
ansuz
9ed26cfeb0 lint compliance 2020-04-15 12:40:18 -04:00
ansuz
dc0460780f Merge branch 'soon' into staging 2020-04-15 09:44:30 -04:00
ansuz
74567e0cf4 return process memory usage in GET CACHE STATS RPC 2020-04-14 15:53:58 -04:00
Martin Alexander Neumann
e3f42cbc20 Allow config.defaultStorageLimit to be zero bytes 2020-04-11 20:46:09 +02:00
ansuz
89262cd29e resolve silly conflict and merge staging 2020-04-09 16:35:24 -04:00
ansuz
e527867e2e queue blob and channel deletions per-user 2020-04-09 16:29:52 -04:00
ansuz
bd3e18d1a8 only cache a user's pinned channels if there are no errors when loading them 2020-04-08 12:08:26 -04:00
ansuz
bef18a9320 hotfix 2020-04-08 10:28:26 -04:00
ansuz
7fac997e93 increase some file storage timeouts related to streams 2020-04-08 09:38:48 -04:00
ansuz
95965c1dee keep a parallel implementation of the pin loader to validate the new one 2020-04-07 20:03:41 -04:00
ansuz
a57bf94058 remove XXX note 2020-04-06 11:31:31 -04:00
ansuz
c53675c9d5 WIP worker rpc call queue 2020-04-06 10:28:26 -04:00
ansuz
d802173325 remove an invalid line 2020-04-06 10:18:11 -04:00
ansuz
2b3f6e3464 Merge branch 'soon' into staging 2020-04-03 10:49:51 -04:00
ansuz
adfd1eaaa2 avoid a silly typeError in the event of an error 2020-04-02 22:57:57 -04:00
yflory
32406a5995 Merge branch 'master' into staging 2020-04-02 10:23:09 +02:00
ansuz
834c96a4fc update comments and remove some notes 2020-04-01 17:43:40 -04:00
ansuz
ed5d97f899 WIP history-keeper fixes 2020-04-01 11:48:16 -04:00
ansuz
eddbe80eef Merge branch 'soon' into staging 2020-03-30 18:30:34 -04:00
ansuz
3d8e78a268 fix overly broad config interpretation for daily check 2020-03-30 18:27:14 -04:00
ansuz
e6ec891d9a improve admin rpc for diagnosing memory used by the cache 2020-03-30 18:26:04 -04:00
ansuz
77961e3954 trim lookup tables for channels without checkpoints 2020-03-30 18:03:37 -04:00
ansuz
db9eaa8d4f Merge branch 'soon' into staging 2020-03-27 19:59:45 -04:00
ansuz
6d0dee979a allow admins to inspect index/metadata caches 2020-03-27 19:59:26 -04:00
ansuz
cbd3547814 remove log statements 2020-03-27 17:20:57 -04:00
ansuz
9058a59555 reassign db tasks if the responsible worker fails 2020-03-27 17:17:42 -04:00
ansuz
172823c954 lint compliance 2020-03-27 16:59:41 -04:00
ansuz
b5649707d1 export 'tryParse' command 2020-03-27 15:36:34 -04:00
ansuz
64b0879984 improve logging for parse errors 2020-03-27 15:15:54 -04:00
ansuz
3f86b6141e rename worker processes 2020-03-27 15:14:45 -04:00
ansuz
d8a88cb4ca run expiration tasks in a worker instead of the main process 2020-03-27 14:38:58 -04:00
ansuz
b0179eaad9 drop XXX note 2020-03-27 14:25:07 -04:00
ansuz
9dbd32758a improve worker logging and move blob deletion to worker processes as well 2020-03-27 13:38:27 -04:00
ansuz
5f69fc18d0 suppress some noisy errors 2020-03-26 17:11:43 -04:00
ansuz
5f2d7c8dcf increase worker rpc wait time before timeout 2020-03-26 15:53:00 -04:00
ansuz
33e8e65507 handle errors in the server's workers 2020-03-26 15:13:20 -04:00
ansuz
a4c8039cc7 improve error handling with rpc response API 2020-03-26 14:45:24 -04:00
ansuz
cb53bd1c15 lint compliance 2020-03-26 14:44:37 -04:00
yflory
e15196bd53 Merge branch 'reconnect' into staging 2020-03-26 15:24:38 +01:00
ansuz
14feef1757 calculate pin list checksums in a worker 2020-03-25 18:37:42 -04:00
ansuz
70a0d4efb4 move more database reads into the database worker 2020-03-25 17:43:57 -04:00
ansuz
bc13a21796 give fileStreams a little bit more time before closing them 2020-03-25 11:50:45 -04:00
ansuz
4ba36a9173 load user pins in the database worker 2020-03-25 11:39:14 -04:00
ansuz
50e8893b24 move the 'getOlderHistory' call into the database worker 2020-03-25 10:54:29 -04:00
ansuz
471e374533 compute metadata in the same child process that builds indexes 2020-03-24 17:43:15 -04:00
ansuz
479b76f848 lint compliance 2020-03-24 17:40:07 -04:00
ansuz
c2fcba31e3 remove an XXX 2020-03-24 15:02:46 -04:00
ansuz
967ca6afa9 suppress ENOENT errors when checking empty chat channels 2020-03-24 15:02:26 -04:00
ansuz
40251948d4 check authenticated rpc signatures in separate threads 2020-03-24 14:31:40 -04:00
ansuz
0e09c73a60 Merge branch 'master' into soon 2020-03-24 13:32:42 -04:00
ansuz
a2facc4a0d
Merge pull request #513 from 7adietri/patch-1 
Add shutdown() to non-disk logger
 2020-03-24 13:32:13 -04:00
ansuz
e128683ffb call back with errors instead of throwing within the Storage module 2020-03-24 13:31:03 -04:00
ansuz
1240f4a2a5 Merge branch 'master' into soon 2020-03-24 13:04:25 -04:00
ansuz
90899aa2f4 start reviewing and removing XXX notes 2020-03-24 13:03:49 -04:00
yflory
fe14399e67 Merge branch 'whiteboard' into soon 2020-03-24 16:15:13 +01:00
ansuz
2ef0af29e5 guard against index worker rpc responses for a txid with no supplied callback 2020-03-23 22:03:59 -04:00
ansuz
ee8d5c9c36 fix the premiumUploadSize validation to check the correct attribute 2020-03-23 16:13:48 -04:00
yflory
dc0b26da0b Merge branch 'soon' into staging 2020-03-23 12:33:17 +01:00
Alexander
a190af079d
Add shutdown() to non-disk logger 
The script `evict-inactive.js` calls `Log.shutdown()` at the end, which currently fails if logging to disk has been disabled via `logPath`.
 2020-03-21 14:43:25 +01:00
ansuz
ed63d98ab0 Merge branch 'scaling-index' into soon 2020-03-20 18:57:51 -04:00
ansuz
590e249500 resolve merge conflict 2020-03-20 13:07:35 -04:00
ansuz
31c7cecaf9 add a few checks for safety 2020-03-20 10:54:01 -04:00
ansuz
cea9705bbe Merge branch 'aggressive-stream-closing' into staging 2020-03-20 10:50:17 -04:00
ansuz
faa133aab8 remove an XXX 2020-03-20 10:22:48 -04:00
ansuz
d386e223e4 simplify open/close of writeStreams 2020-03-19 17:33:22 -04:00
ansuz
32cd0f3c4d increase timeout value from 15s to 45. ought to be enough for anybody 2020-03-19 16:13:24 -04:00
ansuz
05a4e86cdb close streams whenever we finish using them. time out if necessary 2020-03-19 16:11:24 -04:00
ansuz
4522ffa18a compute indexes in child processes 2020-03-19 10:46:18 -04:00
ansuz
f8ad649b45 [style] bail out early to avoid nesting 2020-03-18 10:30:42 -04:00
ansuz
019e5e708b wrap workers in a function scope and add a validateMessage method to HK's Env 2020-03-17 17:23:35 -04:00
ansuz
5467e1ffac replace ad-hoc response handler with Util.response 2020-03-17 16:52:41 -04:00
ansuz
9e85a1411e abstract the logic around worker choice out of message handling 2020-03-17 16:06:01 -04:00
ansuz
631ea54b49 lint compliance 2020-03-17 15:40:06 -04:00
ansuz
67dd4a608c Merge remote-tracking branch 'origin/scaling' into scaling2 2020-03-17 15:36:58 -04:00
ansuz
b2fcde87d8 implement an admin command to update the default storage limit without a restart 2020-03-17 10:11:52 -04:00
yflory
923616aef0 Use each process in order instead of using a random one 2020-03-17 15:10:23 +01:00
ansuz
f345998956 fix undefined reference 2020-03-17 10:10:20 -04:00
yflory
fb0eb1b20c Use more subprocesses 2020-03-17 14:18:41 +01:00
yflory
0d636dabc9 Check signature for history keeper in a different process 2020-03-17 13:29:53 +01:00
ansuz
30f17040ac close streams when possible, simplify some code, enforce asynchrony 2020-03-16 16:13:38 -04:00
ansuz
2a9780abcf lint compliance 2020-03-16 12:29:26 -04:00
yflory
578042154e Better reconnect after invalid lastKnownHash 2020-03-13 17:39:16 +01:00
ansuz
2d47e7e2cf implement an admin rpc to fetch ulimit data 2020-03-12 12:09:44 -04:00
ansuz
81c7416095 lint compliance 2020-03-12 11:40:46 -04:00
ansuz
f644c24362 add an admin-rpc to count the number of open files 2020-03-12 11:35:16 -04:00
ansuz
1230057970 disable IS_PAD_PINNED 2020-03-12 11:31:08 -04:00
ansuz
e1a0daac9c possible server fixes 2020-03-12 11:23:43 -04:00
ansuz
bc034d95a3 enforce asynchrony in 'gethistoryOffset' 2020-03-12 11:13:23 -04:00
ansuz
d4038344ba stop iterating over tasks if you can't read the root directory 2020-03-11 16:34:00 -04:00
ansuz
4e8ebac3c1 guard against the possible non-existence of metadata for a channel 2020-03-09 17:42:10 -04:00
ansuz
b32f3e34af fix local worker debugging on firefox 2020-03-06 11:43:34 -05:00
ansuz
6d8cdca7a3 add 'rejected' field to metadata when getMetadata requests are blocked 2020-03-05 12:09:18 -05:00
ansuz
9d8bb43d03 finally untangle metadata and index caches 2020-03-04 12:56:33 -05:00
ansuz
32d769447a begin standardizing our method of streaming lines from files 2020-03-04 11:38:14 -05:00
ansuz
35eca2c5d2 pull file streaming out into its own file, leave a few notes 2020-03-04 11:38:14 -05:00
ansuz
ce6cf6f727 revert some changes that couldn't have possibly worked 2020-03-04 09:59:17 -05:00