Commit graph

18582 commits

Author SHA1 Message Date
Damir Jelić
4a6623bc00 EventIndex: Rework the crawler cancellation. 2019-11-18 17:00:30 +01:00
Damir Jelić
8d7e7d0cc4 EventIndex: Remove the unused deleteEventIndex method.
We need to support the deletion of the event index even if it's not
currently initialized, therefore the deletion ended up in the
EventIndexPeg class.
2019-11-18 14:40:38 +01:00
Damir Jelić
c48ccf9761 EventIndex: Remove some unnecessary checks if event indexing is supported. 2019-11-18 14:40:04 +01:00
Damir Jelić
b983eaa3f9 EventIndex: Rename the file to be consistent with the class. 2019-11-18 14:36:08 +01:00
Damir Jelić
45e7aab41e EventIndexing: Rename our EventIndexer class. 2019-11-18 14:30:07 +01:00
Damir Jelić
579cbef7b0 EventIndexPeg: Rewrite the module documentation. 2019-11-18 14:29:03 +01:00
Damir Jelić
2bb331cdf0 Searching: Fix a typo. 2019-11-18 10:58:57 +01:00
Damir Jelić
0e3a0008df Searching: Remove the func suffix from our search functions. 2019-11-18 10:58:54 +01:00
Damir Jelić
a4ad8151f8 Searching: Use the short form to build the search arguments object. 2019-11-18 10:58:53 +01:00
Damir Jelić
b4a6123295 Searching: Move a comment to the correct place. 2019-11-18 10:58:50 +01:00
Damir Jelić
3b06c684d2 EventIndexing: Don't capitalize homeserver. 2019-11-18 10:58:47 +01:00
Damir Jelić
050e52ce46 EventIndexPeg: Treat both cases of unavailable platform support the same. 2019-11-18 10:58:45 +01:00
Damir Jelić
ddb536e94a EventIndexPeg: Move a docstring to the correct place. 2019-11-18 10:58:44 +01:00
Damir Jelić
910c3ac08d BaseEventIndexManager: Fix some type annotations. 2019-11-18 10:58:42 +01:00
Damir Jelić
5149164010 MatrixChat: Revert the unnecessary changes in the MatrixChat class. 2019-11-18 10:58:40 +01:00
Damir Jelić
9fa8e8238a BaseEventIndexManager: Fix a typo. 2019-11-18 10:58:38 +01:00
Damir Jelić
ab93745460 Fix the copyright headers from New Vector to The Matrix Foundation. 2019-11-18 10:58:36 +01:00
Damir Jelić
30d4dd36a7 BaseEventIndexManager: Remove the flow annotation. 2019-11-18 10:58:31 +01:00
Damir Jelić
eb0b0a400f EventIndexPeg: Remove the now unused import of MatrixClientPeg. 2019-11-14 16:18:36 +01:00
Damir Jelić
d82d4246e9 BaseEventIndexManager: Remove a return from a docstring. 2019-11-14 16:17:50 +01:00
Damir Jelić
7516f2724a EventIndexing: Rework the index initialization and deletion. 2019-11-14 16:13:22 +01:00
Damir Jelić
448c9a8290 EventIndexPeg: Add a missing return statement. 2019-11-14 16:01:14 +01:00
Damir Jelić
28d2e658a4 EventIndexing: Don't scope the event index per user. 2019-11-14 14:13:49 +01:00
Damir Jelić
9b32ec10b4 EventIndexing: Use the correct timeout value. 2019-11-13 16:47:21 +01:00
Damir Jelić
d4b31cb7e0 EventIndexing: Move the max events per crawl constant into the class. 2019-11-13 16:35:26 +01:00
Damir Jelić
368a77ec3e EventIndexing: Fix a style issue. 2019-11-13 16:35:04 +01:00
Damir Jelić
cc2ee53824 EventIndex: Add some more docs and fix some lint issues. 2019-11-13 16:21:26 +01:00
Damir Jelić
c26df9d9ef EventIndexing: Fix a typo. 2019-11-13 15:57:12 +01:00
Damir Jelić
bf558b46c3 EventIndexPeg: Clean up the event index initialization. 2019-11-13 15:39:39 +01:00
Damir Jelić
c33f5ba0ca BaseEventIndexManager: Add a method to perform runtime checks for indexing support. 2019-11-13 15:39:06 +01:00
Damir Jelić
ab7f34b45a EventIndexing: Don't mention Seshat in the logs. 2019-11-13 15:26:27 +01:00
Damir Jelić
1316e04776 EventIndexing: Check if there is a room when resetting the timeline. 2019-11-13 15:23:08 +01:00
Damir Jelić
f453fea24a BasePlatform: Move the event indexing methods into a separate class. 2019-11-13 14:46:17 +01:00
Damir Jelić
80b28004e1 Searching: Define the room id in the const object. 2019-11-13 11:02:54 +01:00
Damir Jelić
54b352f69c MatrixChat: Fix the limited timeline checkpoint adding. 2019-11-13 10:37:20 +01:00
Damir Jelić
1df28c7526 Fix some lint errors. 2019-11-13 10:30:38 +01:00
Damir Jelić
1cc64f2426 Searching: Move the small helper functions out of the eventIndexSearch function. 2019-11-13 10:10:35 +01:00
Damir Jelić
008554463d Lifecycle: Move the event index deletion into the clear storage method. 2019-11-13 09:52:59 +01:00
Damir Jelić
3502454c61 LifeCycle: Stop the crawler and delete the index when whe log out. 2019-11-12 15:58:38 +01:00
Damir Jelić
d69eb78b66 EventIndexing: Add a missing platform getting. 2019-11-12 15:41:14 +01:00
Damir Jelić
ecbc47c548 EventIndexing: Rename the stop method. 2019-11-12 15:40:49 +01:00
Damir Jelić
d911055f5d MatrixChat: Move the indexing limited room logic to a different event. 2019-11-12 15:39:54 +01:00
Damir Jelić
e296fd05c0 RoomView: Move the search logic into a separate module. 2019-11-12 15:39:26 +01:00
Damir Jelić
cfdcf45ac6 MatrixChat: Move the event indexing logic into separate modules. 2019-11-12 13:29:07 +01:00
Damir Jelić
2c5565e502 MatrixChat: Add some missing semicolons. 2019-11-11 15:08:00 +01:00
Damir Jelić
64061173e1 MatrixChat: Check if our state array is empty in the crawled messages response. 2019-10-18 16:33:07 +02:00
Damir Jelić
89f14e55a2 MatrixChat: Catch errors when fetching room messages in the crawler. 2019-10-18 16:32:43 +02:00
Damir Jelić
1b63886a6b MatrixChat: Add more detailed logging to the event crawler. 2019-10-18 16:31:39 +02:00
Damir Jelić
3f53691834 RoomView: Use platform specific search if our platform supports it.
This patch extends our search to include our platform specific event
index.

There are 3 search scenarios and are handled differently when platform
support for indexing is present:

    - Search a single non-encrypted room: Use the server-side search
        like before.
    - Search a single encrypted room: Search using our platform specific
        event index.
    - Search across all rooms: Search encrypted rooms using our local
        event index. Search non-encrypted rooms using the classic
        server-side search. Combine the results.

The combined search will result in having twice the amount of search
results since comparing the scores fairly wasn't deemed sensible.
2019-10-11 16:55:01 +02:00
Damir Jelić
4acec19d40 MatrixChat: Add new crawler checkpoints if there was a limited timeline.
A sync call may not have all events that happened since the last time
the client synced. In such a case the room is marked as limited and
events need to be fetched separately.

When such a sync call happens our event index will have a gap. To
close the gap checkpoints are added to start crawling our room again.
Unnecessary full re-crawls are prevented by checking if our current
/room/roomId/messages request contains only events that were already
present in our event index.
2019-10-11 16:55:01 +02:00