This patch extends our search to include our platform specific event
index.
There are 3 search scenarios and are handled differently when platform
support for indexing is present:
- Search a single non-encrypted room: Use the server-side search
like before.
- Search a single encrypted room: Search using our platform specific
event index.
- Search across all rooms: Search encrypted rooms using our local
event index. Search non-encrypted rooms using the classic
server-side search. Combine the results.
The combined search will result in having twice the amount of search
results since comparing the scores fairly wasn't deemed sensible.
A sync call may not have all events that happened since the last time
the client synced. In such a case the room is marked as limited and
events need to be fetched separately.
When such a sync call happens our event index will have a gap. To
close the gap checkpoints are added to start crawling our room again.
Unnecessary full re-crawls are prevented by checking if our current
/room/roomId/messages request contains only events that were already
present in our event index.
This patch adds support to create an event index if the clients platform
supports it and starts an event crawler.
The event crawler goes through the room history of encrypted rooms and
eventually indexes the whole room history of such rooms.
It does this by first creating crawling checkpoints and storing them
inside a database. A checkpoint consists of a room_id, direction and
token.
After the checkpoints are added the client starts a crawler method in
the background. The crawler goes through checkpoints in a round-robin
way and uses them to fetch historic room messages using the
rooms/roomId/messages API endpoint.
Every time messages are fetched a new checkpoint is created that will
be stored in the database with the fetched events in an atomic way, the
old checkpoint is deleted at the same time as well.
