Index messages locally in an encrypted on-device store so encrypted rooms can be searched at all (#7249)

* Add a message search abstraction over the Rust SDK's local index

Wraps the SDK's search service so the rest of the app never sees MatrixRustSDK
types, following the existing convention in libraries/matrix.

- MessageSearch, MessageSearchService, MessageSearchResult and
  MessageSearchPaginationState describe the surface in the api module.
- RustMessageSearch and RustMessageSearchService implement it, with
  MessageSearchResultMapper and MessageSearchResultsProcessor turning SDK results
  into Kotlin data classes.
- TantivyQueryEscaper escapes the query so a user typing ":)" or "AND" searches
  for that literal text rather than tripping the tantivy parser, and joins every
  word as a Must clause.
- Fakes and unit tests cover the mapping, the processor and the escaper, the last
  checked against a real tantivy 0.26.1 parser.

* Attach the encrypted search index behind a disabled feature flag

Gives the client an on-device tantivy index encrypted with the session's client
secret, so encrypted rooms can be searched at all.

- The index store is attached at ClientBuilder time via withSearchIndexStore, in
  a search-index directory alongside the session, keyed on the client secret. A
  session without one gets no index.
- MatrixClient exposes isMessageSearchAvailable so callers gate on the real
  indexing capability rather than on the raw flag, and getCacheSize() counts the
  index directory so it stays visible in the settings figures.
- FeatureFlags.MessageSearch (feature.message_search) is off by default and
  isFinished = false. It is read once at ClientBuilder time, so develop stays
  releasable and nothing reaches a user.

* Simplify search index wiring following review

Tie search availability to the MessageSearch feature flag alone and let
the index be created without a client secret, encrypting it only when
the session has one, matching the SDK's SQLite stores. Make
getBaseClientBuilder take the flag as a required parameter and return
ClientBuilder directly again, dropping the BaseClientBuilder wrapper
and the mid-restore snapshot parameter. Check the flag through
FeatureFlagService in RustMatrixAuthenticationService instead of
caching it in a field, and revert the extra matrix_sdk_search tracing
target, which belongs in the SDK's defaults.
This commit is contained in:
hayyaksi
2026-07-30 12:17:40 +03:00
committed by GitHub
parent af9db20f4f
commit e6d4472c0c
24 changed files with 1353 additions and 8 deletions
@@ -39,6 +39,7 @@ import io.element.android.libraries.matrix.api.room.location.BeaconInfoUpdate
import io.element.android.libraries.matrix.api.roomdirectory.RoomDirectoryService
import io.element.android.libraries.matrix.api.roomlist.RoomListService
import io.element.android.libraries.matrix.api.scanner.ContentScanner
import io.element.android.libraries.matrix.api.search.MessageSearchService
import io.element.android.libraries.matrix.api.spaces.SpaceService
import io.element.android.libraries.matrix.api.sync.SlidingSyncVersion
import io.element.android.libraries.matrix.api.sync.SyncService
@@ -70,6 +71,10 @@ interface MatrixClient {
val notificationSettingsService: NotificationSettingsService
val encryptionService: EncryptionService
val roomDirectoryService: RoomDirectoryService
/** Whether this live client was built with a message search index attached. */
val isMessageSearchAvailable: Boolean
val messageSearchService: MessageSearchService
val mediaPreviewService: MediaPreviewService
val matrixMediaLoader: MatrixMediaLoader
val sessionCoroutineScope: CoroutineScope
@@ -0,0 +1,41 @@
/*
* Copyright (c) 2026 Element Creations Ltd.
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial.
* Please see LICENSE files in the repository root for full details.
*/
package io.element.android.libraries.matrix.api.search
import kotlinx.collections.immutable.ImmutableList
import kotlinx.coroutines.flow.StateFlow
/**
* A single, stateful message search cursor: one query at a time, with paginated results.
*
* Obtain an instance via [MessageSearchService.createMessageSearch]. The instance is tied to the
* [kotlinx.coroutines.CoroutineScope] it was created with; cancelling that scope releases the
* underlying SDK resources, so there is no `close()` to call.
*/
interface MessageSearch {
/**
* The current result set for the active query, in the order supplied by the SDK.
*/
val results: StateFlow<ImmutableList<MessageSearchResult>>
/**
* Whether a page is currently loading, and whether the end has been reached.
*/
val paginationState: StateFlow<MessageSearchPaginationState>
/**
* Set (or update) the search query. Clears the current results, restarts pagination from
* scratch and loads the first page. Call [paginate] to load any further pages.
*/
suspend fun setQuery(query: String): Result<Unit>
/**
* Load the next page of results. No-ops if a page is already loading or the end was reached.
*/
suspend fun paginate(): Result<Unit>
}
@@ -0,0 +1,24 @@
/*
* Copyright (c) 2026 Element Creations Ltd.
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial.
* Please see LICENSE files in the repository root for full details.
*/
package io.element.android.libraries.matrix.api.search
/**
* Whether the search is currently loading a page of results.
*/
sealed interface MessageSearchPaginationState {
/**
* Not currently paginating. [endReached] is true once every source has been
* exhausted for the current query.
*/
data class Idle(val endReached: Boolean) : MessageSearchPaginationState
/**
* A page of results is currently being loaded.
*/
data object Loading : MessageSearchPaginationState
}
@@ -0,0 +1,27 @@
/*
* Copyright (c) 2026 Element Creations Ltd.
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial.
* Please see LICENSE files in the repository root for full details.
*/
package io.element.android.libraries.matrix.api.search
import io.element.android.libraries.matrix.api.core.EventId
import io.element.android.libraries.matrix.api.core.RoomId
import io.element.android.libraries.matrix.api.core.UserId
import io.element.android.libraries.matrix.api.timeline.item.event.EventContent
import io.element.android.libraries.matrix.api.timeline.item.event.ProfileDetails
/**
* A single message matching a search query, with its content and sender resolved.
*/
data class MessageSearchResult(
val roomId: RoomId,
val eventId: EventId,
val senderId: UserId,
val senderProfile: ProfileDetails,
val content: EventContent,
/** Origin server timestamp, in milliseconds since the epoch. */
val timestamp: Long,
)
@@ -0,0 +1,31 @@
/*
* Copyright (c) 2026 Element Creations Ltd.
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial.
* Please see LICENSE files in the repository root for full details.
*/
package io.element.android.libraries.matrix.api.search
import io.element.android.libraries.matrix.api.core.RoomId
import kotlinx.coroutines.CoroutineScope
interface MessageSearchService {
/**
* Create a new, independent search cursor.
*
* The underlying SDK service is stateful (one query at a time), so create one per search
* screen — never one per keystroke, and never a shared singleton.
*
* @param scope the lifetime of the search session. The returned [MessageSearch] observes and
* emits results on this scope, and the underlying SDK service is closed once the scope
* completes, so there is nothing to release by hand. Pass the scope of the screen that owns the
* search, not a longer-lived one.
* @param roomId when null, the search is session-wide and results from every room the user is
* in are exposed; when non-null, only results from that room are. Note that the SDK searches
* and ranks across every room regardless, so this is a client-side filter, not a pushdown: a
* room with few matches may need several [MessageSearch.paginate] calls before any of its
* results surface.
*/
fun createMessageSearch(scope: CoroutineScope, roomId: RoomId? = null): MessageSearch
}