No description
  • Go 95.9%
  • Python 2.7%
  • Just 0.7%
  • Shell 0.3%
  • HTML 0.2%
  • Other 0.2%
Find a file
Repository files (latest commit first)
Filename Latest commit message Latest commit date
2026-09-15 14:02:57 +02:00
.github/workflows Merge pull request #269 from matrix-org/poljar/daily-test-runs 2026-08-05 16:19:02 +02:00
cmd Suppress errors encountered during RPC server shutdown 2026-06-19 11:19:49 +01:00
internal Merge pull request #277 from matrix-org/dependabot/npm_and_yarn/internal/api/js/js-sdk/postcss-8.5.27 2026-09-03 12:48:15 +01:00
tests mitmproxy: change config atomically 2026-08-06 17:37:38 +01:00
.gitignore Add Ruma to the git ignore files 2026-08-04 13:48:47 +02:00
CONTRIBUTING.md add DCO 2024-07-24 10:10:35 -07:00
ENVIRONMENT.md Docs: add hint on how to view mitm dump 2026-08-03 19:44:26 +01:00
FAQ.md Merge pull request #266 from matrix-org/poljar/just-for-rust 2026-08-04 14:07:40 +02:00
go.mod Update dependency go-archive 2026-09-04 10:31:32 +01:00
go.sum Update dependency go-archive 2026-09-04 10:31:32 +01:00
justfile Clarify that we're on a fork of uniffi-bindgen-go 2026-09-14 14:22:29 +02:00
LICENSE Add Apache 2.0 license 2024-05-30 10:06:34 +01:00
README.md Clarify that we're on a fork of uniffi-bindgen-go 2026-09-14 14:22:29 +02:00
rebuild_js_sdk.sh Add support for pnpm in install-js-sdk.sh (#223) 2026-02-13 13:01:37 +00:00
SECURITY.md Initial commit 2023-11-01 12:46:35 +00:00
TEST_HITLIST.md Update hitlist 2024-07-10 16:24:53 +01:00
uniffi.toml Fix the uniffi toml file 2026-08-31 14:45:24 +02:00

Complement-Crypto

As of May 2024 this project is being used in CI for existing SDKs. As such, it has a stable API exposed via a Github Action.

Complement Crypto is an end-to-end test suite for next generation Matrix clients, designed to test the full spectrum of E2EE APIs. It currently tests rust SDK FFI bindings and JS SDK, but can be expanded to support any client SDK.

Installing

Please ensure you have met Complement's Dependencies first. In practice, this means you must have go and docker installed.

Complement Crypto can be compiled and run in different modes depending on which SDK is being tested. For example, if you only want to test JS SDK then you do not need to compile rust code or run rust tests, and vice versa. Conversely, if you want to test interoperability between the two SDKs then you need to compile both SDKs.

This means you MUST compile at least 1 SDK in order for Complement-Crypto to compile and run.

JS SDK

Run ./rebuild_js_sdk.sh according to its help page:

Rebuild the version of JS SDK used. (requires on PATH: yarn)
Usage: ./rebuild_js_sdk.sh [version]
  [version]: the yarn/npm package to use. This is fed directly into 'yarn add' so branches/commits can be used

Examples:
  Install a released version:    ./rebuild_js_sdk.sh matrix-js-sdk@29.1.0
  Install develop branch:        ./rebuild_js_sdk.sh matrix-js-sdk@https://github.com/matrix-org/matrix-js-sdk#develop
  Install specific commit:       ./rebuild_js_sdk.sh matrix-js-sdk@https://github.com/matrix-org/matrix-js-sdk#36c958642cda08d32bc19c2303ebdfca470d03c1
  Install from a local checkout: ./rebuild_js_sdk.sh matrix-js-sdk@file:/path/to/local/js/sdk"

Rust SDK

Pre-requisites:

  • cargo installed and on your PATH
  • Install uniffi bindings for Go: just install-uniffi-bindgen - ensure uniffi-bindgen-go is on your PATH.

Note that we're currently using a fork of uniffi-bindgen-go to support Uniffi 0.32, pending its official release with Uniffi 0.32 support.

Run just rebuild-rust-sdk $PATH-TO-RUST-SDK. $PATH-TO-RUST-SDK must be a local checkout of the https://github.com/matrix-org/matrix-rust-sdk/ repo.

Running

Find a complement-compatible homeserver image. If you don't care which image is used, use ghcr.io/matrix-org/synapse-service:v1.117.0 which will Just Work out-of-the-box.

To run only rust tests:

COMPLEMENT_CRYPTO_TEST_CLIENT_MATRIX=rr \
COMPLEMENT_BASE_IMAGE=ghcr.io/matrix-org/synapse-service:v1.114.0 \
LIBRARY_PATH=$LIBRARY_PATH:/path/to/matrix-rust-sdk/target/debug \
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/matrix-rust-sdk/target/debug \
go test -v -count=1 -tags=rust -timeout 15m ./tests

To run only JS tests:

COMPLEMENT_CRYPTO_TEST_CLIENT_MATRIX=jj \
COMPLEMENT_BASE_IMAGE=ghcr.io/matrix-org/synapse-service:v1.114.0 \
go test -v -count=1 -tags=jssdk -timeout 15m ./tests

COMPLEMENT_CRYPTO_TEST_CLIENT_MATRIX controls which SDK is used to create test clients, and the -tags option controls conditional compilation so other SDKs don't need to be compiled for the tests to run.

To test interoperability between the SDKs, mitmdump the traffic, run extra multiprocess tests and more, see ENVIRONMENT.md for the full configuration options.

See FAQ.md for more information around debugging.

Test hitlist

There is an exhaustive set of tests that this repository aims to exercise. See TEST_HITLIST.md.

Architecture

Tests sometimes require reverse proxy interception to let some requests pass through but not others. For this, we use mitmproxy.

     Host        |       Containers
                 |                         
 +----------+    |    +-----------+         +-----+
 | Go tests | <--|--> | mitmproxy | <--+--> | hs1 |
 +----------+    |    +-----------+    |    +-----+
      |          |       ^             +--> | hs2 |
  +---V--------+ |       |                  +-----+
  | RPC Client |-|-------`
  +------------+ |    
  • Go tests can create clients inside the test process or use the Complement Go client to make CSAPI requests. On startup, the first test which calls Deploy will deploy the entire stack of homeservers/proxies/mitmproxy.
  • The RPC client is a single client (e.g JS SDK) which can make CSAPI requests. RPC clients are generally only made when testing multiprocess code or testing SIGKILL behaviour.
  • mitmproxy reverse proxies all the homeservers so all CSAPI traffic from a client goes through it. This enables requests/responses to be intercepted and modified, as well as traffic to be dumped to file.

Rationale

Complement-Crypto extends the existing Complement test suite to support full end-to-end testing of the Matrix Rust SDK. End-to-end testing is defined at the FFI / JS SDK layer through to a real homeserver, a real sliding sync proxy, real federation, to another rust SDK on FFI / JS SDK.

Why:

  • To detect "unable to decrypt" failures and add regression tests for them.
  • To ensure cross-client compatibility (e.g mobile clients work with web clients and vice versa).
  • To enable new kinds of security tests (active attacker tests)

Goals:

  • Must work under Github Actions / Gitlab CI/CD.
  • Must be fast (where fast is no slower than the slowest action in CI, in other words it shouldn't be slowing down existing workflows).
  • Must be able to test next-gen clients Element X and Element-Web R (Rust crypto).
  • Must be able to test Synapse.
  • Must be able to test the full spectrum of E2EE tasks (key backups, x-signing, etc)
  • Should be able to test over real federation.
  • Should be able to manipulate network conditions.
  • Should be able to manipulate program state (e.g restart, sigkill, clear storage).
  • Could test other homeservers than Synapse.
  • Could test other clients than rust SDK backed ones.
  • Could provide benchmarking/performance testing.

Anti-Goals:

  • Esoteric test edge cases e.g Synapse worker race conditions, FFI concurrency control issues. For these, a unit test in the respective project would be more appropriate.
  • UI testing. This is not a goal because it slows down tests, is less portable e.g needs emulators and is usually significantly more flakey than no-UI tests.

Github Action

To run tests for a single SDK, insert this action:

    complement-crypto:
        name: "Run Complement Crypto tests"
        uses: matrix-org/complement-crypto/.github/workflows/single_sdk_tests.yml@main
        with:
            use_js_sdk: "." # example is for JS SDK, use the current checkout

The complete options for with: are as follows:

  • use_js_sdk: Controls the source location of the JS SDK. Provide either a tag, commit or branch on matrix-org/matrix-js-sdk. Alternatively, if you have a local checkout somewhere, specify the path of the checkout e.g . for the working directory, or /full/path/to/js/sdk. Relative paths aren't supported.
  • use_rust_sdk: same as use_js_sdk but for the rust SDK.
  • use_complement_crypto: same as use_js_sdk but for Complement Crypto itself. Defaults to main.