Messaging

The program in src is called wallet, because it is single mindedly focused on key management and secrets management.

We want to add messaging capability. For this, we need libp2p and Gossip sub. We want to implement the simplest possible chat app to start with. We can add the capabilities to make it useful later, but we want to get chat working, and then we develop from there.

What is described here is pre-alpha communication, and it is important not to mistake it for Bitmessage. Gossipsub gives far less than Bitmessage gives: no proof of work against spam, no per recipient encryption, no deniability, no privacy against a peer observing who talks to whom, no global name space of addresses. It also gives far more than a pair of wallets on a trusted local network needs, where an ordinary connected socket and an agreed encoding would do. Both facts are fine, because the point of this first step is not a product. The point is to get communication happening at all, and to have a test environment that verifies that communication. The capabilities Bitmessage actually requires are developed on top of that, later.

The existing program uses a variety of C and C++ libraries, sqlite to store data, wxWidgets for UI.

Unfortunately, the C and C++ implementations of libp2p are lagging. Need to rust libp2p, and a rust application using libp2p must use Asynchronous concurrency, therefore must be written in rust.

Asynchronous concurrency does not, in theory, force Tokio on us: rust-libp2p’s swarm takes any spawner, and tokio is an optional feature of the libp2p crate. In practice it does, because the alternative to using Tokio is to re-invent Tokio — a work stealing scheduler, an io driver, timers, cancellation, and a channel library — which nobody wants to do. Tokio is therefore the de-facto standard for rust-libp2p applications, and this design uses it as such.

The application is written in C++. It is going to need libp2p and gossipsub, but an application to use libp2p and gossip sub has to be written in rust, using asynchronous concurrency and Tokio.

We are going to need to add Rust, Tokio, and Libp2p to the project.

The appropriate toolchain for combining C, Rust, and C++ through the C ABI is CMake plus Corrosion, the standard CMake/Rust integration project. Corrosion is vendored into the source tree like our other dependencies (wxWidgets, libsodium, mpir, utf8proc).

Plan for implementing chat with this program:

1 The overall shape

The natural direction is: the C++ wallet executable stays the process owner; Rust compiles to a static library (staticlib) and is linked into it across a C ABI. Reasons:

• libp2p-rs is async-only, so the Rust side must own an asynchronous runtime, and that runtime is Tokio. But a runtime is just a thread pool — it doesn’t need to own the process. A staticlib can host it.
• The wxWidgets UI is long-lived and single-threaded-evented. C++ calls into Rust through a handful of extern “C” functions; Rust never calls back into C++.
• The alternative — a separate Rust sidecar process talking over a UDS — is also viable and gives crash isolation, but it adds a second binary, a wire protocol, and process-lifecycle management. For “simplest possible chat,” staticlib is simpler; you can always migrate to a sidecar later without touching the chat UI, because the C-ABI-shaped boundary already exists. This is also why server is revised to propose a unitary program: one executable is easier to deploy and easier for the end user to understand.

Design rules that keep the boundary safe:

• Opaque *mut handles only; no Rust types, exceptions, std::string, or std::vector across the boundary.

• Strings: Rust allocates, C++ must call the matching free function.

• Every C-ABI entry point must be non-blocking, because they are called from the net thread (see below), whose whole job is marshalling: a stalled network must not be able to freeze the UI, and must not be able to wedge the net thread either. Internally they just hand a message to a bounded channel with try_send (never a blocking send) and return.

• No C-ABI call is ever made on the wxWidgets main thread. The wallet runs under the wxWidgets event mechanism; the Rust library runs under Tokio’s. The two are entirely incompatible, and any close temporal coupling between them is bound to break something, so the design puts as much distance between them as possible. All traffic across the C ABI is carried by a secondary wxWidgets thread — the net thread — which

  • calls into Rust through the extern "C" functions, and never the reverse: Rust never calls back into C++; and
  • talks to the main UI thread only through the wxWidgets inter-thread event mechanism (wxThreadEvent posted with wxQueueEvent), in both directions.

  The main thread posts “send this” to the net thread; the net thread try_sends it into Rust. Inbound messages are drained by the net thread’s own loop and posted to the main thread as events. Neither thread waits on the other, the UI thread never touches a Rust handle, and the Rust side never sees a wx object. The objective is the greatest possible decoupling between two event mechanisms that have nothing in common.

• Errors: int codes plus a last_error string getter. No panic may escape an extern “C” function: every entry point is wrapped in catch_unwind, and a caught panic becomes an error code and a last_error string. (A panic escaping FFI aborts the process on modern Rust — unacceptable for the wallet.)

2 The C ABI, specified

The surface is tiny and boring. Hand-written net/wallet_net.h (~60 lines; cbindgen is optional later if it ever drifts):

#ifndef WALLET_NET_H
#define WALLET_NET_H
#include <stdint.h>
#include <stddef.h>

typedef struct WalletNet WalletNet;         /* opaque handle */

/* One inbound chat message. All strings are UTF-8, NUL-terminated,
   Rust-allocated. Owner must call wallet_net_free_message(). */
typedef struct WalletNetMessage {
    char    *from;        /* peer id as a multibase string */
    uint64_t ts_millis;   /* sender-claimed unix timestamp, ms */
    char    *text;        /* message body, UTF-8 validated by Rust */
} WalletNetMessage;

/* Error codes */
enum {
    WALLET_NET_OK          =  0,
    WALLET_NET_ERR         = -1,   /* see wallet_net_last_error() */
    WALLET_NET_ERR_BUSY    = -2,   /* outbound queue full; retry later */
    WALLET_NET_ERR_ARG     = -3,   /* null/invalid argument */
    WALLET_NET_ERR_TOO_BIG = -4    /* input exceeds size limit */
};

uint32_t     wallet_net_abi_version(void);            /* returns 1 */
WalletNet   *wallet_net_start(const char *config_json);
int          wallet_net_stop(WalletNet *h);
int          wallet_net_send(WalletNet *h, const char *topic, const char *text);
int          wallet_net_poll(WalletNet *h, WalletNetMessage *out, size_t max, size_t *n);
void         wallet_net_free_message(WalletNetMessage *msg);
const char  *wallet_net_last_error(void);
#endif

Semantics, precisely:

• wallet_net_start is idempotent: the first call builds the runtime and returns a new handle; every subsequent call returns the same handle (and does not disturb the running instance). Returns NULL on failure, with wallet_net_last_error() set. config_json carries the listening multiaddr, the identity key material, and the initial topic.
• wallet_net_stop sends a Stop command, shuts the runtime down with a bounded timeout (see below), frees the handle, and poisons the singleton so a later wallet_net_start creates a fresh instance. Called from the net thread, never from the wx main thread; worst case it blocks the net thread for the shutdown timeout (2 s). The main thread’s only involvement at app exit is to ask the net thread to finish and then join it, so the UI is never the thing that waits.
• wallet_net_send is non-blocking: it try_sends onto a bounded command channel. If the channel is full it returns WALLET_NET_ERR_BUSY immediately; the caller retries on the next tick of the net thread’s loop. Text longer than the 64 KiB limit returns WALLET_NET_ERR_TOO_BIG.
• wallet_net_poll drains up to max inbound messages into the caller-provided array and sets *n to the number written. It never blocks; if the inbox is empty, *n = 0. Each message the caller receives is owned by the caller and must be released with wallet_net_free_message. Rust validates UTF-8 before enqueueing, so C++ may use wxString::FromUTF8 unconditionally.
• Thread safety: the handle is safe to use from multiple threads concurrently (start/stop from the net thread only; send/poll are internally synchronized via the channels and a mutex- protected inbox). All entry points are catch_unwind-wrapped.
• Encoding: message bodies on the wire are a fixed, versioned binary encoding (or JSON, decided in implementation) of {from, ts_millis, text}; from is filled in by the receiver from the libp2p propagation source, never trusted from the payload.

3 Building with Corrosion

CMake has no built-in find_package(Rust), no add_rust_library(), and its native enable_language(Rust) remains experimental. The standard solution is Corrosion, which wraps cargo from CMake and creates proper imported targets. We vendor it: corrosion/ alongside wxWidgets/, libsodium/, etc.

• Corrosion v0.6 requires CMake ≥ 3.22. Only the v0.5 line, which takes critical backports only, works with CMake 3.15. The top-level CMakeLists.txt said cmake_minimum_required(VERSION 3.5), which CMake 3.31 warns about and CMake 4.x rejects; it is bumped to 3.22 as part of this work — not to 3.15, which would be insufficient for a vendored current Corrosion.
• Bumping the top level is not the whole of it. The vendored subprojects declare their own minima (libsodium 3.11, mpir 3.10, utf8proc 3.10, sqlite3 3.10), and each of those also has to be raised, or expressed with the 3.22...3.31 range syntax, before a CMake 4.x configure succeeds.
• Bumping the top level also changes policies for the wallet target itself. The one that bites is CMP0091: with the old minimum, MSVC_RUNTIME_LIBRARY is ignored and MSVC’s default (/MD) is used; with a minimum of 3.22 the property takes effect. See the Windows section below.
• Cargo remains the source of truth for the Rust dependency graph. Corrosion’s job is only: run cargo at build time and surface the static library (and its link requirements) as a CMake target. Do not try to model per-crate dependencies in CMake.

  add_subdirectory(corrosion)
  corrosion_import_crate(MANIFEST_PATH net/Cargo.toml PROFILE release)
  target_link_libraries(wallet PRIVATE walletnet)

• The function is corrosion_import_crate. corrosion_import_cargo is the old name and does not exist in current Corrosion. PROFILE is a valid argument, and requires rustc ≥ 1.57.
• The imported target is named after the crate, with dashes replaced by underscores (Rust 1.79’s name normalization, which Corrosion mirrors). A crate called wallet-net therefore gives a target called wallet_net, not walletnet. Name the crate what we want the CMake target to be called, and read the target names Corrosion prints at configure time rather than assuming them.

Notes on the ecosystem pieces:

• Corrosion propagates the system libraries a Rust staticlib needs (dl, pthread, m on Linux; the correct CRT linkage on Windows), which removes the classic hand-rolled-link failures. How: at configure time it compiles a probe with rustc and reads back the required native libraries and link flags, then sets them as the imported target’s INTERFACE_LINK_LIBRARIES / INTERFACE_LINK_OPTIONS. Two consequences — a working Rust toolchain is needed to configure, not just to build; and the answer is cached per target triple, so it has to be re-derived when cross-compiling or when the toolchain changes.
• The Rust crate needs crate-type = ["staticlib", "lib"] — staticlib for linking into wallet, lib so cargo test works. Corrosion only creates targets for staticlib, cdylib, and bin kinds, so the extra "lib" is invisible to CMake. That cargo test genuinely needs the "lib" entry is an assumption to be tested in phase 1, not an established fact.
• Vendoring Corrosion: check that a plain in-tree copy — no git metadata, no network — configures and builds. The documented paths are FetchContent and add_subdirectory; we want the latter, offline, matching how every other dependency here is vendored.
• Vendoring the Rust dependencies: cargo vendor + a .cargo/config.toml pointing at the vendor dir, committed Cargo.lock. The project already vendors its C libraries; doing the same for Rust deps keeps builds offline and reproducible and matches the project’s existing style. Two things to establish before committing to it: that the vendored libp2p graph really does build offline, and that we are willing to commit it at all — it is hundreds of crates, an order of magnitude more files than any other vendored dependency in this tree.
• The Rust dependency graph is not pure Rust. libp2p-noise depends on snow with the ring-resolver feature, so ring is in the graph: C and assembly compiled by the cc crate. Plan for three consequences — a second cryptographic stack in the binary alongside libsodium; NOTICE and license review covering ring and the rest of the crates; and native code inside the Rust static library, which is precisely the case that breaks MSVC Debug builds.
• Toolchain prerequisite: Rust must be installed. Verified on the development machine: rustc/cargo 1.98.1 (/usr/bin). rust-libp2p master declares rust-version = "1.88.0" and edition = "2024", which 1.98.1 comfortably exceeds. Still to be checked once the libp2p version is chosen: the MSRV of that released version, and that an equally recent toolchain exists on every machine that has to build the wallet.
• Pin the minimum with rust-version = "1.88" in net/Cargo.toml, which any toolchain honours. Do not rely on net/rust-toolchain.toml: that file is a rustup feature and is inert for a distro-packaged /usr/bin/cargo — which is exactly what the development machine has.
• Release profile tuning in Cargo.toml (lto = "thin", codegen-units = 1) to keep the static lib from bloating the binary; libp2p pulls in a lot of crates. Check that these interact sanely with the cc-built crates (ring) and with PROFILE release under a multi-config generator, where the C++ side’s Debug configuration and the Rust side’s release profile can otherwise silently disagree.
• cbindgen is optional; at this ABI size the hand-written header above is fine.

4 How the Tokio runtime is launched and shut down

The net thread calls wallet_net_start() once, when it starts — that is, at app start, or when the user opens the chat panel and the main thread asks for a net thread to be created. The runtime is a multi-thread Tokio runtime; it spawns its own worker threads. The long-lived libp2p task is spawned onto it with rt.spawn() — there is no separate block_on thread, and no thread joining at shutdown. Sketch:

static INSTANCE: Mutex<Option<Arc<NetHandle>>> = Mutex::new(None);   // idempotent start

pub extern "C" fn wallet_net_start(cfg_json: *const c_char) -> *mut NetHandle {
    ffi_catch(|| {
        let mut inst = INSTANCE.lock().unwrap_or_recover();
        if let Some(h) = inst.as_ref() { return Arc::as_ptr(h) as *mut _; }  // same handle

        let cfg = parse_config(cvt(cfg_json)?)?;
        let rt = tokio::runtime::Builder::new_multi_thread()
            .enable_all()
            .build()?;                                   // error -> last_error, NULL out
        let (tx, rx) = tokio::sync::mpsc::channel::<Command>(64);
        let task = rt.spawn(net_main(rx, cfg));           // the heart of the async world
        let h = Arc::new(NetHandle { rt, task, tx, inbox: Mutex::new(VecDeque::new()) });
        *inst = Some(h.clone());
        Ok(Arc::into_raw(h) as *mut NetHandle)
    })
}

async fn net_main(mut rx: mpsc::Receiver<Command>, cfg: Config) -> anyhow::Result<()> {
    let mut swarm = /* tcp + noise + yamux + gossipsub Swarm from libp2p */;
    swarm.behaviour_mut().subscribe(&cfg.topic)?;
    loop {
        tokio::select! {
            ev = swarm.select_next_some() => {
                if let SwarmEvent::Behaviour(Gossipsub::Message { propagation_source, message }) = ev {
                    // validate UTF-8 + size limit, then push {from, ts, text}
                    // into the handle's inbox for wallet_net_poll to drain
                }
            }
            cmd = rx.recv() => match cmd {
                Some(Command::Send { topic, text }) => { let _ = publish(&mut swarm, topic, text); }
                Some(Command::Stop) | None => break,
            }
        }
    }
    Ok(())
}

The call chain, from top to bottom:

1. The main UI thread creates the net thread (a wxThread) and, from then on, communicates with it only by posting wxThreadEvents with wxQueueEvent. The net thread calls wallet_net_start(), which returns almost instantly: it built a runtime and spawned one task.
2. net_main is the thing that drives everything: it owns the libp2p Swarm, and its select! loop runs as a task on the runtime’s worker threads — Noise handshakes, connection manager, gossipsub receive/publish. libp2p-rs is an actor you poll; this task is the heart.
3. Commands (send, stop) travel from the UI thread to the net thread as wx events, and from the net thread down a bounded channel via try_send — never blocking_send, so a full channel returns BUSY and the net thread retries, instead of anything blocking.
4. Inbound chat messages travel up into a mutex-protected inbox. The net thread’s own loop (say every 100 ms) calls wallet_net_poll(), and each drained message becomes a wxThreadEvent posted to the UI thread with wxQueueEvent. (The push alternative — Rust calling a C-registered function pointer — also works and is lower-latency, but polling from the net thread keeps reentrancy away from the FFI boundary entirely, and the UI thread sees nothing but an ordinary wx event arriving in its own queue. For v1 chat, 100 ms latency is invisible.)

So the two event mechanisms meet only at the net thread: wx events on one side of it, the C ABI on the other, and no thread that has to satisfy both. The UI thread never holds a Rust handle and never waits on Rust; Rust never sees a wx object and never calls into C++.

Shutdown runs the same path in reverse: the net thread calls wallet_net_stop(), which sends Command::Stop down the channel (so net_main can exit cleanly and let in-flight work unwind), then calls rt.shutdown_timeout(Duration::from_secs(2)). This waits up to two seconds for all tasks — including the runtime’s own worker threads — to finish, then drops the runtime. No join, no detach, no unbounded wait. The handle is freed, the singleton slot is cleared, and the net thread then exits; the main thread joins it at app exit.

Three rules that keep this sane:

• One runtime per process. Never a second one, never one per feature. wallet_net_start enforces this by returning the existing handle on repeat calls.
• Never block_on on a thread the wx main thread can wait on (it would freeze the UI), and never block_on from inside a runtime task (Tokio forbids it and it deadlocks). With the spawn + shutdown_timeout pattern there is no block_on anywhere.
• The FFI boundary is the only place the two worlds synchronize, and the net thread is the only thing that crosses it; everything else is channels and queues. That means the Rust side is fully unit-testable with cargo test without C++ at all.

5 The chat itself, deliberately minimal

• Identity: each peer’s libp2p identity keypair. rust-libp2p wants to be handed thirty-two bytes, from which it derives the Ed25519 keypair that is the peer identity (libp2p_identity::Keypair::ed25519_from_bytes). The wallet already produces exactly such a value: ristretto255::extended_private_key::thirty_two_byte_identifier() (src/secrets.h, src/secrets.cpp) — the hash<256> of the extended private key in its machine-independent serialized form, that is, of both the secret scalar and the chain code. It is already exposed over the wallet’s own C ABI, in src/c_abi.h, as wallet_thirty_two_byte_identifier_of_loaded_extended_private_key(uint8_t identifier_out[32]), whose all-zeroes return means “no wallet is open”, with an error message already queued for the user.

  Note the mismatch that makes this a derivation and not a reinterpretation: the wallet’s key machinery is built around Ristretto255 (ristretto over Curve25519), while libp2p peer identities are Ed25519 (or secp256k1/ECDSA). These are different groups over related curves — a ristretto255 element is not an Ed25519 public key and must not be reinterpreted as one. Handing the thirty-two-byte identifier to rust-libp2p as a seed is safe in exactly that sense: libp2p derives a fresh Ed25519 keypair from those bytes, and no ristretto255 key material is reinterpreted as anything.

  Because the identifier is a function of the loaded extended private key, this gives the nice property that the network identity is wallet-managed and recoverable from the seed phrase, while keeping it cryptographically separate from coin/spending keys — the network identity is broadcast to peers, so it must never be a key used for anything else.

  Two consequences of getting it through the existing C ABI:

  • src/c_abi.h requires its functions to be called on the main (GUI) thread, because they touch UI state (which wallet is open) and may queue a message box for the user. So the identifier is fetched on the main thread before the net thread is started, and handed to the net thread, which passes it down to Rust in the identity field of config_json. At steady state, therefore, no C-ABI traffic touches the main thread at all — the rule of the previous section is not bent, it is satisfied by ordering. If the open wallet changes while the net thread is running, the main thread tells the net thread to stop and restart, again by wx event; it does not call into Rust itself.
  • src/secrets.h documents the identifier as non-secret, something that “can be displayed, logged, or transmitted”, because a hash of a secret does not reveal it. Using it as the seed of a long-lived private key contradicts that. Whether it remains displayable, or whether the libp2p seed is derived from it by a further step, is a crypto-review question and is deferred with the rest of the identity work.

• Transport: TCP + Noise + yamux to start, with QUIC v1 beside it on the same port number. No DHT, no rendezvous server for v1. Peers are added by pasting a multiaddr / contact string; the user model is “add this peer by address.” (On the local net, mDNS discovery is a trivial later addition since multicast works there; it is deliberately out of scope for the minimal system.)

• Chat: one gossipsub topic, messages are {from, ts, text}. Per-chat topics, signed topics, and history later.

6 The version object, exchanged when each peer joins

A connection report says only that a peer is there. Onaya’s answer to “what is that peer” is its version message, sent on every connection before anything else; the wallet speaks the same message — same field layout, little-endian, onaya’s compact-size prefixes for the variable fields — over its own libp2p substream protocol, /rho/version/1.0.0 (net/src/version.rs): the dialer opens one substream per connection, writes its object and reads the reply; the listener answers and never initiates, so every connection carries exactly one exchange in each direction. It is a dedicated protocol rather than a gossipsub message because the greeting belongs to the connection: on the chat topic a join greeting would reach every peer instead of the one that joined, and the duplicate cache would eat the repeats.

The message as rho sends it:

The wire format carries the peer’s own account and cannot carry the two things it cannot know about itself: the public address its traffic arrives at once a NAT has rewritten it, and the wire its packets actually rode. So the report to the UI (wallet_net_peer_versions) adds fields the receiver fills from its own connection — observed, the ip:port the packets really came from, and transport, the stack that socket rides: the wire named by the connection’s own multiaddr (tcp, circuit, quic…) with this build’s crypto beside it (noise over TCP and circuits; TLS is QUIC’s own by specification; this build listens on QUIC v1 at the TCP listen port’s number over UDP). The Peers page shows the observation and the stack, not the claim; both are kept and traced, and the claim sitting next to the observation is more honest than either alone — the difference between them is the NAT.

Two decisions differ from onaya’s spec on purpose. No nonce: the field exists to catch “you have connected to yourself”, and in this stack that case cannot arrive — the identity is one deterministic keypair per wallet, libp2p authenticates the noise handshake against the peer id, and the swarm refuses a connection to one’s own id; the 8 bytes would carry no actionable statement. Nothing is enforced except the clock: version and services are recorded, traced (-T prints one line per exchange, including “no version object” from a peer that does not speak the protocol) and reported to the UI through wallet_net_peer_versions; a peer running an unknown version is still a peer, and chat must keep working between any two builds until some later rule is written on purpose. The timestamp, however, is a promise about the sender’s clock that both ends can check, and a peer off by an hour or more (MAX_CLOCK_SKEW_SECS in net.rs) is disconnected and left undialed for a minute (CLOCK_REJECT_HOLD_OFF) — a five-second re-dial would only re-run the same handshake against the same broken clock, and sixty seconds is short enough that a conversation recovers on its own once the clock does.

The mechanism is a hand-written ConnectionHandler (dcutr’s shape, ReadyUpgrade + OutboundSubstreamRequest), not libp2p-request-response and not libp2p-ping: neither crate is in net/vendor, and the offline-by-configuration build cannot go and get them. Hand-writing the compact-size codec is the same argument in miniature — the vendored unsigned-varint is LEB128, which is not onaya’s encoding, and a two-byte disagreement of that kind is invisible on the wire.

7 Message inventory: the two-day mempool

Gossipsub is a flood, not a store. Its history spans five heartbeats — seconds — its duplicate cache 60 seconds, and both live in private fields with no application API (libp2p-gossipsub-0.50.0/src/config.rs, mcache.rs: checked, not assumed). A peer that joins late therefore hears nothing that was said before it arrived. The inventory is the missing layer: every message this node has seen — received, pulled, or sent — is kept for 48 hours under its content hash, and each new connection compares inventories and fills the gaps (net/src/inventory.rs).

Why not inside gossipsub. The protocol does carry IHAVE/IWANT, but only between its own handlers, only for ids inside the ~5-heartbeat window, and only as an internal optimisation — no public call lets an application ask what a peer holds or fetch by id. The alternative of publishing inventory lists as chat messages would flood every peer with every join and still deliver old messages wearing new propagation sources. So the sync is its own substream protocol, /rho/inventory/1.0.0, in the dialer-initiated style of the version exchange: offer (IHAVE), ask the difference (IWANT), hand over the bytes (MSG), each round closed by an END frame — four rounds on one full-duplex stream, both directions filled by one connection.

Ids are sha256(wire bytes) — the digest gossipsub’s own message_id_fn already uses, so a message arriving live and the same message arriving by sync are one key, and delivery has exactly one rule everywhere: a message reaches the app when storing it was news to the inventory. That gate also outlives gossipsub’s 60-second duplicate cache: a message that arrives again later is not said twice.

Attribution travels beside the bytes, never inside them. Each stored message carries an origin — the peer id this node believes wrote it: its own id for a message it published, the propagation source for one received live, and the server’s stored origin for one pulled by sync (the MSG frame carries the sidecar). Inside the wire is forbidden: the id is a function of the wire alone, and an embedded author field would give one message two identities — the live copy and the synced copy hashing apart — which is exactly the double-delivery the one rule exists to prevent. The sidecar is honest best-effort, like every label in an anonymous chat: a direct neighbour of the author stores the author, a node further out stores the first hop it saw, and a peer could lie about an origin exactly as easily as about the text — so it is trusted exactly as far as the text is, which is to say: for display. Without it, every message pulled from one peer wore that peer’s name, which read as “everything came from the courier”.

Pull only, no re-flood. A synced message is delivered to the local app and never republished into the mesh: every peer missing it will ask for it over its own connection, so a flood would only re-echo to holders and dress the message in the republisher’s source. A peer running an older build never speaks the protocol, syncs with nobody, and gains nothing from a republish — the documented boundary, not a case to relay for.

Retention runs from the earlier of two clocks. A message lives 48 hours from the earlier of when this node first stored it and the ts its author wrote into it, and one already past that is refused as old news. The local clock alone let a message one node had purged come back from a peer that pulled it later, re-shown and re-dated, so staggered nodes could pass it around forever. The author’s ts alone would let a timestamp from the future make a message immortal. The minimum can only shorten a life: ts is inside the hashed bytes, so only the author chooses it. The runtime store and the database purge on the same rule.

The database is its own file. inventory.sqlite sits beside the wallet files, never inside one (the user’s rule: chat plaintext earns no place in the file holding the wallet’s public identity). The runtime is the live authority — a sync must answer IHAVE in microseconds — and the C++ side mirrors it: wallet_net_inventory_new drains newly-stored lines once a second for INSERT OR IGNORE, and at startup wallet_net_inventory_seed hands every surviving row back, each re-hashed before it is trusted (a file on disk is data, not authority).

Startup replays the carried-over conversation, in the order it was said. Seeded rows and sync-pulled batches are stored undelivered; wallet_net_inventory_flush (called once after seeding, and automatically after every sync) hands the whole undelivered set to the inbox sorted by wire timestamp — storage order is database order, and a log reads in the order things were written. A live message is never part of a backlog: it is delivered the moment it is stored. On the window side the log keeps its lines in a timestamp-ordered memory: an arrival older than the last line drawn — a weekend pulled back by a sync, a startup replay landing after the “(topic …)” notes — is inserted at its timestamp and the log replayed once per burst, so history lands between the lines it belongs between instead of after everything. A message whose origin is this node’s own id draws in the own-message style: after a restart, what he wrote reads as what he wrote.

Caps: 2048 ids per announcement (newest kept; the rest wait for the next connection), 8 MiB per sync, messages bounded by the chat limit, a 30 s idle timeout per frame within a ten-minute bound per sync, and a 10 000-line persist queue whose overflow is a trace-visible loss of durability, not of service.

The Peers page shows the protocol’s own accounting (wallet_net_inventory_stats): cumulative ids announced and asked, messages pulled and served, and a gauge over asked-versus-arrived that is full when idle and advances during a pull.

8 Development phases

We want the absolute minimal basic system working first. The first step is not the minimal viable product — it is the minimal system that compiles all the libraries and gets them to perform.

1. Build integration. Vendor Corrosion; create net/ with a trivial Rust staticlib exposing wallet_net_abi_version(); wire it into CMake; wallet calls it at startup and prints/asserts the version. Success = wallet builds and links with the Rust lib on Linux, and the ABI call returns 1. Windows/MSVC is deliberately not part of this criterion; see the next section. Also settle here: that a vendored Corrosion configures offline, that crate-type = ["staticlib", "lib"] really is what cargo test needs, and what target name Corrosion actually creates for the crate.
2. Runtime lifecycle. Implement wallet_net_start/wallet_net_stop with the Tokio runtime but no libp2p yet (net_main just waits for Stop), plus the net thread itself and the wx inter-thread events on both sides of it. Success = start/stop repeatedly, driven from the UI thread by posted events and executed on the net thread, without hangs, leaks, or crashes; second start returns the same handle; and no C-ABI call is ever observed on the main thread.
3. libp2p on the local net. Add tcp + noise + yamux + gossipsub; two wallet instances on the LAN connect by pasted multiaddr and exchange messages through one topic, verified first via a cargo test two-node in-process test, then via two real wallet processes. Send/poll/stop wired through the C ABI with a minimal chat panel (text box; the net thread polls and posts events). This is the pre-alpha milestone: it proves that communication happens and gives us the test environment that verifies it. It is not yet a messaging product.
4. Only then: the product-level work — contacts, per-chat topics, message history, and the real-internet issues (NAT, hole punching, discovery; see nat.md) when the test environment moves off the local net. On the local net all ports are open and multicast works, so none of that machinery is needed yet; we cross those bridges when we come to them. Everything that Bitmessage requires and gossipsub does not provide — proof of work, per recipient encryption, deniability — belongs here and beyond, not in phases 1 to 3.

9 Windows and MSVC: known problems, deliberately deferred

The plan is to get all of this building, linking, and running on Linux first. Windows/MSVC integration is a substantial piece of work in its own right, and the following known problems are recorded here so that they are not rediscovered later, and then deliberately not solved until Linux works:

• The Windows build does not use the top-level CMakeLists.txt at all. msvc/win.bat and msvc/winConfig.bat build the dependencies with CMake but build the wallet itself with msbuild msvc/wallet.sln, from a hand-written .vcxproj that lists every source file explicitly. Corrosion is a CMake integration, so on Windows either the .vcxproj grows a cargo custom-build step and links the resulting walletnet.lib by hand, or the Windows build moves to CMake. Neither is done.
• The .vcxproj is maintained by hand and is already behind. It does not list src/c_abi.cpp, so the C ABI this design depends on is missing from the Windows build today.
• CRT mismatch, three ways. rustc always links the non-debug Windows runtime on *-msvc targets. The CMake build sets MSVC_RUNTIME_LIBRARY to MultiThreaded/MultiThreadedDebug (static CRT, /MT, /MTd) — currently a no-op, because with cmake_minimum_required(VERSION 3.5) policy CMP0091 is OLD, and the bump to 3.22 switches it on; the .vcxproj uses MultiThreadedDLL/MultiThreadedDebugDLL (/MD, /MDd). Only /MD Release agrees with rustc’s default. Expect LNK2038 RuntimeLibrary and LNK4098 defaultlib until these are reconciled, and expect the CMake bump to change the Windows CRT rather than leave it alone.
• C code in the Rust graph makes the Debug case worse. This is exactly what Corrosion’s own “Common Issues” page warns about: with ring compiled by the cc crate into the Rust static library, Debug C++ linked against it mismatches on both RuntimeLibrary and _ITERATOR_DEBUG_LEVEL. The documented workarounds are to build the C++ side against the non-debug runtime, or to pass CFLAGS/CXXFLAGS to the cc crate with corrosion_set_env_vars.
• Whether Corrosion’s automatic system-library propagation does the right thing for the MSVC CRT and the msvcrt.lib default-library directives is unverified.
• Cross-language debugging (C++ hitting a Rust frame) is assumed to work with debug builds on both sides; PDB generation for a Rust staticlib linked into an MSVC binary is unverified.
• Do not mix a MinGW-built wallet with an MSVC-targeted Rust staticlib; mismatched CRTs are a classic source of obscure LNK errors. If the wallet is built with MSVC, the Rust side targets x86_64-pc-windows-msvc.

10 Pitfalls to keep on the radar

• Windows: see the section above. The short version is that the Windows build is not a CMake build today, the CRT settings disagree with rustc’s in three directions, and ring puts C code in the Rust graph. All of it is deferred until Linux works.
• Undefined-symbol errors at link time are usually a missing system library or CRT mismatch, not a discarded-symbol problem; a plain staticlib whose #[no_mangle] symbols are referenced from C++ does not normally need --whole-archive / /WHOLEARCHIVE. Know the fix, but expect not to need it. That expectation holds only while every exported symbol really is referenced from C++: a crate that registers itself at link time (#[used], inventory, ctor-style registration) would change the answer, so re-check it once libp2p and its dependency graph are actually linked.
• Release profile tuning in Cargo.toml (lto = "thin", codegen-units = 1) to keep the static lib from bloating the binary; libp2p pulls in a lot of crates. Do not set panic = "abort" — the catch_unwind wrappers at the FFI boundary depend on unwinding.
• Don’t let CMake try to model the Cargo graph — resist the urge to write add_dependencies per-crate. Corrosion runs cargo; cargo handles the rest.
• Debuggability: CMAKE_EXPORT_COMPILE_COMMANDS covers the C/C++ side; rust-analyzer covers the Rust side (it reads Cargo.toml directly, no CMake involvement). Cross-language debugging (C++ hitting a Rust frame) is assumed to work with normal debug builds on both sides; verified on neither platform yet, and on Windows it depends on the PDB question above.

11 Deliberately not decided yet

The following are known open questions that this document does not answer, on purpose. Answering them before anything builds is overplanning: each gets settled when we have something building, or when we are trying to get something to build.

• The wire encoding of {from, ts_millis, text} — fixed versioned binary or JSON — and with it the real per-message size budget, which is capped from above by gossipsub’s transmit limit rather than by anything we choose.
• How much of the local-network test environment’s good fortune is real: whether the LAN actually allows peer-to-peer TCP and multicast, what the host firewall does to an unsigned wallet, and therefore whether mDNS is really the trivial later addition it is assumed to be.
• Topic naming, and what wallet_net_send does when the peer is not connected or the topic is not yet joined.
• Bounds and backpressure on the inbound side: the outbound command channel is bounded and returns BUSY, but the inbox the net thread drains is currently described as an unbounded queue.
• What the UI does with BUSY, and how a failed publish is made visible to the user at all.
• Whether the cargo test two-node test tells us anything about the shipped configuration, since it runs under cargo’s own profile and toolchain rather than Corrosion’s PROFILE release.
• Reconciliation with the rest of the design docs. server.md and peer_socket.md have been revised to match this plan; TCP.md, nat.md, protocol_negotiation.md, and the body of peer_socket.md have not, and still describe other transports and other layerings.
• Repo-level mechanics that only matter once the build is being changed: whether wallet’s existing plain-signature target_link_libraries line has to become a keyword signature to accommodate PRIVATE walletnet; where net/ and the vendored crates go in the tree; what NOTICE has to say about them; and what the chat panel does, in wxWidgets terms, beyond receiving posted events.

The summary sentence for the design doc: wallet (C++) owns the process and the UI; it links a Rust staticlib built by vendored Corrosion under CMake ≥ 3.22, and talks to it through ~7 non-blocking extern “C” functions with a specified message struct and ownership rules — always from a secondary wxWidgets net thread, never from the main UI thread, which exchanges wx inter-thread events with that thread and nothing else. The Rust library hosts a single Tokio runtime whose one long-lived spawned task is the libp2p/gossipsub swarm loop, created (idempotently) by wallet_net_start() and torn down by wallet_net_stop() via shutdown_timeout. What it delivers is pre-alpha communication: the first step towards Bitmessage’s capabilities, and a test environment that proves communication happens.

 reaction.la gpg key 154588427F2709CD9D7146B01C99BB982002C39F
This work is licensed under the Creative Commons Attribution 4.0 International License.