logo
vision scalability social networking revelation source
consensus immutability Bitcoin mixer snarks

Petnames and messages

abstract: This is an incoherent and fragmentary work in progress. Pleas ignore

We have key managment and secrets management functionality in src, which is useless by itself.

We need tables and data structures to represent and manage messages and signatures, and a UI resembling that of Bitmessage. But Bitmessage does not support identity, so we need UI resembling that of thunderbird, to access and generate messages by identity.

This is an incoherent and fragmentary work in progress.  Pleas ignore\

1 Existinq sql schema

The schema below is the WALLET_SCHEMA_SQL constant in src/db_accessors.h, shared by Frame::NewWallet (which wraps it in a BEGIN IMMEDIATE TRANSACTION / COMMIT pair after issuing the header pragmas) and by migrate_wallet_keys_foreign_key in src/db_accessors.cpp, which rebuilds the Keys table of wallet files created before "id" became a real foreign key. PRAGMA foreign_keys is a per-connection setting that is not stored in the file, so the ISqlite3Impl constructor (src/ISqlit3Impl.cpp) issues it on every connection it opens.

PRAGMA encoding = 'UTF-8';
PRAGMA journal_mode = WAL;
PRAGMA synchronous = 1;
PRAGMA application_id = 618087585;
PRAGMA user_version = -477629973;
PRAGMA foreign_keys = ON;
BEGIN IMMEDIATE TRANSACTION;
CREATE TABLE IF NOT EXISTS "Names"(
    "ROWID" INTEGER PRIMARY KEY,
    "name"  TEXT NOT NULL UNIQUE
) STRICT;

CREATE UNIQUE INDEX IF NOT EXISTS i_names ON Names (name);

CREATE TABLE IF NOT EXISTS "Keys"(
    "ROWID"   INTEGER PRIMARY KEY,
    "pubkey"  BLOB    NOT NULL UNIQUE,
    "id"      INTEGER NOT NULL,
    "use"     INTEGER NOT NULL,
    FOREIGN KEY ("id") REFERENCES "Names"("ROWID")
) STRICT;

CREATE UNIQUE INDEX IF NOT EXISTS i_pubkey ON Keys (pubkey);
CREATE UNIQUE INDEX IF NOT EXISTS i_id ON Keys (use, id);

CREATE TABLE IF NOT EXISTS "Misc"(
    "ROWID"  INTEGER PRIMARY KEY,
    "m" ANY
) STRICT;

CREATE VIEW IF NOT EXISTS UserZookoIDs AS
SELECT
    "Names".name AS name,
    "Keys".pubkey AS pubkey
FROM "Names" INNER JOIN "Keys"
ON "Names"."ROWID"="Keys"."id" AND "Keys"."use"=1
ORDER BY LOWER("name"), "name" COLLATE BINARY;

CREATE TRIGGER IF NOT EXISTS InsertUserZookoID INSTEAD OF INSERT ON UserZookoIDs FOR EACH ROW BEGIN
    INSERT OR ROLLBACK INTO "Names" VALUES(
        NULL,
        NEW."name"
    );
    INSERT OR ROLLBACK INTO "Keys" VALUES(
        NULL,
        NEW."pubkey",
        last_insert_rowid(),
        1
    );
END;

CREATE TRIGGER IF NOT EXISTS DeleteUserZookoID INSTEAD OF DELETE ON UserZookoIDs FOR EACH ROW BEGIN
    DELETE FROM "Keys" WHERE "Keys"."pubkey" = OLD."pubkey";
    DELETE FROM "Names" WHERE "Names"."name" = OLD."name";
END;
COMMIT;
/*
 The "id" column of keys

 Currently broken, ill defined, and incoherent -- part of the planned future petname architecture.   Clean up this comment section when petnames work as intended.

    Meaning: "id" is a foreign key to Names.ROWID, declared with FOREIGN KEY ("id") REFERENCES "Names"("ROWID") in the Keys table (WALLET_SCHEMA_SQL, db_accessors.h) and
    enforced by sqlite itself on every connection. It answers "which nickname row is this public key bound to?" Despite the generic name it is not the key's own
    identity — that is the "ROWID" column; id is really name_id (the planned v0_2 schema in docs/design/data_structure.md renames it exactly that).

 Effects:

 1. Join column of the virtual table. UserZookoIDs is defined as ... ON "Names"."ROWID"="Keys"."id" AND "Keys"."use"=1 (frame.cpp:758). It is the only link between the two
    base tables.
 2. Populated by the trigger via last_insert_rowid(). InsertUserZookoID first does INSERT OR ROLLBACK INTO "Names", then INSERT ... INTO "Keys" VALUES(NULL, NEW."pubkey",
    last_insert_rowid(), 1) (frame.cpp:763-774) — so id receives the ROWID of the Names row created one statement earlier on the same connection. OR ROLLBACK plus the
    enclosing transaction is what keeps a failed name insert from leaving an orphan key.
 3. Uniqueness scoping. CREATE UNIQUE INDEX i_id ON Keys (use, id) — at most one key per (relationship-kind, name) pair. Since pubkey is itself UNIQUE, the result today is
    a strict 1:1 between a nickname and a key.
 4. NOT NULL — in this schema every key row must reference a name; a nameless key is unrepresentable.
 5. Integrity verification path. The unit test reads it back (auto id = read_keys.column<int>(2), unit_test.cpp:1556), fetches SELECT * FROM "Names" WHERE ROWID = ?1
    (sql_read_name, db_accessors.h:249), then checks MasterExtendedSecretKey(name).timesBase() == pubkey.
 6. Real FK enforcement. The Keys table declares FOREIGN KEY ("id") REFERENCES "Names"("ROWID"), and PRAGMA foreign_keys = ON is issued in the ISqlite3Impl constructor
    (ISqlit3Impl.cpp) -- the pragma is per connection and not stored in the file, so it must be set on every open, which the single constructor of the single implementation
    class guarantees. Wallet files created before the constraint existed carry no FOREIGN KEY clause in their stored schema; migrate_wallet_keys_foreign_key
    (db_accessors.cpp) rebuilds their Keys table in place on open, and is called right after check_wallet_file_pragmas everywhere an existing wallet is opened
    (dbconnect, Frame::PromptForWalletPassphraseIfNeeded, unit test). An orphan key row, or the deletion of a Names row a key still references, is now rejected by sqlite
    (SQLITE_CONSTRAINT_FOREIGNKEY) instead of merely being prevented by the triggers' transactional discipline -- the triggers still do the paired insert and delete, but
    they are no longer the only thing standing between the schema and an orphan.

 The "use" column  of keys.

Currently broken, ill defined, and incoherent -- part of the planned future petname architecture.  Clean up this comment section when petnames work as intended.

    Meaning: the enumerator of the relationship between the nickname and the key — specifically, what the wallet knows about the secret behind pubkey and how it got it.
    Exactly one value is ever written: use = 1 = "the secret key is derived from this name using the master extended secret key" (child = master.privatekey × H(chain ‖
    normalizeWhitespace(name)), secrets.h:147-152). The unit test's error string for any other value, sz_unknown_secret_key_algorithm, shows the column was conceived from the
    start as tagging a derivation method, not a boolean. The planned value use = 2 ("we know only the public key; the name is under someone else's control") exists only in
    docs/design/data_structure.md.

 Effects:

 1. Hardcoded on write. The insert trigger writes literal 1 (frame.cpp:771) — nothing in the code can produce any other value.
 2. Filters the view. "Keys"."use"=1 in UserZookoIDs makes the view mean "identities I own"; a future use=2 population would be invisible to it, which is why the planned
    schema adds the symmetric UserPetZookoIDs view filtered on use=2.
 3. Scopes uniqueness. As part of i_id (use, id), it makes "one name, one key" hold per relationship kind — structurally permitting the same Names row to later be bound to
    keys under different use values without an index change.
 4. Gates validation. unit_test.cpp:1557-1558 throws on use != 1, and display_wallet::refresh_from_database's re-derivation check
    (MasterExtendedSecretKey(name).timesBase() != pubkey → throw) is only semantically valid for use=1 rows — externally controlled keys have no secret to re-derive from,
    so when use=2 arrives, both checks must become use-aware.

2 planned implementation

Bump WALLET_FILE_SCHEMA_VERSION

Don’t worry about upgrade previous databases, because currently the only existing databases are merely for testing purposes.

2.1 Names

Keys.use==2 will signify a public key where we cannot generate the corresponding secret key from the name and the master secret, normally, someone else’s key, but a key that has a nickname that we are locally using to identify it. Keys.id will reference that nickname.

Keys.use==3 will signify a public key where we cannot generate the corresponding secret key from the name and the master secret, normally, someone else’s key, but a key that has a petname that we are locally using to identify it. Keys.id will reference that petname. Its nickname, if known, can be found in the HiddenNicks table, which is a junction table of Keys and Names.

Keyed on the ROWID of Keys, and containing a ROWID of Names.

 CREATE TABLE HiddenNicks(
   ROWID INTEGER PRIMARY KEY REFERENCES keys(ROWID ),
   nick  INTEGER NOT NULL REFERENCES Names(ROWID),
  );

Should we encounter a Key which claims to have a nickname that collides with an existing name, we must generate a petname, which the user can edit arbitrarily.

We will create a new view that maps between all known public keys and all names, regardless of whether we know the secret keys for those names.

CREATE VIEW ZookoIDs AS
SELECT
    "Names".name AS name,
    "Keys".pubkey AS pubkey,
    "Keys.use AS use
FROM "Names" INNER JOIN "Keys"
ON "Names"."ROWID"="Keys"."id"
ORDER BY LOWER("name"), "name" COLLATE BINARY;
COMMIT;

We are not at first going to handle the problem of name collisions, so we are not at first going to generate petnames. First we need code that can send and receive messages, and if it receives a message that results in a name collision, it will just throw. But we want the schema infrastructure in place so that once it is working well enough that we can successfully communicate with non colliding names, and correctly throw on colliding names, then we will get around to handling collisions enerating petnames – but first it will have to pass testing in successful communication in the easy path of no collisions.

We shall create triggers to allow insertion into ZookoIDs and deletion from ZookIDs, analogous to the triggers for UserZookoIDs, but these will initially generate an error if we try to insert or delete a row where the use value is not equal to 1 or 2. We will address the other cases when we need them once we have more things working.

The initial implementation will be intentionally incomplete. When it passes testing on incomplete functionality, then time enough to add more functionality.

2.2 Messages.

What do we want to do with names? We want to see validly signed messages, and send validly signed messages.

Later we are going to implement blacklists, greylists, and white lists on the signer, and we are going to encrypt messages so that they can only be read by those who know the secret keys for certain names, but we are going to address the easy case first, public messages visible to all participants.

2.2.1 Messages on the libp2p network

Eventually, to make posting vast amounts of spam expensive, we are going to require signing with a randomx hash, and an integer such that the randomx hash has a special value. But we are not going to implement that yet. A clear text message passed around in libp2p between peers will consist of a libp2p message that is a concatenation of: 1. the byte 33 to represent indicate that cleartext public message follows (later we will address other cases, starting with private messages) 1. the title header, a valid null terminated utf8 string containing at least one character, and not more than eighty character, 2. the time as an unsigned VLQ integer in seconds past the epoch divided by three hundred (as too exact a time would leak metadata), 4. the body, also a utf8 null terminated string that must contain at least one character and must be less than 2^{16} bytes, 5. the ristretto25519 point that is the public key of the signer as thirty two binary bytes, 7. a human readable null terminated utf8 string that is expected and assumed to be the nickname of the signing key, and may not be longer than sixty characters, 8. the pow, an arbitrary unsigned VLQ integer less than 2^{64} (which will be eventually used in Bitmessage style pow, but we will not implement that yet), 9. and the two ristretto25519 scalars that constitute the Schnorr signature of the hash of all the above values concatinated together, as binary. Eventually this is going to be the randomx hash, but for the moment, will be hash<256>. The signature is computed in accordance with Single Schnorr Signature

The MessageId that libp2p and gossipsub will use for this object is the hash, that will also be used for the signature.

2.2.2 Messages in the database.

CREATE TABLE "Titles"(
    "ROWID"     INTEGER PRIMARY KEY,
    "title"     BLOB    NOT NULL UNIQUE,
) STRICT;

CREATE UNIQUE INDEX i_titles ON Titles (title);

CREATE TABLE "Messages"(
    "ROWID"     INTEGER PRIMARY KEY,
    "title"     INTEGER NOT NULL REFERENCES Titles(ROWID),
    "time"      INTEGER,
    "body"      BLOB    NOT NULL,
    "signer"    INTEGER NOT NULL REFERENCES Keys(ROWID),
    "pow"       INTEGER,
) STRICT;

CREATE TABLE "Ids"(
    "ROWID"     INTEGER PRIMARY KEY,
    "id"  BLOB  NOT NULL UNIQUE,
    "Preimage"  INTEGER NOT NULL,
    "use"       INTEGER NOT NULL
) STRICT;

Creative Commons License reaction.la gpg key 154588427F2709CD9D7146B01C99BB982002C39F
This work is licensed under the Creative Commons Attribution 4.0 International License.