abstract: consensus is a hard problem, and considerably harder when you have shards Nakomoto Satoshi’s implementation of Nakomoto consensus, aptly described as wading through molasses, failed to scale
Merkle DAG (Merkle Directed Acyclic Graph) is the core data structure that powers content addressing, verifiability, and efficient distribution in IPFS (and IPLD-based systems).
A Merkle DAG is a generalization of a Merkle Tree * Directed Acyclic Graph (DAG): Nodes can have multiple parents and multiple children. No cycles (loops) are allowed. * Merkle: Every node is content-addressed by its CID (Content Identifier), which is a cryptographic hash of 1. The node’s own data (payload). 2. The CIDs (hashes) of its children/links.
This makes the entire structure self-verifying and immutable.Feature
When you add a file to IPFS (e.g., via ipfs add): 1. The file is split into blocks (default ~256 KiB chunks). 2. Each block becomes a leaf node with a CID. 3. Intermediate nodes (UnixFS directory nodes, etc.) link to these blocks. 4. A single root CID represents the entire file or directory.
Blockchains use the same mechanism, except that the current state of the blockchain and mempool is ascertained from gossipsub events.
IPFS file structure is fundamentally the same as Git file structure, except that public keys perform the role of named branches. (No Zooko mechanism, and you do not have all the handy dandy Git mechanisms for interrogating history, versions, and merging versions.)
is the block exchange protocol used in IPFS (and libp2p-based systems) for requesting, discovering, and transferring content-addressed blocks of data between peers.It is the main mechanism that actually moves the data after you know a CID (Content Identifier) — the hash of the data.
Bitswap is optimized for efficiently fetching the actual large blocks of Content-addressed data.:
Peers exchange messages containing: * Want-lists (what blocks/CIDs I need). * Blocks (the actual data). * HAVE / DONT_HAVE responses.
Two main jobs * Acquire blocks that your local application has requested. * Serve blocks you have to other peers who want them.
Key Mechanisms * Want-list: When you want a CID, Bitswap adds it to your want-list and sends it to connected peers. * Proactive sending: If a peer has the block you want, it can send it immediately (especially useful with the graph structure). * Session-based: Bitswap maintains sessions with peers to track ongoing transfers efficiently. * Graph-aware: Because IPFS data is usually a Merkle DAG (blocks link to other blocks), Bitswap can fetch many parts in parallel from different peers and verify integrity automatically via hashes. * Discovery bonus: Since v1.2, Bitswap also acts as lightweight content routing — it learns which connected peers have which blocks from the want/block exchanges.
is for broadcasting notifications or small messages to a named topic (e.g., “here is a new CID”), Bitswap is optimized for efficiently fetching the actual large blocks of data.
Provides something somewhat like the Bitcoin mempool. If you subscribe, you probably see most, perhaps all, of what was announced after you are subscribed and active.
It is like a newsfeed with headlines you could click on, once they have very briefly appeared, before they swiftly scroll past into the bit bucket. If message id reference messages containing CIDs, and CIDs reference content containing CIDs Gossipsub tells the application layer to drive Bitswap
publish a CID (or a small “head” pointer) via Gossipsub. Peers receive the notification → then use Bitswap to actually pull the full block(s) and recursively pull any linked blocks in the DAG.
Once you are synchronised, you get most of your new data through gossipsub, with a relatively small number of misses being filled in through Bitswap. When catching up, you get the root from Gossipsub, and then filling in a whole lot of stuff through Bitswap.
Mining pools, asics
RandomX prevented asics, because RandomX mining is best done on a general purpose CPU, but as Monero got bigger, it came to pass that payouts got bigger. And because more and more people were trying to mine a block, they got rarer and rarer
So miners joined in mining pools, so as to get smaller, but more regular and predicable payouts more often. Which destroys the decentralization objective of mining, giving the central authority running the mining pool dangerously great power over the blockchain.
Monero’s workaround for this is P2Pool, a mining pool without centralization. But not everyone wants to use P2Pool.
Monero has a big problem with people preferring to mine in big pools, because of the convenience provided by the dangerously powerful central authority.
It easier for the individual miner to let the center make all the decisions that matter, but many of these decisions matter to other people, and the center could make decisions that are contrary to everyone else’s interests. Even if the individual miner is better off than mining solo, this could well make everyone including the individual miner worse off, because he and everyone may be adversely affected by other people’s decision to pool mine.
why do you hate POW? Because of resource waste? I thought it was the reason for success of BTC. With proof of share, or proof of stake, there will be always discussions of pre-mine, centralization etc. Would it be the case?
Hate the resource waste. It offends me. Plus a system that does not directly handle money, that is a a messaging system between systems that do handle money, cannot incentivise the necessary resource waste.
It could bill people for messaging, and the payments could go to the block winner, but then it would be a dao or yet another crypto currency, and not a neutral platform that other daos and crypto currencies could use.
Suppose we have a filecoin style proof of spacetime . Which also wastes resources, but identifies those peers that are contributing to the network by storing information and are capable of passing it around, and have lots of connections to other peers. All peers that pass the proof of space test become authorized consensus makers for a certain number of blocks, say 8192 blocks. The test is not too hard. Most peers are authorized.
We harvest randomness, possibly from the fact that parties do not know each other’s secret keys, possibly from the proof of space time test, so that each round, or each group of rounds, a peer gets a random weight, such that the inverse of the weight is uniformly distributed between one and two to the fifty sixth
Which means the weight is non uniformly distributed, with a very few peers having most of the weight.
Each peer goes with the consensus block that has the highest chain of weights that it knows of.
Thus the most well known chain is propagated, becoming more well known. The more peers that know of a block, the greater the weight of the block.
It is a better algorithm, but a whole lot more work to implement than RandomX POW.
reaction.la gpg key 154588427F2709CD9D7146B01C99BB982002C39F
This work is licensed under the Creative Commons Attribution 4.0 International License.