can you find all references to OP_RETURN data and outline how it's is handled. When is it serialized/deserialized and in what context? output into a markdown document
⏺ I'll search for all references to OP_RETURN in the codebase and document how it's handled, including serialization/deserialization contexts.
⏺ Search(pattern: "OP_RETURN") ⎿ Found 78 files (ctrl+o to expand)
⏺ Search(pattern: "OP_RETURN", output_mode: "content") ⎿ Found 1355 lines (ctrl+o to expand)
⏺ Read(src/script/script.h) ⎿ Read 631 lines (ctrl+o to expand)
⏺ Read(src/script/solver.cpp) ⎿ Read 239 lines (ctrl+o to expand)
⏺ Read(src/script/interpreter.cpp) ⎿ Error: File content (27421 tokens) exceeds maximum allowed tokens (25000). Please use offset and limit parameters to read specific portions of the file, or use the GrepTool to search for specific content.
⏺ Read(src/policy/policy.cpp) ⎿ Read 492 lines (ctrl+o to expand)
⏺ Read(src/compressor.h) ⎿ Read 119 lines (ctrl+o to expand)
⏺ Read(src/core_write.cpp) ⎿ Read 277 lines (ctrl+o to expand)
⏺ Read(src/blockfilter.cpp) ⎿ Read 284 lines (ctrl+o to expand)
⏺ Read(src/script/script.cpp) ⎿ Read 440 lines (ctrl+o to expand)
⏺ Search(pattern: "IsUnspendable", output_mode: "content") ⎿ Found 122 lines (ctrl+o to expand)
⏺ Read(src/script/interpreter.cpp) ⎿ Read 200 lines (ctrl+o to expand)
⏺ Write(OP_RETURN_ANALYSIS.md) ⎿ Wrote 549 lines to OP_RETURN_ANALYSIS.md # OP_RETURN Data Handling Analysis ## Overview This document outlines how OP_RETURN data is handled throughout the Bitcoin Knots codebase, including where it's serialized/deserialized and the contexts in which it operates. ## What is OP_RETURN? OP_RETURN is a Bitcoin script opcode (0x6a) that marks transaction outputs as provably unspendable. It's commonly used for embedding arbitrary data in the blockchain. … +539 lines (ctrl+o to expand)
⏺ Created comprehensive documentation in OP_RETURN_ANALYSIS.md covering:
Serialization/Deserialization Contexts:
- Standard Script Serialization - Binary format in transactions/blocks via SERIALIZE_METHODS (src/script/script.h:438)
- Script Compression (UTXO DB) - Uses ScriptCompression wrapper, but OP_RETURN outputs are excluded from UTXO set entirely (src/compressor.h:54-96)
- Witness Commitments - SegWit uses OP_RETURN for coinbase commitments (src/validation.cpp:4324-4330)
- Block Filters - OP_RETURN explicitly excluded from BIP 157/158 filters (src/blockfilter.cpp:197)
- RPC/JSON - Decoded via ScriptToAsmStr() and displayed as "nulldata" type (src/core_write.cpp)
Key Handling Points:
- Detection: IsUnspendable() checks if first byte == OP_RETURN (src/script/script.h:560)
- UTXO: Never added to UTXO set via check in AddCoin() (src/coins.cpp:72)
- Policy: Max 42 bytes default, one per transaction (src/policy/policy.cpp)
- Execution: Script fails immediately at interpreter level
- Type: Classified as TxoutType::NULL_DATA (src/script/solver.cpp:185)
The document includes code references, data flow diagrams, configuration examples, and security considerations.
OP_RETURN Data Handling Analysis
Overview#
This document outlines how OP_RETURN data is handled throughout the Bitcoin Knots codebase, including where it's serialized/deserialized and the contexts in which it operates.
What is OP_RETURN?#
OP_RETURN is a Bitcoin script opcode (0x6a) that marks transaction outputs as provably unspendable. It's commonly used for embedding arbitrary data in the blockchain.
Opcode Definition: src/script/script.h:110
OP_RETURN = 0x6a,
Key Constants#
Maximum Size Limits#
- MAX_OP_RETURN_RELAY: 42 bytes (defined in
src/policy/policy.h:98)- 40 bytes of data + 1 for OP_RETURN + 1 for pushdata opcode
- This is the default for
-datacarriersizepolicy setting
- MINIMUM_WITNESS_COMMITMENT: Used for SegWit commitment in coinbase (
src/validation.cpp:4327)
Core Script Operations#
1. Script Definition and Opcodes#
Location: src/script/script.h and src/script/script.cpp
The OP_RETURN opcode is defined as part of the script language:
OP_RETURN = 0x6a, // Control opcode
GetOpName() (src/script/script.cpp:56):
Converts the opcode to string representation "OP_RETURN" for debugging/display.
2. Unspendability Detection#
Location: src/script/script.h:560-563
bool IsUnspendable() const
{
return (size() > 0 && *begin() == OP_RETURN) || (size() > MAX_SCRIPT_SIZE);
}
This is the primary method to detect OP_RETURN outputs. Scripts starting with OP_RETURN are immediately flagged as unspendable.
Used By:
src/coins.cpp:72- Prevents OP_RETURN outputs from being added to UTXO setsrc/validation.cpp:2477- Skips OP_RETURN outputs when validating transactionssrc/policy/policy.cpp:47- Dust threshold calculation (returns 0 for unspendable)src/index/coinstatsindex.cpp:165,449- UTXO statistics trackingsrc/core_write.cpp:123- Script ASM string generation- Block filters (
src/blockfilter.cpp:197) - Excludes OP_RETURN from filters - Wallet operations (
src/wallet/receive.cpp:232)
3. Script Type Identification#
Location: src/script/solver.cpp:185-187
// Provably prunable, data-carrying output
if (scriptPubKey.size() >= 1 &&
scriptPubKey[0] == OP_RETURN &&
scriptPubKey.IsPushOnly(scriptPubKey.begin()+1)) {
return TxoutType::NULL_DATA;
}
Identifies OP_RETURN outputs as TxoutType::NULL_DATA. The script must:
- Start with OP_RETURN
- Have only push operations after OP_RETURN (data only, no executable code)
Type Name: Returns "nulldata" via GetTxnOutputType() (src/script/solver.cpp:26)
4. Data Carrier Byte Counting#
Location: src/script/script.cpp:306-354
The DatacarrierBytes() method analyzes scripts to count data bytes:
std::pair<size_t, size_t> CScript::DatacarrierBytes() const
Returns: {unconditional_bytes, conditional_bytes}
For OP_RETURN (src/script/script.cpp:325-328):
if (opcode == OP_RETURN && !inside_conditional) {
// unconditional OP_RETURN is unspendable
return {size(), 0}; // All bytes counted as unconditional data
}
When OP_RETURN is encountered outside conditionals, the entire script is considered data.
Also Tracks:
OP_FALSE OP_IF ... OP_ENDIFblocks (conditional data)<data> OP_DROPpatterns
Serialization/Deserialization#
1. Standard Script Serialization#
Location: src/script/script.h:438
SERIALIZE_METHODS(CScript, obj) {
READWRITE(AsBase<CScriptBase>(obj));
}
OP_RETURN scripts are serialized like any other script:
- Length-prefixed byte vector
- Binary representation of opcodes and data
Contexts:
- Transaction serialization (inputs/outputs)
- Block serialization
- Network message transmission
- Database storage (leveldb for blockchain/UTXO)
2. Script Compression (UTXO Database)#
Location: src/compressor.h:54-96
struct ScriptCompression
{
void Ser(Stream &s, const CScript& script);
void Unser(Stream &s, CScript& script);
}
Special Handling (src/compressor.h:87-90):
if (nSize > MAX_SCRIPT_SIZE) {
// Overly long script, replace with a short invalid one
script << OP_RETURN;
s.ignore(nSize);
}
When deserializing invalid/oversized scripts, they're replaced with a single OP_RETURN byte to maintain unspendability.
Compression Scheme:
- Pay-to-pubkey-hash: 21 bytes
- Pay-to-script-hash: 21 bytes
- Pay-to-pubkey: 33 bytes
- Others: 1-2 bytes + script length
- OP_RETURN outputs are NOT typically stored in UTXO set (excluded by
IsUnspendable()check)
3. Witness Commitment (SegWit)#
Location: src/validation.cpp:4324-4330
CTxOut out;
out.nValue = 0;
out.scriptPubKey.resize(MINIMUM_WITNESS_COMMITMENT);
out.scriptPubKey[0] = OP_RETURN;
out.scriptPubKey[1] = 0x24;
out.scriptPubKey[2] = 0xaa;
out.scriptPubKey[3] = 0x21;
// ... witness commitment hash follows
SegWit blocks use OP_RETURN in the coinbase transaction to commit to the witness merkle tree. Format:
OP_RETURN <0x24> <0xaa21a9ed> <32-byte commitment>
4. Block Filters (BIP 157/158)#
Location: src/blockfilter.cpp:187-206
for (const CTxOut& txout : tx->vout) {
const CScript& script = txout.scriptPubKey;
if (script.empty() || script[0] == OP_RETURN) continue;
// ... add to filter
}
OP_RETURN outputs are explicitly excluded from compact block filters. This:
- Reduces filter size
- Reflects that OP_RETURN outputs cannot be spent
- No wallet needs to scan for them
Transaction Policy & Validation#
1. Standard Transaction Rules#
Location: src/policy/policy.cpp:79-101
bool IsStandard(const CScript& scriptPubKey,
const std::optional<unsigned>& max_datacarrier_bytes,
TxoutType& whichType)
{
// ...
if (whichType == TxoutType::NULL_DATA) {
if (!max_datacarrier_bytes || scriptPubKey.size() > *max_datacarrier_bytes) {
return false;
}
}
return true;
}
Standardness Checks:
- Must not exceed
max_datacarrier_bytes(default 42 bytes) - Controlled by
-datacarrierand-datacarriersizeconfiguration options - Can be disabled entirely with
-datacarrier=0
2. Multiple OP_RETURN Policy#
Location: src/policy/policy.cpp:190-193
// only one OP_RETURN txout is permitted
if (nDataOut > 1) {
MaybeReject("multi-op-return");
}
Standard transactions allow only ONE OP_RETURN output per transaction. This prevents spam and maintains relay policy.
3. Token/Runes Filtering#
Location: src/policy/policy.cpp:173-176
if (whichType == TxoutType::NULL_DATA) {
if (txout.scriptPubKey.size() > 2 &&
txout.scriptPubKey[1] == OP_13 &&
opts.reject_tokens) {
MaybeReject("tokens-runes");
}
}
Bitcoin Knots includes additional filtering for Runes protocol (identified by OP_RETURN OP_13 ...).
4. Configuration Options#
Location: src/node/mempool_args.cpp and src/kernel/mempool_options.h:72
std::optional<unsigned> max_datacarrier_bytes{
DEFAULT_ACCEPT_DATACARRIER ?
std::optional{MAX_OP_RETURN_RELAY} :
std::nullopt
};
Command-line Options:
-datacarrier(default: true) - Accept/relay OP_RETURN transactions-datacarriersize(default: 42) - Maximum OP_RETURN size in bytes-datacarrierfullcount(default: true) - Count all bytes in size calculation-acceptnonstddatacarrier- Accept non-OP_RETURN data carrier injection-rejecttokens- Reject Runes token protocol (OP_RETURN OP_13)
Consensus vs Policy#
Consensus Rules#
- OP_RETURN outputs are always valid at the consensus level
- No size limits enforced by consensus
- Cannot be spent (script execution fails immediately)
- Any size OP_RETURN is acceptable in a block
Policy Rules (Relay/Mempool)#
- Size limited (default 42 bytes)
- Only one per transaction
- Can be disabled entirely
- Affects transaction relay, not block validity
UTXO Set Handling#
Location: src/coins.cpp:70-74
void CCoinsViewCache::AddCoin(const COutPoint &outpoint, Coin&& coin,
bool possible_overwrite) {
assert(!coin.IsSpent());
if (coin.out.scriptPubKey.IsUnspendable()) return; // OP_RETURN exits here
// ... add to UTXO set
}
Critical Optimization: OP_RETURN outputs are NEVER added to the UTXO set. This:
- Saves database space
- Improves validation performance
- Reduces memory usage
- Reflects their unspendable nature
Script Execution#
Interpreter Behavior#
Location: src/script/interpreter.cpp
When script execution encounters OP_RETURN:
- Immediately fails - Returns false
- Does not process further opcodes - Script terminates
- Error:
SCRIPT_ERR_OP_RETURN
Defined In: src/script/script_error.h and src/script/script_error.cpp
This makes OP_RETURN outputs provably unspendable - there's no input script that can successfully spend them.
RPC and User Interface#
1. Script Decoding#
Location: src/core_write.cpp:104-147
std::string ScriptToAsmStr(const CScript& script,
const bool fAttemptSighashDecode)
{
// ...
// the IsUnspendable check makes sure not to try to decode
// OP_RETURN data that may match the format of a signature
if (fAttemptSighashDecode && !script.IsUnspendable()) {
// ... signature decoding
}
// ...
}
When displaying scripts, OP_RETURN data is shown as hex without attempting to decode as signatures.
Example Output:
OP_RETURN 48656c6c6f20576f726c64 // "Hello World"
2. Transaction JSON Representation#
Location: src/core_write.cpp:156-175
void ScriptToUniv(const CScript& script, UniValue& out,
bool include_hex, bool include_address,
const SigningProvider* provider)
{
out.pushKV("asm", ScriptToAsmStr(script));
// ...
out.pushKV("type", GetTxnOutputType(type)); // Returns "nulldata"
}
OP_RETURN outputs appear in JSON with:
type: "nulldata"asm: Human-readable script representationhex: Raw hex bytes- No
addressfield (unspendable has no address)
3. Test Data#
Location: test/functional/data/rpc_decodescript.json:39-45
[
"6a00",
{
"asm": "OP_RETURN 0",
"desc": "raw(6a00)#ncfmkl43",
"type": "nulldata"
}
]
Testing#
Functional Tests#
Key Test Files:
test/functional/mempool_datacarrier.py- Tests datacarrier policytest/functional/feature_block.py:1217-1263- OP_RETURN reorg handlingtest/functional/wallet_send.py- Wallet interaction with OP_RETURNtest/functional/rpc_rawtransaction.py- RPC handling
Test Coverage Includes:
- Size limits
- Multiple OP_RETURN rejection
- Configuration options
- Reorg handling
- UTXO exclusion
Unit Tests#
Key Files:
src/test/script_tests.cpp- Script parsing and executionsrc/test/script_standard_tests.cpp- Standard script validationsrc/test/transaction_tests.cpp- Transaction validationsrc/test/coins_tests.cpp:187- UTXO set handlingsrc/test/fuzz/script.cpp:66-70- Fuzzing unspendability
Data Flow Summary#
Creating a Transaction with OP_RETURN#
Construction:
CScript script; script << OP_RETURN << data_vector;Standardness Check:
IsStandard()validates size and formatPolicy Check:
IsStandardTx()ensures only one OP_RETURN outputSerialization: Standard transaction serialization (binary)
Relay: Propagated if passes policy checks
Block Inclusion: Miner includes in block (consensus-valid)
Processing a Block with OP_RETURN#
Deserialization: Transaction decoded from block data
Consensus Validation: Output is valid (any size)
UTXO Update:
AddCoin()skips OP_RETURN outputs viaIsUnspendable()checkCoin Stats: Tracked as
total_unspendables_scriptsBlock Filter: Excluded from compact block filters
Querying OP_RETURN Data#
Block Explorer: Reads from block data (not UTXO set)
RPC:
getrawtransaction,decoderawtransactiondecode and displayFormat: Displayed as
type: "nulldata"with hex-encoded data
Special Cases and Edge Cases#
1. Witness Commitments#
- OP_RETURN used for SegWit commitments in coinbase
- Format:
OP_RETURN <0x24> <0xaa21a9ed> <commitment> - Required for SegWit blocks
- Location: Coinbase transaction output
2. Invalid/Oversized Scripts#
- Scripts > 10,000 bytes replaced with single OP_RETURN during decompression
- Maintains unspendability property
- Prevents memory exhaustion
3. Reorg Handling#
- OP_RETURN outputs can appear/disappear during reorgs
- No UTXO impact (never in set)
- Test:
feature_block.py:1217- Reorg blocks with OP_RETURN
4. Conditional OP_RETURN#
OP_FALSE OP_IF OP_RETURN OP_ENDIF
- Not executed (inside false branch)
- Output is spendable
- Tracked differently by
DatacarrierBytes()
Security Considerations#
1. Spam Prevention#
- Size limits prevent blockchain bloat
- One per transaction limits abuse
- Cost: Transaction fees still required
2. Data Injection Attacks#
- Knots includes
-acceptnonstddatacarrierto reject non-OP_RETURN data hiding - Detects data in witness, push-drop patterns, etc.
3. Token Protocol Filtering#
-rejecttokensflag rejects Runes (OP_RETURN OP_13)- Prevents token spam if desired
4. UTXO Bloat#
- OP_RETURN never enters UTXO set
- No long-term storage cost (only blockchain)
- Optimization: Checked at coin addition time
Performance Implications#
Benefits of OP_RETURN#
- Fast Validation:
IsUnspendable()is single byte check - No UTXO Growth: Excluded from UTXO set
- Provable Pruning: Can be safely deleted after validation
- Simple Detection: First byte comparison
Costs#
- Blockchain Size: Data stored forever in blocks
- Bandwidth: Relayed to all nodes
- Validation Time: Must process (though fast)
Configuration Examples#
Allow OP_RETURN (Default)#
bitcoind -datacarrier=1 -datacarriersize=42
Disable OP_RETURN#
bitcoind -datacarrier=0
Custom Size Limit#
bitcoind -datacarrier=1 -datacarriersize=80
Reject Runes Tokens#
bitcoind -rejecttokens=1
Summary#
OP_RETURN is handled throughout Bitcoin Knots as follows:
Serialization: Standard binary script serialization in transactions/blocks
Deserialization: Parsed as opcode 0x6a followed by optional data
Detection: IsUnspendable() checks first byte == OP_RETURN
UTXO: Never stored in UTXO set - excluded at coin addition
Policy: Size-limited (42 bytes default), one per transaction, configurable
Consensus: Always valid, any size, provably unspendable
Execution: Script immediately fails with SCRIPT_ERR_OP_RETURN
Block Filters: Explicitly excluded from BIP 157/158 filters
Use Cases: Data anchoring, commitments, timestamping, SegWit commitments
The key insight is that OP_RETURN provides a standardized, efficient, and provably unspendable way to embed data in Bitcoin transactions while minimizing impact on the UTXO set and validation performance.

Comments
No comments yet. Start the conversation.