Encryption

Encryption is applied after Page compression and reversed before decompression. Headers, Statistics, schemas, and indexes remain visible. TsFile v4 supports two encryption mechanisms, both under version byte 0x04:

Mode	How the reader recognizes it	Chunk and Page layout
Plaintext	No file encryption header; legacy level `0` or absent	Base Chunk Header; unencrypted Page body
Legacy encryption	No file encryption header; legacy level `1` or `2`	Base Chunk Header; algorithm-specific encrypted body
Page AEAD	`0x06` file encryption header at offset `7`	Extended Chunk Header; authenticated Page body defined by a profile

Check for the file encryption header before parsing Chunks. In an AEAD file, the legacy footer properties still say unencrypted; the header takes precedence.

The write and read paths are:

encoded page payload -> compression -> encryption -> stored_data
stored_data -> authenticated decryption when applicable -> decompression -> value decoding

File Encryption Header

An AEAD file puts one FileEncryptionHeader immediately after the seven-byte magic/version prefix, before any Chunk Group or operation-index record:

FileEncryptionHeader :=
    byte     marker = 0x06
    UVarInt  payload_size
    byte     header_version = 0x01
    VarString provider_id
    VarString profile_id
    VarString key_id
    VarString key_version
    UVarInt  file_crypto_id_size = 16
    byte     file_crypto_id[16]
    UVarInt  wrapped_data_key_size
    byte     wrapped_data_key[wrapped_data_key_size]

payload_size counts from header_version through the wrapped key, excluding the marker and the length field itself. Its allowed range is 1..1048576, and parsing must end exactly at that boundary. The four string identifiers must be non-null and non-blank. The file crypto ID is exactly 16 bytes, and the wrapped data key is non-empty. These binary lengths use UVarInt, not the nullable VarBinary used for footer properties.

The provider identifies the encryption implementation or service. Its profile names a fixed byte-format contract: Page framing, nonce and tag representation, size mapping, and Chunk proof size. A provider can offer several profiles. The file stores these public identifiers and the wrapped data key, rather than an implementation class name or the plaintext key. The reader needs the matching provider/profile and access to the external key material.

Header version 0x01 is the supported layout. Unknown versions, truncated fields, trailing bytes, a second encryption header, or a header anywhere except offset 7 are errors.

AEAD Chunk Header

The file encryption header also changes how every Chunk Header is parsed. The Chunk marker stays the same, but three fields follow encoding_type:

    int64  chunk_ordinal
    int32  chunk_authentication_size
    byte   chunk_authentication[chunk_authentication_size]

chunk_ordinal is a non-negative identifier allocated uniquely across the file, including Time and Value Chunks. The proof length must match the profile and lie in 1..1024. These fields are part of the Header, so none of them count towards data_size.

The proof binds the ordinal to the Chunk's absolute position. Verify it before using that identity. Moving a Chunk changes the authenticated position and therefore requires a new proof. Its associated data is the following big-endian sequence:

int32 domain_size = 20
byte  domain[20] = UTF8("TsFile/ChunkPosition")
int32 associated_data_version = 1
byte  file_crypto_id[16]
int64 absolute_chunk_marker_offset
int64 chunk_ordinal

AEAD Page Body and Context

For a non-empty Page, compressed_size records the entire encrypted body, including framing, nonces, tags, and padding. It is larger than the compressed plaintext length, even with UNCOMPRESSED. A profile may use fixed overhead or a size-dependent layout, so use its size mapping to recover the candidate plaintext length.

Page authentication also binds the body to its file, Chunk, and Page position. The associated data is the exact big-endian sequence below. SizedBytes means an int32 byte length followed by those bytes; text is UTF-8.

int32      associated_data_version = 1
SizedBytes provider_id
SizedBytes profile_id
SizedBytes key_id
SizedBytes key_version
SizedBytes file_crypto_id
int64      chunk_ordinal
int32      page_index
int32      uncompressed_size
int32      compressed_plaintext_size

page_index starts at zero for each physical Chunk and counts empty aligned Value Pages as well. An all-null Value Page is still the single UVarInt(0) described in Nulls; it has no ciphertext or Page authentication tag.

The candidate plaintext length becomes trusted only after authentication succeeds. Decryption must return that length, and decompression must then return uncompressed_size bytes. An invalid tag, wrong key, unsupported profile, or length mismatch is an error; do not retry as plaintext.

Legacy File Properties

When there is no file encryption header, read the following TsFileMetadata properties:

Property	Meaning
`encryptLevel`	`0` for no encryption; `1` for a directly stored data key; `2` for a data key encrypted by an external master key
`encryptType`	Legacy encryption/decryption algorithm identifier
`encryptKey`	Text representation of the data key or key-wrapping result

These property values contain text. At level 1, the data key is stored directly. At level 2, an external master-key decryptor unwraps it before Page decryption. Missing encryptLevel means 0; levels 1 and 2 require encryptType and a non-empty encryptKey. Unknown levels and unavailable algorithms are errors.

Legacy encryption uses the algorithm's encrypt/decrypt operations without the AEAD Chunk suffix or Page context. compressed_size is still the actual stored output length; it equals compressed plaintext length only for a length-preserving algorithm.

AEAD writers set these legacy properties to level 0, type org.apache.tsfile.encrypt.UNENCRYPTED, and an empty key. The AEAD header supplies the real mode and wrapped key, so those values must not be interpreted as a plaintext fallback.

Security Scope

Page AEAD authenticates non-empty Page bodies and their context, while Chunk proofs bind ordinals to file positions. It does not authenticate every visible field or provide a whole-file digest. Legacy encryption does not necessarily provide authentication. See Checksumming for whole-file integrity and Error Recovery for retaining encryption context when rebuilding a footer.