Share Leases

A lease is a marker attached to a share indicating that some client has asked for that share to be retained for some amount of time. The intent is to allow clients and servers to collaborate to determine which data should still be retained and which can be discarded to reclaim storage space. Zero or more leases may be attached to any particular share.

Renewal Secrets

Each lease is uniquely identified by its renewal secret. This is a 32 byte string which can be used to extend the validity period of that lease.

To a storage server a renewal secret is an opaque value which is only ever compared to other renewal secrets to determine equality.

Storage clients will typically want to follow a scheme to deterministically derive the renewal secret for a particular share from information the client already holds about that share. This allows a client to maintain and renew single long-lived lease without maintaining additional local state.

The scheme in use in Tahoe-LAFS as of 1.16.0 is as follows.

  • The netstring encoding of a byte string is the concatenation of:
    • the ascii encoding of the base 10 representation of the length of the string
    • ":"
    • the string itself
    • ","
  • The sha256d digest is the sha256 digest of the sha256 digest of a string.
  • The sha256d tagged digest is the sha256d digest of the concatenation of the netstring encoding of one string with one other unmodified string.
  • The sha256d tagged pair digest the sha256d digest of the concatenation of the netstring encodings of each of three strings.
  • The bucket renewal tag is "allmydata_bucket_renewal_secret_v1".
  • The file renewal tag is "allmydata_file_renewal_secret_v1".
  • The client renewal tag is "allmydata_client_renewal_secret_v1".
  • The lease secret is a 32 byte string, typically randomly generated once and then persisted for all future uses.
  • The client renewal secret is the sha256d tagged digest of (lease secret, client renewal tag).
  • The storage index is constructed using a capability-type-specific scheme. See storage_index_hash and ssk_storage_index_hash calls in src/allmydata/uri.py.
  • The file renewal secret is the sha256d tagged pair digest of (file renewal tag, client renewal secret, storage index).
  • The base32 encoding is base64.b32encode lowercased and with trailing = stripped.
  • The peer id is the base32 encoding of the SHA1 digest of the server’s x509 certificate.
  • The renewal secret is the sha256d tagged pair digest of (bucket renewal tag, file renewal secret, peer id).

A reference implementation is available.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87

"""
This is a reference implementation of the lease renewal secret derivation
protocol in use by Tahoe-LAFS clients as of 1.16.0.
"""

from allmydata.util.base32 import (
    a2b as b32decode,
    b2a as b32encode,
)
from allmydata.util.hashutil import (
    tagged_hash,
    tagged_pair_hash,
)


def derive_renewal_secret(lease_secret: bytes, storage_index: bytes, tubid: bytes) -> bytes:
    assert len(lease_secret) == 32
    assert len(storage_index) == 16
    assert len(tubid) == 20

    bucket_renewal_tag = b"allmydata_bucket_renewal_secret_v1"
    file_renewal_tag = b"allmydata_file_renewal_secret_v1"
    client_renewal_tag = b"allmydata_client_renewal_secret_v1"

    client_renewal_secret = tagged_hash(lease_secret, client_renewal_tag)
    file_renewal_secret = tagged_pair_hash(
        file_renewal_tag,
        client_renewal_secret,
        storage_index,
    )
    peer_id = tubid

    return tagged_pair_hash(bucket_renewal_tag, file_renewal_secret, peer_id)

def demo():
    secret = b32encode(derive_renewal_secret(
        b"lease secretxxxxxxxxxxxxxxxxxxxx",
        b"storage indexxxx",
        b"tub idxxxxxxxxxxxxxx",
    )).decode("ascii")
    print("An example renewal secret: {}".format(secret))

def test():
    # These test vectors created by intrumenting Tahoe-LAFS
    # bb57fcfb50d4e01bbc4de2e23dbbf7a60c004031 to emit `self.renew_secret` in
    # allmydata.immutable.upload.ServerTracker.query and then uploading a
    # couple files to a couple different storage servers.
    test_vector = [
        dict(lease_secret=b"boity2cdh7jvl3ltaeebuiobbspjmbuopnwbde2yeh4k6x7jioga",
             storage_index=b"vrttmwlicrzbt7gh5qsooogr7u",
             tubid=b"v67jiisoty6ooyxlql5fuucitqiok2ic",
             expected=b"osd6wmc5vz4g3ukg64sitmzlfiaaordutrez7oxdp5kkze7zp5zq",
        ),
        dict(lease_secret=b"boity2cdh7jvl3ltaeebuiobbspjmbuopnwbde2yeh4k6x7jioga",
             storage_index=b"75gmmfts772ww4beiewc234o5e",
             tubid=b"v67jiisoty6ooyxlql5fuucitqiok2ic",
             expected=b"35itmusj7qm2pfimh62snbyxp3imreofhx4djr7i2fweta75szda",
        ),
        dict(lease_secret=b"boity2cdh7jvl3ltaeebuiobbspjmbuopnwbde2yeh4k6x7jioga",
             storage_index=b"75gmmfts772ww4beiewc234o5e",
             tubid=b"lh5fhobkjrmkqjmkxhy3yaonoociggpz",
             expected=b"srrlruge47ws3lm53vgdxprgqb6bz7cdblnuovdgtfkqrygrjm4q",
        ),
        dict(lease_secret=b"vacviff4xfqxsbp64tdr3frg3xnkcsuwt5jpyat2qxcm44bwu75a",
             storage_index=b"75gmmfts772ww4beiewc234o5e",
             tubid=b"lh5fhobkjrmkqjmkxhy3yaonoociggpz",
             expected=b"b4jledjiqjqekbm2erekzqumqzblegxi23i5ojva7g7xmqqnl5pq",
        ),
    ]

    for n, item in enumerate(test_vector):
        derived = b32encode(derive_renewal_secret(
            b32decode(item["lease_secret"]),
            b32decode(item["storage_index"]),
            b32decode(item["tubid"]),
        ))
        assert derived == item["expected"] , \
            "Test vector {} failed: {} (expected) != {} (derived)".format(
                n,
                item["expected"],
                derived,
            )
    print("{} test vectors validated".format(len(test_vector)))

test()
demo()

Cancel Secrets

Lease cancellation is unimplemented. Nevertheless, a cancel secret is sent by storage clients to storage servers and stored in lease records.

The scheme for deriving cancel secret in use in Tahoe-LAFS as of 1.16.0 is similar to that used to derive the renewal secret.

The differences are:

  • Use of client renewal tag is replaced by use of client cancel tag.
  • Use of file renewal secret is replaced by use of file cancel tag.
  • Use of bucket renewal tag is replaced by use of bucket cancel tag.
  • client cancel tag is "allmydata_client_cancel_secret_v1".
  • file cancel tag is "allmydata_file_cancel_secret_v1".
  • bucket cancel tag is "allmydata_bucket_cancel_secret_v1".