carddav: answer sync-collection from the change log
The report knowingly violated RFC 6578 twice — no sync-token in the response, every contact on every ask — because nothing read the change log the writes have been keeping since the beginning. The token now carries the log's sequence, the response is the delta since the client's (removals as bare 404s), and a token the server never issued is a 410 naming valid-sync-token, the full-resync cue. The ctag is the sequence too, so getctag and sync-token move together for one indexed read; it moves on identical-byte rewrites where the membership hash did not — a spurious resync that finds nothing, never a missed change. Depth stays unread: the section defines the report at 0 and macOS sends 1. Assisted-by: GLM-5.3 via pi
diff --git a/README.md b/README.md
index 56e791b..42d0ebb 100644
--- a/README.md
+++ b/README.md
@@ -48,10 +48,11 @@ macOS Contacts displays them over Tailscale serve as of 2026-08-14, so
later work has a known-good baseline to change. See
`docs/plans/2026-08-12-one-card-on-macos.md` for what that milestone
established. A contact's etag hashes the card it serves and the
-collection tags hash the membership, so a change to a card reaches synced
-clients on their next poll. Writes arrive the same way: PUT stores the
-submitted card verbatim (RFC 6352 section 6.3.2), and the change log the
-sync tokens count on is written with it, in one transaction.
+collection tags carry the change log's sequence, so a change to a card
+reaches synced clients on their next poll. Writes arrive the same way:
+PUT stores the submitted card verbatim (RFC 6352 section 6.3.2), and the
+change log the sync tokens count on is written with it, in one
+transaction.
Requests are authenticated by the `Tailscale-User-Login` header that
`tailscale serve` injects, which it strips from incoming requests so a
@@ -90,8 +91,12 @@ carry:
the collection itself needs no self-entry.
- `REPORT addressbook-multiget`: `getetag` plus `address-data` for each
requested href.
-- `REPORT sync-collection`: `getetag` only — the client refetches changed
- cards through multiget or `GET` on its own.
+- `REPORT sync-collection`: the members added, changed, or removed
+ since the client's token — `getetag` only, removals as bare 404s —
+ plus the new `DAV:sync-token`. A token the server never issued is a
+ 410 naming `DAV:valid-sync-token`, the client's cue to resync from
+ scratch. The client refetches changed cards through multiget or `GET`
+ on its own.
- `GET` a card: the vCard body plus an `ETag` header.
- `PUT` a card: 201 for a create at an unmapped href, 204 for a replace,
each carrying the strong `ETag` only when what is stored is what was
diff --git a/lib/pro_tacts/store.rb b/lib/pro_tacts/store.rb
index 3a19f40..7e24733 100644
--- a/lib/pro_tacts/store.rb
+++ b/lib/pro_tacts/store.rb
@@ -1,5 +1,4 @@
require "date"
-require "digest"
require "pathname"
require "sequel"
require "sentry-ruby"
@@ -147,20 +146,24 @@ module ProTacts
# The collection's content tag: one value that changes when any card
# is added, removed, or changed and never otherwise, so a client
# comparing two of them learns whether to resync — never what
- # changed. Sorting the id-and-etag lines keeps it independent of the
- # order rows come back in while staying sensitive to membership and
- # content.
- #
- # It costs a read of every card, because an etag is derived from a
- # card's bytes rather than stored beside them. The changes table
- # already holds a monotonic sequence that answers the same question
- # for the price of one indexed read, and switching to it is part of
- # the incremental-sync work: it would change the value a client
- # holds, and it moves on every write rather than on every change,
- # which is a different promise than the one this makes.
+ # changed. The change log's sequence is that value, for the one
+ # indexed read it costs. It moves on a rewrite that stores identical
+ # bytes too, where the hash this replaced did not: the promise
+ # trades a spurious resync — every etag unchanged, nothing fetched —
+ # for never missing a change.
#: () -> String
def ctag
- Digest::SHA256.hexdigest(contacts.map { "#{it.id} #{it.etag}" }.sort.join("\n"))
+ latest_sequence.to_s
+ end
+
+ # The change log's current sequence: the collection's version, and
+ # the number a sync token carries (RFC 6578 section 3). Zero when
+ # nothing has ever been written. Ordered and taken rather than
+ # Dataset#max, which steep reads as Enumerable#max — an Integer
+ # argument, not a column.
+ #: () -> Integer
+ def latest_sequence
+ change_log.order(Sequel.desc(:sequence)).first&.fetch(:sequence).to_i
end
# Birthdays that land on a coming calendar day, in arrival order:
@@ -295,10 +298,11 @@ module ProTacts
end
end
- # The change log from a sequence number on, oldest first. Nothing
- # reads it yet — incremental sync is the task that will — but the
- # entries have to be written from the very first card, because a gap
- # in them is a change some client is never told about.
+ # The change log from a sequence number on, oldest first: the window
+ # a sync-collection report answers from (RFC 6578 section 3.2) — the
+ # entries after the client's token are the changes it has not seen.
+ # The entries have to be written from the very first card, because a
+ # gap in them is a change some client is never told about.
#: (?after: Integer) -> Array[Change]
def changes(after: 0)
change_log.where(Sequel[:sequence] > after).order(:sequence).map { change_from(it) }
diff --git a/lib/pro_tacts/web.rb b/lib/pro_tacts/web.rb
index 9722604..1f45d54 100644
--- a/lib/pro_tacts/web.rb
+++ b/lib/pro_tacts/web.rb
@@ -331,17 +331,55 @@ module ProTacts
# a value like the others rather than a return.
case root.name
when "sync-collection"
- # DAV:sync-collection (RFC 6578 section 3.2). The warm-sync ask
- # is etag-only; a changed etag sends the client back through
- # multiget, so no address-data here.
+ # DAV:sync-collection (RFC 6578 section 3.2): the members
+ # added, changed, or removed since the token the client
+ # holds, and the token naming the state the answer reaches.
+ # The warm-sync ask is etag-only; a changed etag sends the
+ # client back through multiget, so no address-data here.
#
- # Knowingly violates that section twice: it requires the
- # multistatus to carry a DAV:sync-token and to report only what
- # changed since the client's token, and this returns every
- # contact with no token. macOS resyncs the whole collection
- # anyway, so it works; a client that trusts the token would
- # break. Fixing it is the incremental-sync work in the backlog.
- multistatus(contacts.map { etag_response(it) })
+ # The token carries the change log's sequence (see
+ # #sync_token), and the delta is the log after it: one
+ # response per member, the net of everything it did in the
+ # window — a member put twice since the token answers once,
+ # at its current etag, the multiple-changes case section 3.5
+ # allows one response for. A member whose net is a removal
+ # answers as section 3.2 spells it: href and 404, no
+ # propstat. #
+ # Depth stays unread though the section defines the report
+ # only at 0: macOS sends Depth: 1 (fixture 08), and 400-ing
+ # the one real client over a header it ignores serves the
+ # letter over the exchange. sync-level likewise — 1 and
+ # infinite are the same answer here, the collection has no
+ # member collections (section 3.3).
+ token = doc.xpath("//sync-token").first&.text.to_s.strip
+ if token.empty?
+ # No token is the initial sync: every member, changed
+ # (section 3.4).
+ multistatus(contacts.map { etag_response(it) }, sync_token:)
+ elsif (sequence = token[%r{\Ahttp://pro-tacts/sync/(\d+)\z}, 1]) &&
+ sequence.to_i <= store.latest_sequence
+ net = store.changes(after: sequence.to_i).map { it.card_id }.uniq
+ responses = net.map do |id|
+ contact = contacts.find { it.id == id }
+ contact ? etag_response(contact) : missing_response(contact_href(id))
+ end
+ multistatus(responses, sync_token:)
+ else
+ # A token this server never issued, or one naming a state
+ # past the present: the section's DAV:valid-sync-token
+ # precondition, marshalled per RFC 4918 section 16. 410
+ # rather than 403 because its fallback is a full resync —
+ # the request will not always fail, and the state the
+ # token names is gone.
+ response.status = 410
+
+ <<~XML
+ <?xml version="1.0" encoding="UTF-8"?>
+ <d:error xmlns:d="DAV:">
+ <d:valid-sync-token/>
+ </d:error>
+ XML
+ end
when "addressbook-multiget"
# CARDDAV:addressbook-multiget (RFC 6352 section 8.7); the
# address-data the client asks for is section 10.4.
@@ -620,12 +658,16 @@ module ProTacts
!existing.nil? && header.split(",").map(&:strip).include?(existing.etag)
end
- #: (Array[String] responses) -> String
- def multistatus(responses)
+ # A 207 body. The trailing DAV:sync-token is RFC 6578 section 3.2's
+ # — a sync-collection response MUST carry one naming the state the
+ # answer reaches; no other multistatus here has a state to name.
+ #: (Array[String] responses, ?sync_token: String) -> String
+ def multistatus(responses, sync_token: nil)
+ trailing = sync_token ? " <d:sync-token>#{sync_token}</d:sync-token>\n" : ""
<<~XML
<?xml version="1.0" encoding="UTF-8"?>
<d:multistatus xmlns:d="DAV:" xmlns:card="urn:ietf:params:xml:ns:carddav">
- #{responses.join}
+ #{responses.join}#{trailing}
</d:multistatus>
XML
end
@@ -670,6 +712,10 @@ module ProTacts
# An href with no match is reported as a 404 inside the 207 rather than
# failing the request (RFC 6352 section 8.7).
#: (String requested) -> String
+ # A member the collection does not answer for: the multiget miss,
+ # and the removal shape a sync-collection delta reports (RFC 6578
+ # section 3.2 — href and 404, no propstat).
+ #: (String requested) -> String
def missing_response(requested)
<<~XML
<d:response>
diff --git a/test/fixtures/macos-exchange/05-propfind-addressbook-ctag/response b/test/fixtures/macos-exchange/05-propfind-addressbook-ctag/response
index b8c554b..e760787 100644
--- a/test/fixtures/macos-exchange/05-propfind-addressbook-ctag/response
+++ b/test/fixtures/macos-exchange/05-propfind-addressbook-ctag/response
@@ -16,8 +16,8 @@ Content-Type: text/xml
<d:report><d:sync-collection/></d:report>
</d:supported-report>
</d:supported-report-set>
- <cs:getctag>76485478514579ee058182584bc912cda71bdc8fdf5f045a164027091d46b7f8</cs:getctag>
- <d:sync-token>http://pro-tacts/sync/76485478514579ee058182584bc912cda71bdc8fdf5f045a164027091d46b7f8</d:sync-token>
+ <cs:getctag>16</cs:getctag>
+ <d:sync-token>http://pro-tacts/sync/16</d:sync-token>
<d:current-user-privilege-set>
<d:privilege><d:read/></d:privilege>
<d:privilege><d:write/></d:privilege>
diff --git a/test/fixtures/macos-exchange/08-report-sync-collection/response b/test/fixtures/macos-exchange/08-report-sync-collection/response
index a88cb2e..875ad9c 100644
--- a/test/fixtures/macos-exchange/08-report-sync-collection/response
+++ b/test/fixtures/macos-exchange/08-report-sync-collection/response
@@ -4,15 +4,6 @@ Content-Type: text/xml
<?xml version="1.0" encoding="UTF-8"?>
<d:multistatus xmlns:d="DAV:" xmlns:card="urn:ietf:params:xml:ns:carddav">
<d:response>
- <d:href>/dav/addressbook/AB12C345-6789-0DEF-1234-567890ABCDEF.vcf</d:href>
- <d:propstat>
- <d:prop>
- <d:getetag>"ce8bef6ad9abb83e81da4d5aceb7999b43781726f148e2607ed853532f9b8db6"</d:getetag>
- </d:prop>
- <d:status>HTTP/1.1 200 OK</d:status>
- </d:propstat>
-</d:response>
-<d:response>
<d:href>/dav/addressbook/D0504597-CF12-4AF6-9894-43724366D39D.vcf</d:href>
<d:propstat>
<d:prop>
@@ -147,5 +138,6 @@ Content-Type: text/xml
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
</d:response>
+ <d:sync-token>http://pro-tacts/sync/16</d:sync-token>
</d:multistatus>
diff --git a/test/fixtures/macos-exchange/09-propfind-addressbook-bootstrap/response b/test/fixtures/macos-exchange/09-propfind-addressbook-bootstrap/response
index 7d5cfd4..2d8839b 100644
--- a/test/fixtures/macos-exchange/09-propfind-addressbook-bootstrap/response
+++ b/test/fixtures/macos-exchange/09-propfind-addressbook-bootstrap/response
@@ -16,8 +16,8 @@ Content-Type: text/xml
<d:report><d:sync-collection/></d:report>
</d:supported-report>
</d:supported-report-set>
- <cs:getctag>76485478514579ee058182584bc912cda71bdc8fdf5f045a164027091d46b7f8</cs:getctag>
- <d:sync-token>http://pro-tacts/sync/76485478514579ee058182584bc912cda71bdc8fdf5f045a164027091d46b7f8</d:sync-token>
+ <cs:getctag>16</cs:getctag>
+ <d:sync-token>http://pro-tacts/sync/16</d:sync-token>
<d:current-user-privilege-set>
<d:privilege><d:read/></d:privilege>
<d:privilege><d:write/></d:privilege>
diff --git a/test/pro_tacts/test_store.rb b/test/pro_tacts/test_store.rb
index 6a6f562..302de76 100644
--- a/test/pro_tacts/test_store.rb
+++ b/test/pro_tacts/test_store.rb
@@ -211,7 +211,12 @@ class StoreTest < Minitest::Test
end
end
- def test_the_ctag_moves_with_membership_both_ways
+ # The ctag is the change log's sequence, so it moves forward on a
+ # removal too — it never returns to an earlier value, where the
+ # membership hash this replaced did. A client that saw the earlier
+ # value resyncs and finds every etag unchanged; the trade the
+ # sequence makes for one indexed read and a monotonic answer.
+ def test_the_ctag_moves_on_a_removal_and_never_returns
with_store({"aiden" => AIDEN}) do |store|
alone = store.ctag
@@ -219,7 +224,7 @@ class StoreTest < Minitest::Test
refute_equal alone, store.ctag
store.delete("znorth")
- assert_equal alone, store.ctag
+ refute_equal alone, store.ctag
end
end
diff --git a/test/pro_tacts/test_web.rb b/test/pro_tacts/test_web.rb
index af5ffba..06c7649 100644
--- a/test/pro_tacts/test_web.rb
+++ b/test/pro_tacts/test_web.rb
@@ -649,6 +649,21 @@ class WebTest < Minitest::Test
XML
end
+ # The warm-sync ask as macOS sends it (fixture 08): token, level,
+ # etag-only prop.
+ def sync_collection(token)
+ <<~XML
+ <?xml version="1.0" encoding="UTF-8"?>
+ <A:sync-collection xmlns:A="DAV:">
+ <A:sync-token>#{token}</A:sync-token>
+ <A:sync-level>1</A:sync-level>
+ <A:prop>
+ <A:getetag/>
+ </A:prop>
+ </A:sync-collection>
+ XML
+ end
+
def multiget(*ids)
hrefs = ids.map { "<A:href xmlns:A=\"DAV:\">/dav/addressbook/#{it}.vcf</A:href>" }.join("\n ")
@@ -713,23 +728,79 @@ class WebTest < Minitest::Test
end
end
- def test_sync_collection_returns_etags_only
- with_contacts({"aiden" => "Aiden"}) do
- request "/dav/addressbook/", method: "REPORT", input: <<~XML
- <?xml version="1.0" encoding="UTF-8"?>
- <A:sync-collection xmlns:A="DAV:">
- <A:sync-token>http://pro-tacts/sync/1</A:sync-token>
- <A:sync-level>1</A:sync-level>
- <A:prop>
- <A:getetag/>
- </A:prop>
- </A:sync-collection>
- XML
+ # RFC 6578 section 3.2: the delta since the client's token, one
+ # response per net-changed member, and a DAV:sync-token naming the
+ # state the answer reaches. The warm-sync ask is etag-only; a changed
+ # etag sends the client back through multiget, so no address-data
+ # here.
+ def test_sync_collection_reports_the_delta_since_the_token
+ with_contacts({"aiden" => "Aiden"}) do |store|
+ request "/dav/addressbook/", method: "REPORT", input: sync_collection("http://pro-tacts/sync/0")
assert_equal 207, last_response.status
assert_includes last_response.body, "/dav/addressbook/aiden.vcf"
- assert_includes last_response.body, "getetag"
+ assert_includes last_response.body, "<d:getetag>"
refute_includes last_response.body, "address-data"
+ token = last_response.body[%r{<d:sync-token>(.+)</d:sync-token>}, 1]
+
+ # Nothing new under the token it just handed out.
+ request "/dav/addressbook/", method: "REPORT", input: sync_collection(token)
+ assert_equal 207, last_response.status
+ refute_includes last_response.body, "d:response"
+ assert_includes last_response.body, "<d:sync-token>#{token}</d:sync-token>"
+
+ # A change after the token is the whole of the next delta.
+ store.put("aiden", card("aiden", "Aiden Smith"))
+ request "/dav/addressbook/", method: "REPORT", input: sync_collection(token)
+ assert_includes last_response.body, "/dav/addressbook/aiden.vcf"
+ refute_equal token, last_response.body[%r{<d:sync-token>(.+)</d:sync-token>}, 1]
+ end
+ end
+
+ # An empty token is the initial sync (RFC 6578 section 3.4): every
+ # member reported as changed.
+ def test_sync_collection_with_no_token_lists_every_member
+ with_contacts({"aiden" => "Aiden", "znorth" => "Znorth"}) do
+ request "/dav/addressbook/", method: "REPORT", input: sync_collection("")
+
+ assert_equal 207, last_response.status
+ assert_includes last_response.body, "/dav/addressbook/aiden.vcf"
+ assert_includes last_response.body, "/dav/addressbook/znorth.vcf"
+ assert_includes last_response.body, "<d:sync-token>http://pro-tacts/sync/2</d:sync-token>"
+ end
+ end
+
+ # A removal answers as RFC 6578 section 3.2 spells it: href and 404,
+ # no propstat — and nothing else moves.
+ def test_sync_collection_reports_a_removal_as_404
+ with_contacts({"aiden" => "Aiden", "znorth" => "Znorth"}) do |store|
+ request "/dav/addressbook/", method: "REPORT", input: sync_collection("http://pro-tacts/sync/2")
+ token = last_response.body[%r{<d:sync-token>(.+)</d:sync-token>}, 1]
+ store.delete("znorth")
+
+ request "/dav/addressbook/", method: "REPORT", input: sync_collection(token)
+
+ body = last_response.body
+ assert_equal 207, last_response.status
+ assert_includes body, "/dav/addressbook/znorth.vcf"
+ assert_includes body, "HTTP/1.1 404 Not Found"
+ refute_includes body, "aiden"
+ end
+ end
+
+ # A token this server never issued, or one naming a state past the
+ # present: the DAV:valid-sync-token precondition of RFC 6578
+ # section 3.2, marshalled per RFC 4918 section 16.
+ def test_sync_collection_refuses_a_token_it_never_issued
+ with_contacts({"aiden" => "Aiden"}) do
+ request "/dav/addressbook/", method: "REPORT", input: sync_collection("http://pro-tacts/sync/9")
+
+ assert_equal 410, last_response.status
+ assert_includes last_response.body, "<d:valid-sync-token/>"
+
+ request "/dav/addressbook/", method: "REPORT", input: sync_collection("https://elsewhere/sync/1")
+
+ assert_equal 410, last_response.status
end
end