Store contacts as raw vCards in SQLite
The KDL files under data/contacts are gone, and so is every line that read them. Contacts live in a SQLite database holding the card itself, because a server that accepts writes has to keep properties it does not model (RFC 6352 section 6.3.2.2) and no schema can do that. docs/plans/2026-08-24-vcard-storage-and-groups.md makes the argument; docs/plans/2026-08-25-sqlite-schema.md records the schema that came out of it. One database rather than a directory of files because of the change log. The parsed index is a projection of the cards and rake index:rebuild derives it again from them alone, but "what changed since token X" is not recoverable from current state, so a card and its change-log entry are written in one transaction or not at all. Two tests fail part way through a write that has already put the card row in — one at the log, one from a transaction opened around the put — and both leave nothing behind. Sequel defines the tables, in a migration under db/migrations, and queries them over the sqlite3 driver. Its transactions join one already open rather than failing on SQLite's lack of nesting, which is what will let the group fan-out call put from inside a larger write without hand-rolling that. One store serves the process, with Sequel pooling connections behind it, and config.ru builds it and hands it to the app rather than the app reaching for a global. Migrations therefore run at boot, where a bad one fails the deploy instead of somebody's first request, and there is no lazy memo for threads to race. Cards carry created_at and updated_at and change-log rows created_at, written by SQLite so one clock stamps them all. The index tables carry neither: their rows are replaced wholesale on every reindex, so a stamp there would record when the projection ran, not when anything changed. Nothing stores an etag beside a card. An etag is the hash of the card served, and that card is the row, so a column holding one would be a second copy of a fact already there and a second copy that could drift. Contact derives it, through the one constructor reads and writes share. The change log's etag is a different fact and is stored: it is what the card hashed to at that write, which nothing can recompute afterwards. The collection's tags come off the store as well. The ctag hashes every card's id and etag, and Addressbook was the object that held that hash beside the list it came from; a store answers both, so the class was a wrapper around one call and is gone. What was protocol in it — the sync token's URI form — stays in the app, which memoizes the ctag rather than hashing the collection twice for one PROPFIND, as it used to. That hash costs a read of every card, because an etag is derived from a card's bytes rather than stored beside them. The changes table already answers "has anything changed" for one indexed read, and moving the tags onto it belongs with incremental sync rather than here: it would change the value clients hold, and it moves on every write rather than on every change, which is a different promise. lib/sequel/extensions/sole.rb adds the read Sequel lacks. `first` on a filter meant to identify one row answers with one of several rather than saying the filter was too loose; `sole` raises either way it is wrong, and a caller for whom no row is ordinary rescues Sequel::NoMatchingRow. VCard::Parser reads a card's bytes with a StringScanner over the grammar in RFC 2426 section 4, raising on almost nothing: an unrecognized property is a client using the spec, and a card that will not parse is still served from its bytes, it just does not get indexed. vcard.rb keeps escaping, folding, and unfolding, as the writer's half, with tests of their own rather than through the renderer that used to call them. The recorded macOS exchange replays byte for byte: the fixture card is now a .vcf under test/fixtures/cards, seeding a database built under tmp/test-data on every run. There is no import path off the old format. An existing data/contacts is migrated by checking out the last commit that could read it, or not at all — the reasoning is in the schema record. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015Hb6nbbzwbowSPU6sZ5KEB
diff --git a/AGENTS.md b/AGENTS.md
index 1b65355..c576538 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -1,7 +1,8 @@
# Project conventions
A read-only CardDAV server for a family address book, in Ruby on Roda.
-Contacts are KDL files on disk, rendered to vCard 3.0 on request.
+Contacts are vCard 3.0 documents in SQLite, served as the bytes that
+were stored.
`README.md` covers the protocol, the minimal property set macOS Contacts
needs, and the client quirks behind it. Read it before changing any
@@ -13,6 +14,7 @@ response — the shape of those responses is empirical, not arbitrary.
rake # Tests (the default task)
rake steep # Type check lib against the RBS comments in it
rake fixtures # Re-record response fixtures from current behavior
+rake index:rebuild # Derive the parsed index again from the stored cards
rake dev # Dev server, reloading on change (needs fd and entr)
rake profile:install # Render and stage carddav.mobileconfig for approval
```
@@ -30,16 +32,20 @@ is already `pro-tacts` — don't pass `--backlog` or verify it.
```
lib/pro_tacts/
├── web.rb # The Roda app: every route and response body
-├── addressbook.rb # The collection, its ctag, and its sync token
-├── contact.rb # One KDL file → id, vCard, etag
-├── vcard.rb # KDL → vCard 3.0 rendering
+├── store.rb # Sequel over SQLite: cards, change log, derived index
+├── contact.rb # id and vCard, with the etag derived from the card
+├── vcard.rb # vCard 3.0 escaping and folding, and what a card is
+├── vcard/parser.rb # One card's bytes into properties
├── config.rb # Every environment read in the app
├── profile.rb # carddav.mobileconfig generation
└── debug_logger.rb # Full request/response dumps, off by default
-lib/roda/plugins/dav_verbs.rb # PROPFIND and REPORT routing verbs
+lib/roda/plugins/dav_verbs.rb # PROPFIND and REPORT routing verbs
+lib/sequel/extensions/sole.rb # `first`, minus the ambiguity
+db/migrations/ # Sequel migrations, run on every store open
sig/ # RBS for what an inline comment cannot say
docs/rfcs/ # Vendored spec texts the code cites
docs/plans/ # Dated design records
+test/fixtures/cards/ # Seed cards for the test database
test/fixtures/macos-exchange/ # A real client session, replayed
```
@@ -57,6 +63,10 @@ standing:
A fixture diff you didn't intend is the test suite doing its job. Read it
before regenerating.
+`test/fixtures/cards/` holds the vCards the test database is seeded from.
+The database itself is built under `tmp/test-data` on every run and is
+not a fixture; edit a `.vcf` to change what the replay serves.
+
## Gotchas
- Response bodies are built with heredocs in `web.rb`. A comment written
@@ -94,6 +104,56 @@ before regenerating.
`push-transports`) are in no RFC at all.
- Adding a property to a response is a real decision, not a freebie. The
current set was found by removing properties until the client broke.
+- `cards` and `changes` carry timestamps; the index tables deliberately
+ do not, because their rows are thrown away and rebuilt. SQLite writes
+ the stamps, so nothing in Ruby should set one.
+- Schema changes are Sequel migrations under `db/migrations/`, applied
+ by `Store#migrate` on every open — there is no separate migrate task
+ to forget, and no way to serve from a database a deploy left behind.
+ Never edit a migration that has run anywhere; add the next one.
+- Those tables are `STRICT`, so every string column needs `text: true`:
+ Sequel's plain `String` is a varchar, which STRICT refuses. Sequel
+ also literalizes values into UTF-8 SQL, so a card or an id that
+ arrived as ASCII-8BIT has to go through `Store#text` first or it
+ raises on the first non-ASCII byte.
+- The app is handed its store rather than reaching for one:
+ `config.ru` builds it and sets `ProTacts::Web.store`, and a test does
+ the same with a throwaway. There is no global `ProTacts.store`, so
+ nothing in a request depends on load order or on a lazy memo several
+ threads could race into. `Store.connect` is for a database that is not
+ the one being served — a fixture, a test, a task pointed elsewhere.
+- `test/test_helper.rb` sets `ENV` before it requires the app, and the
+ requires cannot all move to the top because of it: the app reads
+ configuration as it loads, since the unhandled-request middleware is
+ given its directory at class-definition time. Set it afterwards and
+ the 404 captures land in `log/` instead of `tmp/`.
+- Only two things in the database cannot be rebuilt: the cards and the
+ change log. Everything else is a projection of the cards that
+ `rake index:rebuild` will make again, so no repair is ever needed for
+ it. A write that touches a card must land its change-log entry in the
+ same transaction, because a client's sync token silently skips
+ whatever the log missed.
+- A read whose filter is meant to identify one row uses `sole`, not
+ `first` — `first` answers with one of several rather than saying the
+ filter was too loose. `sole` raises either way it is wrong:
+ `Sequel::NoMatchingRow` for none, `Sequel::Sole::TooManyRows` for
+ more. A caller for whom no row is ordinary rescues the first, which is
+ what `Store#contact` does for the 404 path. The store loads the
+ extension on every open.
+- An etag is derived from the card, never stored beside it: `Contact`
+ hashes what it serves, and `Contact.for` is the only way to make one.
+ The etag in `changes` is the exception and is not the same fact — it
+ is what the card hashed to at that write, which nothing can recompute
+ once the card moves on.
+- `vcard.rb` no longer raises on what it does not recognize, and must
+ not start again. It used to render a hand-edited format, where an
+ unknown key meant a typo silently losing data; it now reads stored
+ cards, where an unknown property means a client using the spec, and
+ RFC 6352 section 6.3.2.2 requires keeping it. A card that will not
+ parse is still served, it just does not get indexed.
+- `escape` and `fold` in `vcard.rb` have no caller yet. They are the
+ writer's half of the module, kept deliberately for PUT and the web
+ editor, and tested directly rather than through anything that serves.
- `docs/plans/` entries are dated records of what was decided then. Write
a new one rather than editing an old one to match current behavior.
- Types live in the code, as RBS comments: `#:` above a method for its
diff --git a/Gemfile b/Gemfile
index b956271..b62c9a7 100644
--- a/Gemfile
+++ b/Gemfile
@@ -1,8 +1,5 @@
-# frozen_string_literal: true
-
source "https://rubygems.org"
-gem "kdl"
gem "logger"
gem "nokogiri"
gem "puma"
@@ -10,6 +7,8 @@ gem "rackup"
gem "rake"
gem "roda"
gem "sentry-ruby"
+gem "sequel"
+gem "sqlite3"
group :development do
gem "hegeltest", git: "https://github.com/meganemura/hegel-ruby"
diff --git a/Gemfile.lock b/Gemfile.lock
index 28cc642..d5b6241 100644
--- a/Gemfile.lock
+++ b/Gemfile.lock
@@ -9,7 +9,6 @@ GEM
remote: https://rubygems.org/
specs:
ast (2.4.3)
- base64 (0.2.0)
bigdecimal (3.1.9)
concurrent-ruby (1.3.8)
csv (3.3.6)
@@ -26,11 +25,6 @@ GEM
rdoc (>= 4.0.0)
reline (>= 0.4.2)
json (2.21.2)
- kdl (2.2.0)
- base64 (~> 0.2.0)
- bigdecimal (~> 3.1.6)
- racc (~> 1.5)
- simpleidn (~> 0.2.1)
language_server-protocol (3.17.0.6)
listen (3.10.0)
logger
@@ -89,7 +83,11 @@ GEM
bigdecimal
concurrent-ruby (~> 1.0, >= 1.0.2)
logger
- simpleidn (0.2.3)
+ sequel (5.107.0)
+ bigdecimal
+ sqlite3 (2.9.6-aarch64-linux-gnu)
+ sqlite3 (2.9.6-arm64-darwin)
+ sqlite3 (2.9.6-x86_64-linux-gnu)
steep (2.1.0.dev.1)
concurrent-ruby (>= 1.1.10)
csv (>= 3.0.9)
@@ -123,7 +121,6 @@ PLATFORMS
DEPENDENCIES
hegeltest!
irb
- kdl
logger
minitest
nokogiri
@@ -134,11 +131,12 @@ DEPENDENCIES
roda
ruby-lsp
sentry-ruby
+ sequel
+ sqlite3
steep (= 2.1.0.dev.1)
CHECKSUMS
ast (2.4.3) sha256=954615157c1d6a382bc27d690d973195e79db7f55e9765ac7c481c60bdb4d383
- base64 (0.2.0) sha256=0f25e9b21a02a0cc0cea8ef92b2041035d39350946e8789c562b2d1a3da01507
bigdecimal (3.1.9) sha256=2ffc742031521ad69c2dfc815a98e426a230a3d22aeac1995826a75dabfad8cc
concurrent-ruby (1.3.8) sha256=b2f1be836e968ccc78ccfce277ea79c72a88633f22306782c16ff23fb415d1e1
csv (3.3.6) sha256=aba61e7e507a66f03d45cb1f3c4b6359861c3504038b422962875dce099e4456
@@ -152,7 +150,6 @@ CHECKSUMS
io-console (0.9.2) sha256=efa74f891dd03c0939a931dfc6e74c2813d904763d456ea9762b0525e748db08
irb (1.18.0) sha256=de9454a0703a54704b9811a5ef31a60c86949fbf4013fcf244fabc7c775248e3
json (2.21.2) sha256=1f1d3b7cf2b3ba1a69beca0bb6db13d5438b80bff3cd54cdaaa620b9b07c1c6a
- kdl (2.2.0) sha256=46e56c6686dbb681625a0d6c6193e4b732d6627e8758f8d5e9295309ded958e6
language_server-protocol (3.17.0.6) sha256=5ef2c0c138f8267e1bc631d3328347d354f96724b0af22f2c79516120443b7f0
listen (3.10.0) sha256=c6e182db62143aeccc2e1960033bebe7445309c7272061979bb098d03760c9d2
logger (1.7.0) sha256=196edec7cc44b66cfb40f9755ce11b392f21f7967696af15d274dde7edff0203
@@ -181,7 +178,10 @@ CHECKSUMS
ruby-lsp (0.26.11) sha256=4cc8f1587237ff8c9031680b5491df901a82559eeaa1ae96863ffe700b32583d
securerandom (0.4.1) sha256=cc5193d414a4341b6e225f0cb4446aceca8e50d5e1888743fac16987638ea0b1
sentry-ruby (6.7.0) sha256=b4b915d79a0395ad02106d4012aa1914641456e042ce36393f15bd1630d8db42
- simpleidn (0.2.3) sha256=08ce96f03fa1605286be22651ba0fc9c0b2d6272c9b27a260bc88be05b0d2c29
+ sequel (5.107.0) sha256=019629313205295b007f6df2c5eab19ddc40710d549fd445ac300f2853a370db
+ sqlite3 (2.9.6-aarch64-linux-gnu) sha256=d8b1f7d23efd7abac285775a9566562fc7debfef79d594e3a20354406fb7907c
+ sqlite3 (2.9.6-arm64-darwin) sha256=849b5d7f795e60fe25076d62c72dd722beb45b3850b516ad978d60ee848ec15b
+ sqlite3 (2.9.6-x86_64-linux-gnu) sha256=613188ce02f614126ddbc38c5e217ccffd6306d0dcd9adca9764547aa890a634
steep (2.1.0.dev.1) sha256=af0b17c8d45a5e76d66e5218512db21e13e6a173e0763a109c2af97511cd5215
strscan (3.1.8) sha256=aae2db611a225559f21ffbb71765c9a4e60fd262534a9ea84f4f11c7f32f679e
terminal-table (4.0.0) sha256=f504793203f8251b2ea7c7068333053f0beeea26093ec9962e62ea79f94301d2
diff --git a/README.md b/README.md
index 14478ed..146a75f 100644
--- a/README.md
+++ b/README.md
@@ -9,31 +9,46 @@ A CardDAV server for my family.
- Apple devices only (it might happen to work on other CardDAV clients, but
only accidentally)
-## Features
+## Features it is being built for
+
+None of these exist yet; they are what the design is shaped around. What
+does work is under Status below.
- Groups with attributes (e.g., an address shared by all members)
- Selective sync (choose which contacts to sync rather than all-or-nothing)
-- Plaintext storage in git (version history, human-readable)
+- vCards exported to git after each write (version history, human-readable)
## Status
-Read-only and serving real data. Contacts live as KDL files under
-`data/contacts` (override the root with `PRO_TACTS_DATA_DIR`), one contact per
-file, the filename doubling as the contact ID and the vCard UID:
+Read-only and serving real data. Contacts live in a SQLite database at
+`data/contacts.db` (override the root with `PRO_TACTS_DATA_DIR`, or the
+database path alone with `PRO_TACTS_DATABASE`), one row per contact,
+holding the vCard itself:
-```kdl
-name "John Smith"
-phone "+1-555-1234" type="mobile"
-email "john@example.com"
```
+BEGIN:VCARD
+VERSION:3.0
+N:Smith;John;;;
+FN:John Smith
+TEL;TYPE=mobile:+1-555-1234
+UID:john-smith
+END:VCARD
+```
+
+Storing the card rather than a parse of it is what lets the server keep
+properties it does not model, which RFC 6352 section 6.3.2.2 requires of
+anything accepting writes; `docs/plans/2026-08-24-vcard-storage-and-groups.md`
+makes the case at length. The same database holds the change log, which
+is the one thing in it that cannot be rebuilt, and an index of parsed
+properties, which can — `rake index:rebuild` derives that again from the
+cards alone.
macOS Contacts displays them over Tailscale serve as of 2026-08-14, so
later work has a known-good baseline to change. See
`docs/plans/2026-08-12-one-card-on-macos.md` for what that milestone
-established. Etags, the ctag, and the sync token are derived from file
-state — a contact's etag hashes its rendered vCard, and the collection
-tags hash the membership — so an edit on disk reaches synced clients on
-their next poll.
+established. A contact's etag hashes the card it serves and the
+collection tags hash the membership, so a change to a card reaches synced
+clients on their next poll.
Requests are authenticated by the `Tailscale-User-Login` header that
`tailscale serve` injects, which it strips from incoming requests so a
diff --git a/Rakefile b/Rakefile
index 159fbbe..71ce95d 100644
--- a/Rakefile
+++ b/Rakefile
@@ -19,9 +19,12 @@ desc "Regenerate macOS exchange response fixtures from current responses"
task :fixtures do
# Mirrors test/test_helper.rb, which cannot be required here without
# minitest/autorun running its at_exit hook inside rake.
- ENV["PRO_TACTS_DATA_DIR"] = (Pathname.new(__dir__) / "test/fixtures").to_s
+ data_dir = Pathname.new(__dir__) / "tmp" / "test-data"
+ ENV["PRO_TACTS_DATA_DIR"] = data_dir.to_s
require "pro_tacts/web"
+ require_relative "test/fixture_data"
require_relative "test/pro_tacts/exchange_fixtures"
+ ProTacts::Web.store = FixtureData.install(data_dir)
ExchangeFixtures.record_responses(ProTacts::Web)
end
diff --git a/Steepfile b/Steepfile
index d4ca4ea..472c068 100644
--- a/Steepfile
+++ b/Steepfile
@@ -10,5 +10,5 @@ target :lib do
check "lib", inline: true
signature "sig"
- library "digest", "fileutils", "logger", "pathname"
+ library "digest", "fileutils", "logger", "pathname", "strscan"
end
diff --git a/config.ru b/config.ru
index c6ab6b3..afdbb86 100644
--- a/config.ru
+++ b/config.ru
@@ -8,12 +8,16 @@ require "sentry-ruby"
$LOAD_PATH.unshift(Pathname.new(__dir__) / "lib")
require "pro_tacts/web"
require "pro_tacts/sentry_scrubber"
+require "pro_tacts/store"
config = ProTacts.config
-# A fresh checkout has no contacts dir; an empty address book beats a
-# 500 on every request.
-FileUtils.mkdir_p(config.contacts_dir)
+# A fresh checkout has no database. Opening it here creates and migrates
+# one on this thread, before any request, rather than leaving several
+# request threads to race into an empty schema -- and a migration that
+# fails takes the deploy down rather than someone's first request.
+FileUtils.mkdir_p(config.data_dir)
+ProTacts::Web.store = ProTacts::Store.at(config.database_path)
# A nil DSN initializes Sentry but leaves it inert: capture_message
# returns nil and the rack middleware reports nothing.
diff --git a/db/migrations/001_create_contacts_schema.rb b/db/migrations/001_create_contacts_schema.rb
new file mode 100644
index 0000000..8aa5e16
--- /dev/null
+++ b/db/migrations/001_create_contacts_schema.rb
@@ -0,0 +1,79 @@
+# The whole schema, in one migration: this is the first one, and every
+# table in it arrived together.
+#
+# `text: true` on every string column because these are STRICT tables,
+# which admit only SQLite's own type names, and Sequel's plain String is
+# a varchar.
+#
+# Timestamps are UTC ISO 8601 to the millisecond, written by SQLite
+# rather than by Ruby so that one clock stamps every row. There is no
+# ON UPDATE in SQLite, so a column default covers the insert and
+# Store#put sets updated_at itself on the way through.
+# A local rather than a constant: a migration is loaded into the top
+# level, and the next one would collide with it.
+now = "(strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))"
+
+Sequel.migration do
+ change do
+ # The card as submitted, and nothing derived from it. No etag column:
+ # an etag is a hash of the card served, which is this row, so storing
+ # one would be a second copy of a fact already here and a second copy
+ # that could drift. Contact computes it. The change log's etag is a
+ # different thing and does need storing, see below.
+ create_table(:cards, strict: true) do
+ String :id, text: true, primary_key: true, null: false
+ String :vcard, text: true, null: false
+ String :created_at, text: true, null: false, default: Sequel.lit(now)
+ String :updated_at, text: true, null: false, default: Sequel.lit(now)
+ end
+
+ # Sync history, and the one table with nothing behind it. A card id
+ # rather than a foreign key, because a tombstone has to outlive the
+ # card it is about. `primary_key` gives an AUTOINCREMENT rather than
+ # a bare rowid, because a sequence number reused after a delete would
+ # let a client holding an older token skip a change it never saw.
+ #
+ # The etag here is the one thing in the schema that is a hash and
+ # still has to be stored: it is what the card hashed to at that
+ # write, and the card has moved on since, so nothing can recompute
+ # it.
+ create_table(:changes, strict: true) do
+ primary_key :sequence, type: Integer
+ String :card_id, text: true, null: false
+ String :action, text: true, null: false
+ String :etag, text: true
+ String :created_at, text: true, null: false, default: Sequel.lit(now)
+ constraint(:action_is_known, action: %w[put delete])
+ end
+
+ # Derived from the cards from here down: a projection, not a copy,
+ # and safe to drop. `position` is the property's place in the card,
+ # which is the only identity a repeated property has. Names are
+ # compared without case because `FN` and `fn` are one name (RFC 6350
+ # section 3.3, which RFC 2426 leaves unsaid), and parameter values
+ # because Contacts uppercases them on every card it touches (see
+ # docs/plans/2026-08-24-corrections-from-the-first-write.md).
+ # No timestamps from here down. These rows are replaced wholesale
+ # every time a card is indexed, so a created_at on one would record
+ # when the projection last ran and imply a history the property does
+ # not have.
+ create_table(:card_properties, strict: true) do
+ foreign_key :card_id, :cards, type: String, text: true, null: false, on_delete: :cascade
+ Integer :position, null: false
+ String :property_group, text: true
+ String :name, text: true, null: false, collate: "NOCASE"
+ String :value, text: true, null: false
+ primary_key [:card_id, :position]
+ index [:name, :value]
+ end
+
+ create_table(:card_parameters, strict: true) do
+ String :card_id, text: true, null: false
+ Integer :position, null: false
+ String :name, text: true, null: false, collate: "NOCASE"
+ String :value, text: true, null: false, collate: "NOCASE"
+ foreign_key [:card_id, :position], :card_properties, on_delete: :cascade
+ index [:name, :value]
+ end
+ end
+end
diff --git a/docs/plans/2026-08-25-sqlite-schema.md b/docs/plans/2026-08-25-sqlite-schema.md
new file mode 100644
index 0000000..22bf0f4
--- /dev/null
+++ b/docs/plans/2026-08-25-sqlite-schema.md
@@ -0,0 +1,224 @@
+# The contacts database
+
+2026-08-25. The storage half of
+`2026-08-24-vcard-storage-and-groups.md`, built. That record argued for
+SQLite with the card in a column and said why; this one is the schema
+that came out of it, and the six choices inside it that were not
+obvious.
+
+KDL is gone, and so is the code that read it. There is no import path:
+the storage record already had the web UI owning creation and Monica
+seeding the rest, and a renderer kept alive only to move one directory
+once is a second contact format to maintain for as long as anyone
+forgets to delete it. An existing `data/contacts` is migrated by
+checking out the last commit that could read it, or not at all.
+
+## What the tables are for
+
+```
+cards id, vcard, created_at, updated_at
+changes sequence, card_id, action, etag, created_at
+card_properties card_id, position, property_group, name, value
+card_parameters card_id, position, name, value
+```
+
+Sequel defines them, in a migration under `db/migrations/`, and queries
+them. The sqlite3 gem is still there, as the driver underneath it.
+`Store#migrate` runs the migrator on every open, which on a current
+database is one read of `schema_info`: cheap enough to leave in, and it
+removes the deploy step everyone forgets.
+
+The first two are authoritative. The last two are a projection of the
+first, and `rake index:rebuild` derives them again from the cards alone,
+which is the property the storage record leans on when it says nothing
+is authoritative twice.
+
+Groups, membership, and provenance are not here yet. They are the other
+half of that record and nothing in this schema forecloses them.
+
+## Four decisions
+
+**The etag is computed, not stored.** The first cut of this schema had
+an `etag` column on `cards`, reasoning that the task asks for a write
+landing the card, its etag, and its change-log entry together, and that
+groups would eventually make the served card differ from the stored one.
+Both halves were wrong about where the etag belongs.
+
+An etag is the hash of the card served, and that card is the row. A
+column holding it is a second copy of a fact already present, and the
+only thing a second copy adds is the chance of disagreeing — a card
+written without its etag, by a repair or a later migration, serves a
+stale one silently. It also left `Contact` with two constructors, one
+deriving the etag and one trusting a column, where everything the server
+actually served took the second path and nothing checked they agreed.
+
+The etag that genuinely cannot be recomputed is `changes.etag`: what the
+card hashed to at that write, when the card has since moved on. That one
+is stored, in the table that cannot be rebuilt, which is the same rule
+the rest of this schema follows. The atomicity the task asks for is the
+log entry, and it is in the transaction.
+
+Groups do not change this. The served card becomes stored plus
+inherited, so the etag becomes a hash of more inputs, still all of them
+in the database. What groups do change is cost: a `Depth:1` listing
+returns an etag per contact and no bodies, so composing every card to
+hash it stops being free. If that ever bites, the column comes back as
+what it would have been all along — a cache, labelled as one, repaired
+the way the index is.
+
+**`changes.sequence` is AUTOINCREMENT.** SQLite reuses a bare rowid
+after the highest row is deleted. A sync token is a promise that
+everything above it is unseen, and a reused sequence number breaks that
+promise silently: the client is told nothing changed. AUTOINCREMENT
+costs a row in `sqlite_sequence` and buys monotonicity.
+
+**Only the authoritative tables carry timestamps.** A card gets
+`created_at` and `updated_at`; a change-log row gets `created_at` alone,
+because it is written once and never touched again. The index rows get
+neither. They are replaced wholesale every time a card is indexed, so a
+stamp on one would say when the projection last ran and imply a history
+the property does not have — the card's own `updated_at` is the honest
+answer to when that data last changed.
+
+SQLite writes them, not Ruby, so that rows written in one transaction
+agree; there is no ON UPDATE, so a column default covers the insert and
+`Store#put` stamps `updated_at` itself in the upsert. `updated_at` moves
+on every accepted write, including one that stores identical bytes: it
+records when the store last took a card, which is a different question
+from when the card last differed, and the etag already answers that one.
+
+**The change log is never pruned.** It grows by one row per write,
+forever, and that is the choice rather than an oversight: a sync token is
+a promise about everything above it, so the safe moment to drop an entry
+is once every client has moved past it, and this server does not track
+clients. At a family's rate of edits the table is measured in kilobytes
+per decade. If it ever matters, the shape of the answer is a floor —
+discard entries below a sequence number and refuse tokens older than it,
+sending those clients through a full resync — not a size cap that
+silently loses history a token still refers to.
+
+**`changes.card_id` is not a foreign key.** A tombstone is about a card
+that no longer exists; that is the whole point of it. The index rows do
+carry a foreign key, with a cascade, because they should never outlive
+their card.
+
+**Writes nest.** SQLite has no nested transactions, and the fan-out the
+storage record describes — one member's card rewriting its group and
+every other member's card — is writes inside writes. Sequel's
+`transaction` joins one already open rather than opening a second, so
+`put` can be called from inside a larger write and the whole thing is
+still one atomic unit. That behavior is the largest single reason the
+store goes through Sequel rather than driving sqlite3 directly; hand
+transaction nesting is exactly the kind of thing that looks fine until
+the day half a fan-out commits.
+
+## Parsing inverts
+
+`vcard.rb` used to render KDL and raise on any key it did not know.
+`VCard::Parser` now reads vCards and raises on almost nothing: a
+property is a name, its parameters, and the text of its value, and it
+does not care what any of them mean. That is what RFC 6352 section
+6.3.2.2 asks for, and it is why the parse is a `StringScanner` over the
+grammar in RFC 2426 section 4 rather than anything property-aware. It is
+a class rather than a set of functions because the scanner is a position
+that every step moves, and threading that through arguments only
+disguises the state.
+
+The strictness was right for what it guarded and wrong for this. A typo
+in a hand-edited file is lost data; an unrecognized property in a stored
+card is a client using the spec. It went out with the format it was
+protecting.
+
+What survived the renderer is `escape` and `fold`, which nothing calls
+yet. They are the writer's half of `vcard.rb` and the next thing to
+write — a PUT, or the web editor — needs both on its first line, so they
+are kept and tested directly instead of being deleted and rewritten.
+`unfold` sits with them rather than in the parser, because it is fold's
+inverse and reads as one pair with it.
+
+Parsing fails open. A card that will not parse is still served — the
+bytes are in the column — it just contributes nothing to the index, and
+`rebuild_index` returns the ids it could not read. Surfacing those to a
+human is the web UI's job, not the store's.
+
+## What the index does not do
+
+Values are indexed as they are stored, still escaped, and not split on
+the component separator. Unescaping is per-property semantics, and the
+comparison the storage record needs for classifying a submitted card has
+to normalize far more than that (see
+`2026-08-24-corrections-from-the-first-write.md`) — it will do its own
+normalizing, from the card, and would not have trusted a normalization
+baked into the index anyway.
+
+What the index does carry is enough to query on: property name and
+value, parameter name and value, and the position that gives a repeated
+property its only identity. Names are compared without case, because
+`FN` and `fn` are one name and not two: RFC 6350 section 3.3 says so
+outright for vCard 4.0, RFC 2426 does not say it in its own text, and no
+client has ever meant them differently. Parameter values are compared
+the same way for a blunter reason — Contacts uppercases them on every
+card it touches, so case there is the client's habit rather than
+anything a user typed.
+
+## One thing Sequel is missing
+
+`first` on a filter meant to identify one row will return one of several
+without complaint, which turns a corrupted table or a filter that was
+never as narrow as it looked into a plausible-looking answer.
+`lib/sequel/extensions/sole.rb` adds `sole`, which raises either way it
+is wrong: `Sequel::NoMatchingRow` — the error Sequel's own `first!`
+raises — for none, and `Sequel::Sole::TooManyRows` for more. One method
+rather than a nil-returning and a raising variant, because the two would
+have been the same assertion written twice; a caller for whom no row is
+ordinary says so by rescuing, which is what `Store#contact` does to turn
+a missing href back into the 404 it is.
+
+Its error message names the table and never the SQL. A filter can carry
+card content, and an exception message is one of the things that reaches
+Sentry.
+
+## Two things Sequel makes awkward
+
+The tables are `STRICT`, which admits only SQLite's own type names, and
+Sequel's `String` is a `varchar(255)`. Every string column therefore
+carries `text: true`. It is noise on each line and the alternative —
+dropping `STRICT` — trades a column-by-column annotation for a table
+that will quietly accept an Integer where a card belongs.
+
+Values are literalized into UTF-8 SQL rather than bound, so a card that
+arrived as ASCII-8BIT raises `Encoding::UndefinedConversionError` on the
+first byte above 7 bits. `Store#text` relabels ids and cards at the edge.
+Under the raw driver the same input failed differently and worse: it was
+bound as a BLOB, which compares equal to nothing, so a card fetched by
+the id it was stored under simply came back missing. Loud beats silent,
+but neither is free, and the coercion has a test of its own.
+
+## One store, not one per request
+
+A single `Store` serves the whole process, with Sequel pooling the
+connections behind it, the way the library expects to be used. Requests
+read it; nothing in a route opens a database.
+
+The first draft opened and closed one inside each request, which read as
+prudence — a connection that does not outlive a request is one Puma's
+threads cannot share — and was really just refusing to use the pool that
+already solves that. It also meant a single request could open two, one
+for the collection and one for the card, which is the tell.
+
+The second draft made it a lazily memoized global, `ProTacts.store`, and
+had `config.ru` touch it once at boot to force the migrations. Two things
+were wrong with that. The boot line was a getter called for its side
+effect, which reads like something safe to delete. And the memo it was
+warming was `@store ||=`, which is not thread safe: eight threads racing
+a fresh one built more than one store in twenty rounds out of twenty, so
+the warm-up was not an optimization but the thing holding the race shut.
+
+So the store is injected. `config.ru` builds it and hands it to the app
+through `ProTacts::Web.store=`, which keeps it in Roda's own `opts` and
+is therefore frozen along with the app — a write after that raises
+rather than quietly taking effect. There is no global to race, migrations
+run at boot where a bad one fails the deploy instead of somebody's first
+request, and a test hands the app a throwaway store the same way. What
+`Store.connect` is for is the other case: a database that is not the one
+this process serves from.
diff --git a/lib/pro_tacts/addressbook.rb b/lib/pro_tacts/addressbook.rb
deleted file mode 100644
index dba85a1..0000000
--- a/lib/pro_tacts/addressbook.rb
+++ /dev/null
@@ -1,34 +0,0 @@
-require "digest"
-
-require "pro_tacts/contact"
-
-module ProTacts
- # The address book collection: every contact under the contacts
- # directory, plus the collection-wide state derived from those members.
- # The ctag changes when any card is added, removed, or changed and
- # nothing else, so a client comparing two of them learns whether a
- # resync is needed — never what changed.
- #
- # The signature lives in sig/pro_tacts/addressbook.rbs, for the same
- # reason Contact's does.
- # @rbs skip
- class Addressbook < Data.define(:contacts)
- def self.load(directory)
- new(contacts: Contact.all(directory))
- end
-
- # Sorting the id-and-etag lines makes the value independent of
- # directory listing order while staying sensitive to membership and
- # content.
- def ctag
- Digest::SHA256.hexdigest(contacts.map { "#{it.id} #{it.etag}" }.sort.join("\n"))
- end
-
- # Sync tokens are opaque to the client (RFC 6578 section 3); the URI
- # form is conventional. Built on the ctag so a client polling either
- # one sees changes at the same points.
- def sync_token
- "http://pro-tacts/sync/#{ctag}"
- end
- end
-end
diff --git a/lib/pro_tacts/config.rb b/lib/pro_tacts/config.rb
index bcda6d9..e9bf084 100644
--- a/lib/pro_tacts/config.rb
+++ b/lib/pro_tacts/config.rb
@@ -31,19 +31,23 @@ module ProTacts
!value.nil? && value.match?(TRUTHY)
end
- # Root data directory: holds the contacts directory and, later, the
- # database. Overridable with PRO_TACTS_DATA_DIR.
+ # Root data directory: holds the contacts database, and the exported
+ # card mirror once there is one. Overridable with
+ # PRO_TACTS_DATA_DIR.
#: () -> Pathname
def data_dir
Pathname.new(@env.fetch("PRO_TACTS_DATA_DIR", "data"))
end
- # Contacts live at data/contacts, one KDL file per contact; the
- # filename is the contact ID. See
- # docs/plans/2026-01-12-carddav-architecture.md.
+ # The contacts database: every card, the change log, and the index
+ # derived from the cards (see ProTacts::Store). PRO_TACTS_DATABASE
+ # overrides the whole path rather than a name under the data
+ # directory, so a deployment can put the database on a different
+ # volume from the exports.
#: () -> Pathname
- def contacts_dir
- data_dir / "contacts"
+ def database_path
+ path = @env.fetch("PRO_TACTS_DATABASE", nil)
+ path.nil? ? data_dir / "contacts.db" : Pathname.new(path)
end
# Where requests the app could not answer are kept, one directory per
diff --git a/lib/pro_tacts/contact.rb b/lib/pro_tacts/contact.rb
index 3470470..d380aa2 100644
--- a/lib/pro_tacts/contact.rb
+++ b/lib/pro_tacts/contact.rb
@@ -1,50 +1,41 @@
require "digest"
-require "kdl"
-require "pathname"
-
-require "pro_tacts/vcard"
module ProTacts
- # A contact parsed from one KDL file under the contacts directory; the
- # file is a bare KDL document, and the filename is the id, which maps
- # to the vCard UID (see docs/plans/2026-01-12-carddav-architecture.md).
+ # One contact: its id, the vCard bytes served for it, and the etag over
+ # those bytes. The id is the vCard's UID and the name in its href (see
+ # docs/plans/2026-01-12-carddav-architecture.md).
+ #
+ # The etag hashes the card that goes out, so it changes exactly when
+ # what the client downloads changes. It is derived here and stored
+ # nowhere: a contact read back from the store hashes its card the same
+ # way one about to be written does, which is why there is one
+ # constructor rather than one that computes and one that trusts. It is
+ # in the entity-tag's quoted form (RFC 7232 section 2.3), which is what
+ # both the ETag header and getetag properties carry.
#
- # The etag hashes the rendered vCard rather than the file's bytes or
- # mtime, so it changes exactly when what the client downloads changes:
- # git rewrites mtimes on every checkout, and a reformat-only edit
- # changes the bytes without changing the card. It is stored in the
- # entity-tag's quoted form (RFC 7232 section 2.3), which is what both
- # the ETag header and getetag properties carry.
+ # Stored and served are the same bytes today. Once a group contributes
+ # properties at render time they will not be, and this becomes the hash
+ # of the composed card — still derived, from more inputs.
#
# The signature lives in sig/pro_tacts/contact.rbs: a Data class has
# no constant super class for the inline syntax to read.
# @rbs skip
class Contact < Data.define(:id, :vcard, :etag)
- # Ids end up in paths and arrive from client-supplied hrefs, so a
- # filename outside this charset cannot be served.
+ # Ids end up in paths and arrive from client-supplied hrefs, so an id
+ # outside this charset cannot be served.
ID_FORMAT = /\A[\w-]+\z/
- # Every contact in the directory, one file per contact. An empty
- # directory is a valid empty address book, and dotfiles are skipped
- # (Finder drops .DS_Store into any directory it opens). Anything
- # else raises: a bad file 500s the request and lands in Sentry
- # rather than quietly serving a partial address book. A missing
- # directory surfaces as Errno::ENOENT from Pathname#children.
- def self.all(directory)
- Pathname.new(directory).children
- .reject { it.basename.to_s.start_with?(".") }
- .map { parse(it) }
- end
-
- # File to Contact; raises on any filename or content that would make
- # the contact unloadable.
- def self.parse(path)
- path = Pathname.new(path)
- id = path.basename(".kdl").to_s
+ # A contact from its id and its card. The only way to make one: an
+ # etag that came from anywhere but the card in hand is an etag that
+ # can be wrong.
+ def self.for(id:, vcard:)
raise ArgumentError, "invalid contact id: #{id}" unless id.match?(ID_FORMAT)
- vcard = VCard.render(KDL.parse(path.read), uid: id)
- new(id:, vcard:, etag: %("#{Digest::SHA256.hexdigest(vcard)}"))
+ new(id:, vcard:, etag: etag_for(vcard))
+ end
+
+ def self.etag_for(vcard)
+ %("#{Digest::SHA256.hexdigest(vcard)}")
end
end
end
diff --git a/lib/pro_tacts/store.rb b/lib/pro_tacts/store.rb
new file mode 100644
index 0000000..8bbf088
--- /dev/null
+++ b/lib/pro_tacts/store.rb
@@ -0,0 +1,295 @@
+
+require "digest"
+require "pathname"
+require "sequel"
+
+require "pro_tacts/contact"
+require "pro_tacts/vcard/parser"
+
+Sequel.extension :migration
+
+module ProTacts
+ # The transactional store: one SQLite database holding each contact's
+ # card exactly as it was submitted.
+ #
+ # Three kinds of state live here and they are not equally precious.
+ # The cards are the truth about contact data, and the reason they are
+ # stored as the card rather than a parse of it is
+ # docs/plans/2026-08-24-vcard-storage-and-groups.md. The change log is
+ # the truth about history, and is why this is one database rather than
+ # a directory of files: "what changed since token X" cannot be
+ # recovered from current state, so a card, its etag, and its
+ # change-log entry have to land or fail together. Everything else is
+ # an index derived from the cards, and #rebuild_index will make it
+ # again from nothing.
+ #
+ # Sequel's transactions join one already open rather than failing on
+ # SQLite's lack of nesting, which is what lets the group fan-out this
+ # design needs — one member's card rewriting its group and every other
+ # member's card — call #put from inside a larger write and still be a
+ # single atomic unit.
+ #
+ # The signature lives in sig/pro_tacts/store.rbs, for the Change Data
+ # class the inline syntax cannot read.
+ class Store
+ # @rbs @database: Sequel::Database
+
+ # One entry in the change log. The signature is in
+ # sig/pro_tacts/store.rbs with Store's own.
+ # @rbs skip
+ Change = Data.define(:sequence, :card_id, :action, :etag, :created_at)
+
+ # SQLite has no ON UPDATE, so the column default stamps a row on
+ # insert and this stamps it again on the way past. Same expression as
+ # the migration's, deliberately: the database keeps the clock, so two
+ # rows written in one transaction agree.
+ NOW = Sequel.lit("strftime('%Y-%m-%dT%H:%M:%fZ', 'now')") #: untyped
+
+ # Every write is short, so a writer that finds the database locked is
+ # better off waiting than raising: two requests saving at once is
+ # ordinary, and SQLITE_BUSY straight back to the client is not.
+ BUSY_TIMEOUT = 5_000 #: Integer
+
+ # Sequel's migrations, run on open. They ship with the code rather
+ # than with a deployment, so the path is relative to this file.
+ # `__dir__` is nil only for code with no file behind it, which a
+ # required library is not, and the assertion says so to Steep.
+ MIGRATIONS = Pathname.new(
+ __dir__ #: String
+ ).parent.parent / "db" / "migrations" #: Pathname
+
+ # The store a process keeps: config.ru builds one of these and hands
+ # it to the app, and Sequel pools the connections behind it. Not
+ # named `open`, which every object already has from Kernel.
+ #: (Pathname | String path) -> Store
+ def self.at(path)
+ new(Sequel.sqlite(path.to_s, timeout: BUSY_TIMEOUT))
+ end
+
+ # The same, closed again when the block returns, for a database that
+ # is not the one this process serves from: a fixture, a test, a rake
+ # task pointed somewhere else.
+ #: [T] (Pathname | String path) { (Store) -> T } -> T
+ def self.connect(path)
+ store = at(path)
+ begin
+ yield store
+ ensure
+ store.close
+ end
+ end
+
+ #: (Sequel::Database database) -> void
+ def initialize(database)
+ @database = database
+ # WAL so a poll can read while a save writes. Foreign keys need no
+ # pragma: Sequel turns them on for every SQLite connection it
+ # opens, which is what makes the index's cascades fire.
+ @database.run("PRAGMA journal_mode = WAL")
+ # lib/sequel/extensions/sole.rb, for the reads that mean one row.
+ @database.extension(:sole)
+ migrate
+ end
+
+ #: () -> void
+ def close
+ @database.disconnect
+ end
+
+ # Every contact, ordered by id so that a listing does not depend on
+ # what SQLite feels like returning.
+ #: () -> Array[Contact]
+ def contacts
+ cards.order(:id).map { contact_from(it) }
+ end
+
+ # The collection's content tag: one value that changes when any card
+ # is added, removed, or changed and never otherwise, so a client
+ # comparing two of them learns whether to resync — never what
+ # changed. Sorting the id-and-etag lines keeps it independent of the
+ # order rows come back in while staying sensitive to membership and
+ # content.
+ #
+ # It costs a read of every card, because an etag is derived from a
+ # card's bytes rather than stored beside them. The changes table
+ # already holds a monotonic sequence that answers the same question
+ # for the price of one indexed read, and switching to it is part of
+ # the incremental-sync work: it would change the value a client
+ # holds, and it moves on every write rather than on every change,
+ # which is a different promise than the one this makes.
+ #: () -> String
+ def ctag
+ Digest::SHA256.hexdigest(contacts.map { "#{it.id} #{it.etag}" }.sort.join("\n"))
+ end
+
+ # `sole` rather than `first`: the id is the primary key, so a second
+ # row is a corruption and not a choice to make quietly. It raises on
+ # no row too, which here is the ordinary answer for an href nobody
+ # has — the 404 path — so that one is caught and turned back into
+ # nil.
+ #: (String id) -> Contact?
+ def contact(id)
+ contact_from(cards.where(id: text(id)).sole)
+ rescue Sequel::NoMatchingRow
+ nil
+ end
+
+ # Stores a card and everything that has to move with it: the
+ # change-log entry a client's sync token counts on, carrying the etag
+ # this card hashed to now, and the index rows read off the card. One
+ # transaction, because the log entry cannot be rebuilt from anything.
+ #: (String id, String vcard) -> Contact
+ def put(id, vcard)
+ contact = Contact.for(id: text(id), vcard: text(vcard))
+ @database.transaction do
+ cards
+ .insert_conflict(target: :id, update: {vcard: Sequel[:excluded][:vcard], updated_at: NOW})
+ .insert(id: contact.id, vcard: contact.vcard)
+ record(contact.id, "put", contact.etag)
+ reindex(contact)
+ end
+ contact
+ end
+
+ # Removes a card, leaving the tombstone that tells a syncing client
+ # it is gone. Returns whether there was anything to remove.
+ #: (String id) -> bool
+ def delete(id)
+ id = text(id)
+ @database.transaction do
+ deleted = cards.where(id:).delete.positive?
+ record(id, "delete", nil) if deleted
+ deleted
+ end
+ end
+
+ # The change log from a sequence number on, oldest first. Nothing
+ # reads it yet — incremental sync is the task that will — but the
+ # entries have to be written from the very first card, because a gap
+ # in them is a change some client is never told about.
+ #: (?after: Integer) -> Array[Change]
+ def changes(after: 0)
+ change_log.where(Sequel[:sequence] > after).order(:sequence).map { change_from(it) }
+ end
+
+ # Drops the index and derives it again from the stored cards alone.
+ # Always safe to run: nothing is authoritative here, so if the
+ # projection ever disagrees with the cards, this is the repair.
+ # Returns the ids of cards that did not parse, which are served in
+ # full but contribute nothing to the index.
+ #: () -> Array[String]
+ def rebuild_index
+ @database.transaction do
+ card_properties.delete
+ contacts.each.with_object([]) do |contact, unindexed|
+ unindexed << contact.id unless reindex(contact)
+ end
+ end
+ end
+
+ private
+
+ #: () -> Sequel::Dataset
+ def cards
+ @database[:cards]
+ end
+
+ #: () -> Sequel::Dataset
+ def change_log
+ @database[:changes]
+ end
+
+ #: () -> Sequel::Dataset
+ def card_properties
+ @database[:card_properties]
+ end
+
+ #: () -> Sequel::Dataset
+ def card_parameters
+ @database[:card_parameters]
+ end
+
+ # Anything that came off the wire is ASCII-8BIT, and Sequel writes a
+ # value into UTF-8 SQL, so a card with a byte above 7 bits raises
+ # Encoding::UndefinedConversionError on the way in. Ids are ASCII by
+ # Contact::ID_FORMAT and cards go out as `text/vcard; charset=utf-8`,
+ # so this relabels bytes rather than converting them.
+ #
+ # Relabelling is not validating, and deliberately so — the card is
+ # kept as it arrived. Bytes that are not UTF-8 at all do not slip
+ # through, though: SQLite refuses them on the insert, which is where
+ # a card that cannot honestly be served as UTF-8 should stop. There
+ # is a test pinning that.
+ #: (String value) -> String
+ def text(value)
+ value.encoding == Encoding::BINARY ? value.dup.force_encoding(Encoding::UTF_8) : value
+ end
+
+ #: (String card_id, String action, String? etag) -> void
+ def record(card_id, action, etag)
+ change_log.insert(card_id:, action:, etag:)
+ end
+
+ # Replaces a card's rows in the index. Rebuilt wholesale rather than
+ # diffed because it is derived data and replacing it is the cheaper
+ # correct thing. Returns whether the card parsed: one that does not
+ # is still served, so refusing to index it must not fail the write.
+ #: (Contact contact) -> bool
+ def reindex(contact)
+ # Before the parse, so that a card which has stopped parsing does
+ # not keep the rows from when it did.
+ card_properties.where(card_id: contact.id).delete
+ properties = properties_of(contact.vcard)
+ return false if properties.nil?
+
+ properties.each.with_index do |property, position|
+ card_properties.insert(
+ card_id: contact.id,
+ position:,
+ property_group: property.group,
+ name: property.name,
+ value: property.value,
+ )
+ property.parameters.each do |name, value|
+ card_parameters.insert(card_id: contact.id, position:, name:, value:)
+ end
+ end
+ true
+ end
+
+ # A card's properties, or nil when the bytes are not a vCard.
+ #: (String vcard) -> Array[VCard::Property]?
+ def properties_of(vcard)
+ VCard::Parser.parse(vcard)
+ rescue VCard::ParseError
+ nil
+ end
+
+ # Applies whatever migrations the database has not seen. On one it is
+ # already current with, this is a read of Sequel's schema_info table
+ # and nothing else, which is cheap enough to do on every open and
+ # leaves no way to serve from a database a deploy forgot to migrate.
+ #: () -> void
+ def migrate
+ Sequel::Migrator.run(@database, MIGRATIONS.to_s)
+ end
+
+ # The etag comes back out of the card rather than out of a column, so
+ # a row can never carry one that disagrees with the bytes beside it.
+ #: (Hash[Symbol, untyped] row) -> Contact
+ def contact_from(row)
+ Contact.for(id: row.fetch(:id).to_s, vcard: row.fetch(:vcard).to_s)
+ end
+
+ #: (Hash[Symbol, untyped] row) -> Change
+ def change_from(row)
+ Change.new(
+ sequence: row.fetch(:sequence).to_i,
+ card_id: row.fetch(:card_id).to_s,
+ action: row.fetch(:action).to_s,
+ etag: row.fetch(:etag),
+ created_at: row.fetch(:created_at).to_s,
+ )
+ end
+ end
+end
diff --git a/lib/pro_tacts/vcard.rb b/lib/pro_tacts/vcard.rb
index 785f67f..fe132b8 100644
--- a/lib/pro_tacts/vcard.rb
+++ b/lib/pro_tacts/vcard.rb
@@ -1,30 +1,14 @@
-require "kdl"
-
module ProTacts
- # Translates a parsed contact file — a bare KDL document — into a
- # vCard 3.0 (RFC 2426).
- #
- # The UID is passed separately because it lives in the filename rather
- # than the file (see docs/plans/2026-01-12-carddav-architecture.md).
+ # vCard 3.0 (RFC 2426): the line-level rules a card obeys, and the
+ # shape a parsed one takes. VCard::Parser reads a card into those;
+ # escape and fold write one back out.
module VCard
# Folded lines must not exceed 75 octets, excluding the line break
# (RFC 2426 section 2.6). The octet count, not character count, is
# what matters: a continuation must never split a multibyte
# character.
- LINE_LIMIT = 75
-
- NAME_COMPONENTS = %w[family given additional prefix suffix].freeze #: Array[String]
- ADDRESS_PARTS = %w[street city state zip country].freeze #: Array[String]
- CONTACT_FIELDS = %w[name phone email address].freeze #: Array[String]
-
- # Properties each node accepts; anything else is a typo silently
- # dropping data, so it raises.
- ALLOWED_PROPERTIES = {
- "phone" => %w[type],
- "email" => %w[type],
- "address" => %w[type],
- }.freeze #: Hash[String, Array[String]]
+ LINE_LIMIT = 75 #: Integer
TEXT_ESCAPES = {
"\\" => "\\\\",
@@ -33,130 +17,53 @@ module ProTacts
"\n" => "\\n",
}.freeze #: Hash[String, String]
- module_function
-
- #: (KDL::Document document, uid: String) -> String
- def render(document, uid:)
- nodes = document.nodes
- validate_children(nodes, CONTACT_FIELDS, "contact")
-
- names = nodes.select { it.name == "name" }
- name = names.first
- raise ArgumentError, "contact requires a name" if name.nil?
- raise ArgumentError, "contact takes a single name" if names.length > 1
-
- n, fn = name_fields(name)
-
- lines = [
- "BEGIN:VCARD",
- "VERSION:3.0",
- "N:#{components(n)}",
- "FN:#{escape(fn)}",
- *typed_property_lines(nodes, "phone", "TEL"),
- *typed_property_lines(nodes, "email", "EMAIL"),
- *address_lines(nodes),
- "UID:#{escape(uid)}",
- "END:VCARD",
- "", # trailing newline
- ]
-
- lines.map { fold(it) }.join("\r\n")
- end
-
- # A name is either a display string — `name "John Smith"`, where N
- # is derived (last token family, the rest given) and FN is the string
- # itself — or component children, where N is exactly those components
- # and FN is derived from them (prefix, given, additional, family,
- # suffix; empty parts skipped). Providing both is an error: they are
- # two spellings of the same truth, and silently preferring one would
- # hide the disagreement. Returns [N components, FN].
- #: (KDL::Node name) -> [Array[String], String]
- def name_fields(name)
- validate_children(name, NAME_COMPONENTS, "name")
- validate_properties(name)
-
- overrides = name.children.select { NAME_COMPONENTS.include?(it.name) }
-
- case [name.arguments, overrides]
- in [[], []]
- raise ArgumentError, "name requires a display string or component children"
- in [Array[_, *], Array[_, *]]
- raise ArgumentError, "name takes a display string or component children, not both"
- in [Array[_, *], []]
- display = name.arguments.first.value.to_s
- tokens = display.split
- family = tokens.last || ""
- given = tokens.length > 1 ? tokens.first(tokens.length - 1).join(" ") : ""
- [[family, given, "", "", ""], display]
- in [[], _]
- values = overrides.to_h { [it.name, string_argument(it)] }
- n = NAME_COMPONENTS.map { values.fetch(it, "") }
- fn = %w[prefix given additional family suffix]
- .map { values.fetch(it, "") }
- .reject(&:empty?)
- .join(" ")
- # FN is required (RFC 2426 section 4.1.1), so components that are
- # all empty have nothing to render it from.
- raise ArgumentError, "name components cannot all be empty" if fn.empty?
-
- [n, fn]
- end
- end
-
- #: (Array[KDL::Node] nodes, String kdl_name, String vcard_name) -> Array[String]
- def typed_property_lines(nodes, kdl_name, vcard_name)
- nodes.select { it.name == kdl_name }.map { |node|
- validate_properties(node)
- type = node.properties["type"]&.value
- prefix = type ? "#{vcard_name};TYPE=#{type}" : vcard_name
- "#{prefix}:#{escape(string_argument(node))}"
- }
- end
-
- # ADR's seven components in order: pobox, extended address, street,
- # locality, region, postal code, country. The first two have no KDL
- # counterpart and stay empty.
- #: (Array[KDL::Node] nodes) -> Array[String]
- def address_lines(nodes)
- nodes.select { it.name == "address" }.map { |node|
- validate_children(node, ADDRESS_PARTS, "address")
- validate_properties(node)
- parts = node.children
- .select { ADDRESS_PARTS.include?(it.name) }
- .to_h do |part|
- # A two-element literal is an Array until something says
- # otherwise, and to_h takes pairs.
- [part.name, string_argument(part)] #: [String, String]
- end
-
- components = ["", "", *ADDRESS_PARTS.map { parts.fetch(it, "") }]
- type = node.properties["type"]&.value
- prefix = type ? "ADR;TYPE=#{type}" : "ADR"
- "#{prefix}:#{components(components)}"
- }
- end
+ # A bare CR is not a line break in the grammar, but it ends a value
+ # and has to end a line here too: without it the parser's scanner
+ # would sit on one forever, because no token can consume it.
+ LINE_BREAK = /\r\n|[\r\n]/ #: Regexp
+
+ # Reads like an alternation-precedence bug and is not: Ruby wraps an
+ # interpolated Regexp in a non-capturing group, so this is
+ # `(?:\r\n|[\r\n])[ \t]` rather than `\r\n` or `[\r\n][ \t]`.
+ FOLD = /#{LINE_BREAK}[ \t]/ #: Regexp
+
+ # A card whose bytes are not a vCard at all. Callers are expected to
+ # carry on serving the card: failing to parse costs an index entry,
+ # not the contact.
+ class ParseError < StandardError; end
+
+ # One parsed content line.
+ #
+ # `parameters` is a list of name-and-value pairs rather than a hash
+ # because a parameter can repeat, and RFC 2426 section 3.3.1 makes
+ # `TYPE=work;TYPE=voice` and `TYPE=work,voice` two spellings of the
+ # same thing — both arrive here as two pairs.
+ #
+ # `value` is the value's text, unfolded and otherwise exactly as it
+ # was stored, still escaped. Unescaping it and splitting it on the
+ # component separator are per-property semantics, and this layer has
+ # none.
+ #
+ # The signature lives in sig/pro_tacts/vcard.rbs: a Data class has no
+ # constant super class for the inline syntax to read.
+ # @rbs skip
+ Property = Data.define(:group, :name, :parameters, :value)
# Text values escape backslash, the component separator, and the
# sub-component separator (RFC 2426 section 2.4.2); CRLF and CR are
# normalized to the `\n` escape because a raw line break would end
# the property line.
#: (String text) -> String
- def escape(text)
+ def self.escape(text)
text.gsub(/\r\n|\r/, "\n").gsub(/[\\;,\n]/) { TEXT_ESCAPES.fetch(it) }
end
- # Escapes each component, then joins with the component separator.
- #: (Array[String] values) -> String
- def components(values)
- values.map { escape(it) }.join(";")
- end
-
# Folds a logical line into physical lines of at most LINE_LIMIT
# octets, each continuation starting with a single space (RFC 2426
# section 2.6). The walk is character-wise so a multibyte character
# is never split mid-sequence.
#: (String line) -> String
- def fold(line)
+ def self.fold(line)
return line if line.bytesize <= LINE_LIMIT
folded = +""
@@ -172,30 +79,14 @@ module ProTacts
folded
end
- #: (KDL::Node node) -> String
- def string_argument(node)
- argument = node.arguments.first
- raise ArgumentError, "#{node.name} requires a string argument" if argument.nil?
-
- argument.value.to_s
- end
-
- # Takes a node list or a single node, whose children it walks.
- #: (Array[KDL::Node] | KDL::Node nodes, Array[String] known, String context) -> void
- def validate_children(nodes, known, context)
- unknown = nodes.find { !known.include?(it.name) }
- return if unknown.nil?
-
- raise ArgumentError, "unknown key in #{context}: #{unknown.name}"
- end
-
- #: (KDL::Node node) -> void
- def validate_properties(node)
- allowed = ALLOWED_PROPERTIES.fetch(node.name, [])
- unknown = node.properties.keys - allowed
- return if unknown.empty?
-
- raise ArgumentError, "unknown property on #{node.name}: #{unknown.first}"
+ # Fold's inverse, and the first thing a parse does: RFC 2426 section
+ # 2.6 has a content line unfolded before it is read. It runs over the
+ # whole card rather than token by token because a fold can land
+ # anywhere in a line — Contacts folds at 75 octets without regard for
+ # what it splits — so there is no boundary to do it at.
+ #: (String card) -> String
+ def self.unfold(card)
+ card.gsub(FOLD, "")
end
end
end
diff --git a/lib/pro_tacts/vcard/parser.rb b/lib/pro_tacts/vcard/parser.rb
new file mode 100644
index 0000000..96f8335
--- /dev/null
+++ b/lib/pro_tacts/vcard/parser.rb
@@ -0,0 +1,125 @@
+
+require "strscan"
+
+require "pro_tacts/vcard"
+
+module ProTacts
+ module VCard
+ # Reads a stored card into VCard::Property values.
+ #
+ # Deliberately shallow: a property is a name, its parameters, and the
+ # text of its value, and nothing here knows what any of them mean.
+ # That is not a shortcut. The stored card is authoritative and this
+ # is only a projection of it, so a property this code has never heard
+ # of has to travel through untouched, which is what RFC 6352 section
+ # 6.3.2.2 requires of a CardDAV server. See
+ # docs/plans/2026-08-24-vcard-storage-and-groups.md.
+ #
+ # One instance reads one card. The scanner is the whole of its state,
+ # which is why it is a class and not a handful of functions passing
+ # a position between them.
+ class Parser
+ # @rbs @scanner: StringScanner
+
+ # The content line, RFC 2426 section 4:
+ #
+ # contentline = [group "."] name *(";" param) ":" value CRLF
+ # group = 1*(ALPHA / DIGIT / "-")
+ # name = iana-token / x-name
+ # param = param-name "=" param-value *("," param-value)
+ #
+ # A group, a property name, and a parameter name are all the same
+ # token. SAFE-CHAR — an unquoted parameter value — is anything but
+ # a control character, DQUOTE, ";", ":", or ","; QSAFE-CHAR, inside
+ # quotes, allows everything but the controls and DQUOTE;
+ # VALUE-CHAR is everything that is not a line break.
+ TOKEN = /[A-Za-z0-9-]+/ #: Regexp
+ GROUP = /#{TOKEN}(?=\.)/ #: Regexp
+ PTEXT = /[^\x00-\x08\x0A-\x1F\x7F";:,]*/ #: Regexp
+ QUOTED_STRING = /"([^\x00-\x08\x0A-\x1F\x7F"]*)"/ #: Regexp
+ VALUE = /[^\r\n]*/ #: Regexp
+
+ #: (String card) -> Array[Property]
+ def self.parse(card)
+ new(card).parse
+ end
+
+ #: (String card) -> void
+ def initialize(card)
+ @scanner = StringScanner.new(VCard.unfold(card))
+ end
+
+ # Every content line in the card, in order. Blank lines are
+ # skipped, and BEGIN, VERSION, and END come back like any other
+ # property: deciding that a card is well-formed is not this class's
+ # job.
+ #: () -> Array[Property]
+ def parse
+ properties = [] #: Array[Property]
+ until @scanner.eos?
+ next if @scanner.skip(LINE_BREAK)
+
+ properties << content_line
+ end
+ properties
+ end
+
+ private
+
+ #: () -> Property
+ def content_line
+ group = @scanner.scan(GROUP)
+ @scanner.skip(/\./) if group
+
+ name = @scanner.scan(TOKEN)
+ raise ParseError, "expected a property name #{here}" if name.nil?
+
+ parameters = [] #: Array[[String, String]]
+ parameters.concat(parameter) while @scanner.skip(/;/)
+
+ raise ParseError, "expected ':' after #{name} #{here}" unless @scanner.skip(/:/)
+
+ value = @scanner.scan(VALUE).to_s
+ @scanner.skip(LINE_BREAK)
+
+ Property.new(group:, name:, parameters:, value:)
+ end
+
+ # One `;`-delimited parameter, as a pair per value.
+ #: () -> Array[[String, String]]
+ def parameter
+ name = @scanner.scan(TOKEN)
+ raise ParseError, "expected a parameter name #{here}" if name.nil?
+
+ # vCard 2.1 let a parameter be written as a bare value with its
+ # name left implicit. The 3.0 grammar does not, and guessing
+ # which parameter was meant is the kind of invention that loses
+ # data.
+ raise ParseError, "parameter #{name} has no value #{here}" unless @scanner.skip(/=/)
+
+ values = [parameter_value]
+ values << parameter_value while @scanner.skip(/,/)
+ values.map {
+ # A two-element literal is an Array until something says
+ # otherwise, and an inline annotation needs its own line.
+ [name, it] #: [String, String]
+ }
+ end
+
+ # The quotes around a quoted-string are syntax, so what comes back
+ # is what they contained.
+ #: () -> String
+ def parameter_value
+ return @scanner[1].to_s if @scanner.scan(QUOTED_STRING)
+
+ @scanner.scan(PTEXT).to_s
+ end
+
+ # Where the scan gave up, for the message that says so.
+ #: () -> String
+ def here
+ "at offset #{@scanner.pos}"
+ end
+ end
+ end
+end
diff --git a/lib/pro_tacts/web.rb b/lib/pro_tacts/web.rb
index 06a771b..6d9633a 100644
--- a/lib/pro_tacts/web.rb
+++ b/lib/pro_tacts/web.rb
@@ -7,14 +7,33 @@ require "nokogiri"
require "roda"
require "pro_tacts/debug_logger"
-require "pro_tacts/addressbook"
+require "pro_tacts/contact"
+require "pro_tacts/store"
require "pro_tacts/tailscale_auth"
require "pro_tacts/unhandled_requests"
require "roda/plugins/dav_verbs"
module ProTacts
class Web < Roda
- # @rbs @addressbook: Addressbook?
+ # @rbs @contacts: Array[Contact]?
+ # @rbs @ctag: String?
+
+ # The store this app serves from. config.ru builds it and hands it in;
+ # nothing here reaches for a global to find one, which is what lets a
+ # test point the app at a throwaway database. Kept in Roda's own opts
+ # rather than a class variable, so it is frozen with the app and a
+ # write after that raises instead of quietly taking effect.
+ #: (Store store) -> void
+ def self.store=(store)
+ opts[:store] = store
+ end
+
+ #: () -> Store
+ def self.store
+ opts.fetch(:store) do
+ raise "no store: hand one to ProTacts::Web.store= before serving"
+ end
+ end
# RewindableInput allows us to read the request body for Sentry logging
# and then rewind it so the application can still access it.
@@ -194,8 +213,8 @@ module ProTacts
<d:report><d:sync-collection/></d:report>
</d:supported-report>
</d:supported-report-set>
- <cs:getctag>#{addressbook.ctag}</cs:getctag>
- <d:sync-token>#{addressbook.sync_token}</d:sync-token>
+ <cs:getctag>#{ctag}</cs:getctag>
+ <d:sync-token>#{sync_token}</d:sync-token>
<d:current-user-privilege-set>
<d:privilege><d:read/></d:privilege>
<d:privilege><d:write/></d:privilege>
@@ -210,7 +229,7 @@ module ProTacts
end
# Depth: 0 returns only collection, Depth: 1 includes members
- members = depth == "0" ? "" : addressbook.contacts.map { etag_response(it) }.join
+ members = depth == "0" ? "" : contacts.map { etag_response(it) }.join
<<~XML
<?xml version="1.0" encoding="UTF-8"?>
@@ -252,7 +271,7 @@ module ProTacts
# contact with no token. macOS resyncs the whole collection
# anyway, so it works; a client that trusts the token would
# break. Fixing it is the incremental-sync work in the backlog.
- multistatus(addressbook.contacts.map { etag_response(it) })
+ multistatus(contacts.map { etag_response(it) })
when "addressbook-multiget"
# CARDDAV:addressbook-multiget (RFC 6352 section 8.7); the
# address-data the client asks for is section 10.4.
@@ -260,7 +279,7 @@ module ProTacts
multistatus(doc.xpath("//href").map { it.text }.map { |requested|
id = requested[%r{\A/dav/addressbook/([^/]+)\.vcf\z}, 1]
- contact = id && addressbook.contacts.find { it.id == id }
+ contact = id && contacts.find { it.id == id }
if contact
wants_cards ? card_response(contact) : etag_response(contact)
@@ -293,8 +312,10 @@ module ProTacts
end
end
+ # Read on its own rather than through the collection: serving
+ # one href has no reason to load every other contact first.
r.get String do |filename|
- contact = addressbook.contacts.find { it.id == filename.delete_suffix(".vcf") }
+ contact = store.contact(filename.delete_suffix(".vcf"))
# No match falls through to the empty-body 404 that the
# not_found handler fills in.
@@ -312,13 +333,30 @@ module ProTacts
private
- # Loaded once per request — Roda builds a fresh app instance for each
- # one — with the directory coming from config. Real etags and ctags
- # make an mtime-keyed cache possible, but re-parsing a family address
- # book per request is cheap and can never serve a stale change tag.
- #: () -> Addressbook
- def addressbook
- @addressbook ||= Addressbook.load(ProTacts.config.contacts_dir)
+ # Read once per request — Roda builds a fresh app instance for each
+ # one — from the store the whole process shares. Reading a family
+ # address book per request is cheap and can never serve a stale one.
+ #: () -> Array[Contact]
+ def contacts
+ @contacts ||= store.contacts
+ end
+
+ #: () -> String
+ def ctag
+ @ctag ||= store.ctag
+ end
+
+ # Sync tokens are opaque to the client (RFC 6578 section 3); the URI
+ # form is conventional. Built on the ctag so a client polling either
+ # one sees changes at the same points.
+ #: () -> String
+ def sync_token
+ "http://pro-tacts/sync/#{ctag}"
+ end
+
+ #: () -> Store
+ def store
+ self.class.store
end
#: (Array[String] responses) -> String
diff --git a/lib/sequel/extensions/sole.rb b/lib/sequel/extensions/sole.rb
new file mode 100644
index 0000000..d810cbb
--- /dev/null
+++ b/lib/sequel/extensions/sole.rb
@@ -0,0 +1,47 @@
+
+require "sequel"
+
+module Sequel
+ # The row a filter identifies, or a raise.
+ #
+ # A filter meant to identify one row — a lookup by primary key, later
+ # the group that contributed a property to a card — reads naturally as
+ # `first`, and `first` answers with one of several without a word when
+ # the filter turns out not to identify anything. That is the failure
+ # worth catching: the query claimed one row, so anything else is a
+ # broken assumption rather than a result to pick from.
+ #
+ # Both ways of being wrong raise, so the assertion is in one method
+ # rather than split across a pair. A caller for whom no row is
+ # ordinary says so by rescuing Sequel::NoMatchingRow, which is the
+ # error Sequel's own `first!` raises for it.
+ #
+ # Load it with `DB.extension(:sole)`, which reaches every dataset the
+ # database makes.
+ # @rbs module-self: Sequel::_Queryable -- it is mixed into datasets
+ module Sole
+ # More rows than the query said there could be. A Sequel::Error, so
+ # it is caught by anything already catching database failures.
+ class TooManyRows < Sequel::Error; end
+
+ #: () -> Hash[Symbol, untyped]
+ def sole
+ # Two is enough to know, and a LIMIT keeps a filter that turned out
+ # to match the whole table from dragging it back to prove the
+ # point. It replaces any limit already set, which is why this is
+ # for identifying a row rather than for sampling one.
+ rows = limit(2).all
+
+ # The table rather than the SQL: a filter can carry card content,
+ # and an exception message is one of the things that reaches
+ # Sentry. See ProTacts::SentryScrubber.
+ raise TooManyRows, "more than one row in #{first_source}" if rows.length > 1
+ raise Sequel::NoMatchingRow.new(self) if rows.empty?
+
+ rows.fetch(0)
+ end
+ end
+
+ Dataset.register_extension(:sole, Sole)
+ Database.register_extension(:sole) { |db| db.extend_datasets(Sole) }
+end
diff --git a/sig/gems/kdl.rbs b/sig/gems/kdl.rbs
deleted file mode 100644
index 0a43eca..0000000
--- a/sig/gems/kdl.rbs
+++ /dev/null
@@ -1,36 +0,0 @@
-# The kdl gem ships no signatures, so this describes the parts pro-tacts
-# calls: a document is nodes, a node is a name with arguments, properties
-# and children, and every leaf value is wrapped in a KDL::Value.
-#
-# Written against kdl 2.2.0 (lib/kdl/document.rb, node.rb, value.rb).
-module KDL
- # KDL 2.0 scalars: strings, numbers, booleans, and null.
- type value = String | Integer | Float | bool | nil
-
- def self.parse: (String input, **untyped options) -> Document
-
- class Document
- include Enumerable[Node]
-
- attr_accessor nodes: Array[Node]
-
- def each: () { (Node) -> void } -> void
- end
-
- class Node
- include Enumerable[Node]
-
- attr_accessor name: String
- attr_accessor arguments: Array[Value]
- attr_accessor properties: Hash[String, Value]
- attr_accessor children: Array[Node]
-
- # Yields the children, which is what makes a node interchangeable
- # with a node list wherever pro-tacts walks one.
- def each: () { (Node) -> void } -> void
- end
-
- class Value
- attr_reader value: KDL::value
- end
-end
diff --git a/sig/gems/roda.rbs b/sig/gems/roda.rbs
index 4748381..9203d73 100644
--- a/sig/gems/roda.rbs
+++ b/sig/gems/roda.rbs
@@ -10,6 +10,10 @@ class Roda
def self.use: (untyped middleware, **untyped options) -> void
+ # App-level configuration, which is where web.rb keeps the store handed
+ # to it. Frozen along with the app, so a write after that raises.
+ def self.opts: () -> Hash[Symbol, untyped]
+
# The block is the router, run against a fresh instance per request.
def self.route: () { (RodaRequest) [self: instance] -> untyped } -> void
diff --git a/sig/gems/sequel.rbs b/sig/gems/sequel.rbs
new file mode 100644
index 0000000..f733a44
--- /dev/null
+++ b/sig/gems/sequel.rbs
@@ -0,0 +1,146 @@
+# Sequel ships no signatures, so this describes the parts pro-tacts
+# calls: one SQLite database, four datasets over it, and the schema DSL
+# that creates them. A row is a hash keyed by column, whose values carry
+# whatever type the column held, which is why the store converts them at
+# its edge rather than trusting them.
+#
+# The create_table generator is declared because the block runs against
+# it rather than against the calling object, and `[self: ...]` is how
+# RBS says so; without it every column line in the schema reads as a
+# call on Store.
+#
+# Written against sequel 5.107.0 (lib/sequel/database/, dataset/,
+# adapters/sqlite.rb).
+module Sequel
+ type row = Hash[Symbol, untyped]
+
+ def self.sqlite: (String path, **untyped options) -> Database
+
+ # Loads an extension into Sequel itself; `:migration` is the one this
+ # app asks for, and it is what defines Migrator.
+ def self.extension: (*Symbol names) -> void
+
+ # A qualified or unqualified identifier: `Sequel[:excluded][:vcard]`,
+ # and the comparisons built from one.
+ def self.[]: (Symbol name) -> untyped
+
+ # SQL written out by hand, for the one default expression the schema
+ # DSL has no spelling for.
+ def self.lit: (String sql) -> untyped
+
+ class Database
+ def []: (Symbol table) -> Dataset
+
+ # Loads an extension into this database and the datasets it makes.
+ def extension: (*Symbol names) -> Database
+
+ def extend_datasets: (Module extension) -> void
+
+ def self.register_extension: (Symbol name, ?Module extension) ?{ (Database) -> void } -> void
+
+ # Runs a statement for its effect. Used only for the pragmas Sequel
+ # has no option for.
+ def run: (String sql) -> untyped
+
+ # Wraps a statement whose rows are wanted, as a dataset.
+ def fetch: (String sql) -> Dataset
+
+ # Joins a transaction already open rather than nesting, so a write
+ # inside a write is still one atomic unit.
+ def transaction: [T] () { () -> T } -> T
+
+ def create_table: (Symbol name, **untyped options) { () [self: Schema::CreateTableGenerator] -> void } -> void
+
+ def disconnect: () -> void
+ end
+
+ # The dataset surface lib/sequel/extensions/sole.rb needs of whatever
+ # it is mixed into. A structural interface rather than Dataset itself,
+ # because Dataset includes Sole and naming it here would be circular.
+ interface _Queryable
+ def limit: (Integer count) -> Dataset
+
+ def first_source: () -> Symbol
+ end
+
+ class Dataset
+ include Enumerable[Sequel::row]
+
+ # lib/sequel/extensions/sole.rb. RBS has no way to say "only once
+ # DB.extension(:sole) has run", and Store runs it on every open.
+ include Sole
+
+ def self.register_extension: (Symbol name, Module extension) -> void
+
+ def where: (*untyped conditions) -> Dataset
+ def order: (*untyped columns) -> Dataset
+ def limit: (Integer count) -> Dataset
+ def join: (Symbol table, Array[Symbol] using) -> Dataset
+
+ # The table a dataset selects from; raises when it has no source.
+ def first_source: () -> Symbol
+
+ # An upsert: `target` is the conflicting column, `update` what to set
+ # on the row already there.
+ def insert_conflict: (**untyped options) -> Dataset
+
+ def first: () -> Sequel::row?
+ def all: () -> Array[Sequel::row]
+ def count: () -> Integer
+ def columns: () -> Array[Symbol]
+ def single_value: () -> untyped
+ def select_map: (untyped columns) -> Array[untyped]
+
+ def insert: (Sequel::row values) -> Integer
+ def update: (Sequel::row values) -> Integer
+
+ # Rows removed.
+ def delete: () -> Integer
+
+ def each: () { (Sequel::row) -> void } -> void
+ end
+
+ module Schema
+ # The schema DSL. Each column method takes the column name and the
+ # options for it; `text: true` is what keeps a String a SQLite TEXT
+ # rather than a varchar, which a STRICT table refuses.
+ class CreateTableGenerator
+ def String: (Symbol name, **untyped options) -> void
+ def Integer: (Symbol name, **untyped options) -> void
+
+ # A single argument is the table's own key (an AUTOINCREMENT
+ # integer on SQLite); an array is a composite primary key over
+ # columns already declared.
+ def primary_key: (Symbol | Array[Symbol] name, **untyped options) -> void
+
+ def foreign_key: (Symbol | Array[Symbol] name, ?Symbol table, **untyped options) -> void
+
+ def constraint: (Symbol name, **untyped condition) -> void
+
+ def index: (Symbol | Array[Symbol] columns, **untyped options) -> void
+ end
+ end
+
+ # Applies the migrations in a directory that the database has not run
+ # yet, and does nothing to one that is already current.
+ class Migrator
+ def self.run: (Database database, String directory, **untyped options) -> untyped
+ end
+
+ class Error < StandardError
+ end
+
+ # What `first!` raises on an empty dataset, and `sole` with it.
+ class NoMatchingRow < Error
+ def initialize: (?untyped message_or_dataset) -> void
+ end
+
+ class DatabaseError < Error
+ end
+
+ class ConstraintViolation < DatabaseError
+ end
+
+ class CheckConstraintViolation < ConstraintViolation
+ end
+end
diff --git a/sig/pro_tacts/addressbook.rbs b/sig/pro_tacts/addressbook.rbs
deleted file mode 100644
index f58f0d5..0000000
--- a/sig/pro_tacts/addressbook.rbs
+++ /dev/null
@@ -1,15 +0,0 @@
-# A Data class, like Contact: see sig/pro_tacts/contact.rbs for why the
-# signature is here rather than in the Ruby.
-module ProTacts
- class Addressbook
- attr_reader contacts: Array[Contact]
-
- def self.new: (contacts: Array[Contact]) -> instance
-
- def self.load: (Pathname | String directory) -> Addressbook
-
- def ctag: () -> String
-
- def sync_token: () -> String
- end
-end
diff --git a/sig/pro_tacts/contact.rbs b/sig/pro_tacts/contact.rbs
index b5ee3a4..6f80dd3 100644
--- a/sig/pro_tacts/contact.rbs
+++ b/sig/pro_tacts/contact.rbs
@@ -12,8 +12,8 @@ module ProTacts
def self.new: (id: String, vcard: String, etag: String) -> instance
- def self.all: (Pathname | String directory) -> Array[Contact]
+ def self.for: (id: String, vcard: String) -> Contact
- def self.parse: (Pathname | String path) -> Contact
+ def self.etag_for: (String vcard) -> String
end
end
diff --git a/sig/pro_tacts/store.rbs b/sig/pro_tacts/store.rbs
new file mode 100644
index 0000000..283b202
--- /dev/null
+++ b/sig/pro_tacts/store.rbs
@@ -0,0 +1,16 @@
+# Change is a Data class, whose members the inline parser cannot read
+# (see sig/pro_tacts/contact.rbs). Store itself is an ordinary class and
+# its types are inline in store.rb.
+module ProTacts
+ class Store
+ class Change
+ attr_reader sequence: Integer
+ attr_reader card_id: String
+ attr_reader action: String
+ attr_reader etag: String?
+ attr_reader created_at: String
+
+ def self.new: (sequence: Integer, card_id: String, action: String, etag: String?, created_at: String) -> instance
+ end
+ end
+end
diff --git a/sig/pro_tacts/vcard.rbs b/sig/pro_tacts/vcard.rbs
index 9ffbeb0..b1482a9 100644
--- a/sig/pro_tacts/vcard.rbs
+++ b/sig/pro_tacts/vcard.rbs
@@ -1,8 +1,16 @@
-# `module_function` copies every method onto the module itself, and
-# inline RBS has no syntax for that. Extending the module is how RBS
-# says the same thing, so the method types can stay inline in vcard.rb.
+# Property is a Data class, whose members the inline parser cannot read
+# (see sig/pro_tacts/contact.rbs). Everything else about vcard.rb is
+# inline: its methods are singleton methods rather than
+# `module_function`, which is a shape RBS reads without help.
module ProTacts
module VCard
- extend VCard
+ class Property
+ attr_reader group: String?
+ attr_reader name: String
+ attr_reader parameters: Array[[String, String]]
+ attr_reader value: String
+
+ def self.new: (group: String?, name: String, parameters: Array[[String, String]], value: String) -> instance
+ end
end
end
diff --git a/tasks/index.rake b/tasks/index.rake
new file mode 100644
index 0000000..1719bf1
--- /dev/null
+++ b/tasks/index.rake
@@ -0,0 +1,19 @@
+# The index is a projection of the stored cards and nothing is
+# authoritative in it, so rebuilding is always safe: run this after a
+# change to how cards are parsed, or if the index is ever suspected of
+# disagreeing with the cards. See ProTacts::Store.
+
+namespace :index do
+ desc "Drop the derived index and rebuild it from the stored cards"
+ task :rebuild do
+ require "pro_tacts"
+ require "pro_tacts/store"
+
+ database = ProTacts.config.database_path
+ ProTacts::Store.connect(database) do |store|
+ unindexed = store.rebuild_index
+ puts "rebuilt the index in #{database}"
+ warn "not indexed, still served: #{unindexed.join(', ')}" unless unindexed.empty?
+ end
+ end
+end
diff --git a/test/fixture_data.rb b/test/fixture_data.rb
new file mode 100644
index 0000000..8fda05d
--- /dev/null
+++ b/test/fixture_data.rb
@@ -0,0 +1,37 @@
+
+require "fileutils"
+require "pathname"
+
+require "pro_tacts/store"
+
+# Builds the data directory the tests and `rake fixtures` run against: a
+# database seeded from the cards in test/fixtures/cards, rebuilt from
+# scratch every time so a replay never depends on what a previous run
+# left behind.
+#
+# The cards are files rather than a checked-in database because they are
+# evidence, the same standing the request files in macos-exchange have:
+# a reviewer can read a .vcf and a diff means something. The database is
+# derived from them, and lives under tmp/ with everything else that is.
+module FixtureData
+ CARDS = Pathname.new(__dir__) / "fixtures" / "cards"
+
+ # Every seed card, keyed by the id its filename gives it.
+ def self.cards
+ CARDS.glob("*.vcf").sort.to_h { [it.basename(".vcf").to_s, it.read] }
+ end
+
+ # Builds the database and returns a store still open on it, for the
+ # caller to hand to the app the way config.ru does.
+ def self.install(directory)
+ directory = Pathname.new(directory)
+ FileUtils.rm_rf(directory)
+ FileUtils.mkdir_p(directory)
+
+ store = ProTacts::Store.at(directory / "contacts.db")
+ cards.each do |id, card|
+ store.put(id, card)
+ end
+ store
+ end
+end
diff --git a/test/fixtures/cards/AB12C345-6789-0DEF-1234-567890ABCDEF.vcf b/test/fixtures/cards/AB12C345-6789-0DEF-1234-567890ABCDEF.vcf
new file mode 100644
index 0000000..6a927e7
--- /dev/null
+++ b/test/fixtures/cards/AB12C345-6789-0DEF-1234-567890ABCDEF.vcf
@@ -0,0 +1,6 @@
+BEGIN:VCARD
+VERSION:3.0
+N:Contact;Test;;;
+FN:Test Contact
+UID:AB12C345-6789-0DEF-1234-567890ABCDEF
+END:VCARD
diff --git a/test/fixtures/contacts/AB12C345-6789-0DEF-1234-567890ABCDEF.kdl b/test/fixtures/contacts/AB12C345-6789-0DEF-1234-567890ABCDEF.kdl
deleted file mode 100644
index c040be4..0000000
--- a/test/fixtures/contacts/AB12C345-6789-0DEF-1234-567890ABCDEF.kdl
+++ /dev/null
@@ -1 +0,0 @@
-name "Test Contact"
diff --git a/test/pro_tacts/test_addressbook.rb b/test/pro_tacts/test_addressbook.rb
deleted file mode 100644
index 12a9c80..0000000
--- a/test/pro_tacts/test_addressbook.rb
+++ /dev/null
@@ -1,58 +0,0 @@
-require_relative "../test_helper"
-
-require "pathname"
-require "tmpdir"
-
-require "pro_tacts/addressbook"
-
-class AddressbookTest < Minitest::Test
- def with_contacts(files)
- Dir.mktmpdir do |dir|
- contacts_dir = Pathname.new(dir) / "contacts"
- Dir.mkdir(contacts_dir)
- files.each { |name, content| File.write(contacts_dir / name, content) }
- yield contacts_dir
- end
- end
-
- def ctag(contacts_dir)
- ProTacts::Addressbook.load(contacts_dir).ctag
- end
-
- def test_the_ctag_is_stable_across_loads
- with_contacts({"aiden.kdl" => "name \"Aiden\""}) do |contacts_dir|
- assert_equal ctag(contacts_dir), ctag(contacts_dir)
- end
- end
-
- def test_the_ctag_moves_with_a_cards_content
- with_contacts({"aiden.kdl" => "name \"Aiden\""}) do |contacts_dir|
- before = ctag(contacts_dir)
- File.write(contacts_dir / "aiden.kdl", "name \"Aiden Smith\"")
-
- refute_equal before, ctag(contacts_dir)
- end
- end
-
- def test_the_ctag_moves_with_membership_both_ways
- with_contacts({"aiden.kdl" => "name \"Aiden\""}) do |contacts_dir|
- alone = ctag(contacts_dir)
-
- File.write(contacts_dir / "znorth.kdl", "name \"Zed\"")
- refute_equal alone, ctag(contacts_dir)
-
- File.delete(contacts_dir / "znorth.kdl")
- assert_equal alone, ctag(contacts_dir)
- end
- end
-
- def test_the_sync_token_moves_with_the_ctag
- with_contacts({"aiden.kdl" => "name \"Aiden\""}) do |contacts_dir|
- token = ProTacts::Addressbook.load(contacts_dir).sync_token
-
- File.write(contacts_dir / "aiden.kdl", "name \"Aiden Smith\"")
-
- refute_equal token, ProTacts::Addressbook.load(contacts_dir).sync_token
- end
- end
-end
diff --git a/test/pro_tacts/test_config.rb b/test/pro_tacts/test_config.rb
index a28daaa..a6bf895 100644
--- a/test/pro_tacts/test_config.rb
+++ b/test/pro_tacts/test_config.rb
@@ -21,9 +21,16 @@ class ConfigTest < Minitest::Test
assert_equal Pathname.new("/tmp/state"), ProTacts::Config.new("PRO_TACTS_DATA_DIR" => "/tmp/state").data_dir
end
- def test_contacts_dir_lives_under_the_data_dir
- assert_equal Pathname.new("data/contacts"), ProTacts::Config.new({}).contacts_dir
- assert_equal Pathname.new("/tmp/state/contacts"), ProTacts::Config.new("PRO_TACTS_DATA_DIR" => "/tmp/state").contacts_dir
+ def test_the_database_lives_under_the_data_dir
+ assert_equal Pathname.new("data/contacts.db"), ProTacts::Config.new({}).database_path
+ assert_equal Pathname.new("/tmp/state/contacts.db"), ProTacts::Config.new("PRO_TACTS_DATA_DIR" => "/tmp/state").database_path
+ end
+
+ def test_the_database_path_is_overridable_on_its_own
+ config = ProTacts::Config.new("PRO_TACTS_DATA_DIR" => "/tmp/state", "PRO_TACTS_DATABASE" => "/srv/cards.db")
+
+ assert_equal Pathname.new("/srv/cards.db"), config.database_path
+ assert_equal Pathname.new("/tmp/state"), config.data_dir
end
def test_debug_defaults_off
diff --git a/test/pro_tacts/test_contact.rb b/test/pro_tacts/test_contact.rb
index 50486a9..b4ae63c 100644
--- a/test/pro_tacts/test_contact.rb
+++ b/test/pro_tacts/test_contact.rb
@@ -1,122 +1,45 @@
require_relative "../test_helper"
require "digest"
-require "pathname"
-require "tmpdir"
require "pro_tacts/contact"
class ContactTest < Minitest::Test
- def with_contacts(files)
- Dir.mktmpdir do |dir|
- directory = Pathname.new(dir)
- files.each { |name, content| File.write(directory / name, content) }
- yield directory
- end
- end
-
- def test_all_parses_every_contact
- with_contacts({
- "znorth.kdl" => "name \"Zed\"",
- "aiden.kdl" => "name \"Aiden\"",
- }) do |directory|
- ids = ProTacts::Contact.all(directory).map { it.id }
-
- assert_includes ids, "aiden"
- assert_includes ids, "znorth"
- end
- end
-
- def test_the_uid_comes_from_the_filename
- with_contacts({"kqmtnwpxlrvszoyp.kdl" => "name \"Aiden\""}) do |directory|
- assert_includes ProTacts::Contact.all(directory).first.vcard, "UID:kqmtnwpxlrvszoyp"
- end
- end
+ CARD = "BEGIN:VCARD\r\nVERSION:3.0\r\nFN:Aiden\r\nEND:VCARD\r\n"
- def test_the_etag_hashes_the_rendered_card
- with_contacts({"aiden.kdl" => "name \"Aiden\""}) do |directory|
- contact = ProTacts::Contact.all(directory).first
+ def test_the_etag_hashes_the_card
+ contact = ProTacts::Contact.for(id: "aiden", vcard: CARD)
- assert_equal %("#{Digest::SHA256.hexdigest(contact.vcard)}"), contact.etag
- end
+ assert_equal %("#{Digest::SHA256.hexdigest(CARD)}"), contact.etag
end
- def test_the_etag_is_stable_across_parses_and_moves_with_content
- with_contacts({"aiden.kdl" => "name \"Aiden\""}) do |directory|
- etag = ProTacts::Contact.all(directory).first.etag
- assert_equal etag, ProTacts::Contact.all(directory).first.etag
+ def test_the_etag_moves_with_the_card
+ etag = ProTacts::Contact.for(id: "aiden", vcard: CARD).etag
- File.write(directory / "aiden.kdl", "name \"Aiden Smith\"")
- refute_equal etag, ProTacts::Contact.all(directory).first.etag
- end
+ assert_equal etag, ProTacts::Contact.for(id: "aiden", vcard: CARD).etag
+ refute_equal etag, ProTacts::Contact.for(id: "aiden", vcard: CARD.sub("Aiden", "Aiden Smith")).etag
end
- def test_an_empty_directory_lists_no_contacts
- with_contacts({}) do |directory|
- assert_empty ProTacts::Contact.all(directory)
- end
+ # The same card under two ids is two contacts with the same etag: the
+ # etag is about the bytes a client downloads, and the id is not in them.
+ def test_the_etag_ignores_the_id
+ assert_equal(
+ ProTacts::Contact.for(id: "aiden", vcard: CARD).etag,
+ ProTacts::Contact.for(id: "znorth", vcard: CARD).etag,
+ )
end
- def test_dotfiles_are_ignored
- with_contacts({
- ".DS_Store" => "junk",
- "aiden.kdl" => "name \"Aiden\"",
- }) do |directory|
- assert_equal %w[aiden], ProTacts::Contact.all(directory).map { it.id }
- end
- end
+ def test_ids_outside_the_href_charset_raise
+ ["John Smith", "../etc/passwd", "a/b", ""].each do |id|
+ error = assert_raises(ArgumentError) { ProTacts::Contact.for(id:, vcard: CARD) }
- def test_a_missing_directory_raises
- error = assert_raises(Errno::ENOENT) do
- ProTacts::Contact.all(Pathname.new(Dir.mktmpdir) / "nonexistent")
+ assert_equal "invalid contact id: #{id}", error.message
end
-
- assert_match(/nonexistent/, error.message)
end
- def test_non_kdl_files_raise
- with_contacts({"notes.txt" => "hello"}) do |directory|
- error = assert_raises(ArgumentError) { ProTacts::Contact.all(directory) }
-
- assert_equal "invalid contact id: notes.txt", error.message
- end
- end
-
- def test_filenames_that_are_not_ids_raise
- with_contacts({"John Smith.kdl" => "name \"John\""}) do |directory|
- error = assert_raises(ArgumentError) { ProTacts::Contact.all(directory) }
-
- assert_equal "invalid contact id: John Smith", error.message
- end
- end
-
- def test_unparseable_files_raise
- with_contacts({"broken.kdl" => "contact {"}) do |directory|
- assert_raises(KDL::ParseError) { ProTacts::Contact.all(directory) }
- end
- end
-
- def test_files_whose_keys_are_not_contact_fields_raise
- with_contacts({"person.kdl" => "person { name \"A\" }"}) do |directory|
- error = assert_raises(ArgumentError) { ProTacts::Contact.all(directory) }
-
- assert_equal "unknown key in contact: person", error.message
- end
- end
-
- def test_empty_files_raise
- with_contacts({"empty.kdl" => ""}) do |directory|
- error = assert_raises(ArgumentError) { ProTacts::Contact.all(directory) }
-
- assert_equal "contact requires a name", error.message
- end
- end
-
- def test_files_whose_card_cannot_render_raise
- with_contacts({"nameless.kdl" => "phone \"+1-555-1234\""}) do |directory|
- error = assert_raises(ArgumentError) { ProTacts::Contact.all(directory) }
-
- assert_equal "contact requires a name", error.message
+ def test_uuids_and_slugs_are_ids
+ %w[AB12C345-6789-0DEF-1234-567890ABCDEF aiden znorth_2].each do |id|
+ assert_equal id, ProTacts::Contact.for(id:, vcard: CARD).id
end
end
end
diff --git a/test/pro_tacts/test_store.rb b/test/pro_tacts/test_store.rb
new file mode 100644
index 0000000..fb31300
--- /dev/null
+++ b/test/pro_tacts/test_store.rb
@@ -0,0 +1,436 @@
+require_relative "../test_helper"
+
+require "pathname"
+require "tmpdir"
+
+require "sequel"
+
+require "pro_tacts/store"
+
+class StoreTest < Minitest::Test
+ AIDEN = "BEGIN:VCARD\r\nVERSION:3.0\r\nFN:Aiden\r\nUID:aiden\r\nEND:VCARD\r\n"
+ # UTC ISO 8601 to the millisecond, which is what SQLite is asked for.
+ TIMESTAMP = /\A\d{4}-\d\d-\d\dT\d\d:\d\d:\d\d\.\d{3}Z\z/
+ ZED = "BEGIN:VCARD\r\nVERSION:3.0\r\nFN:Zed\r\nUID:znorth\r\nEND:VCARD\r\n"
+
+ # A store on disk rather than in memory: WAL, the busy timeout, and
+ # reopening a database are all part of what is under test.
+ def with_store(cards = {})
+ Dir.mktmpdir do |dir|
+ ProTacts::Store.connect(Pathname.new(dir) / "contacts.db") do |store|
+ cards.each do |id, card|
+ store.put(id, card)
+ end
+ yield store
+ end
+ end
+ end
+
+ ## Cards
+
+ def test_a_stored_card_comes_back_byte_for_byte
+ with_store({"aiden" => AIDEN}) do |store|
+ assert_equal AIDEN, store.contact("aiden").vcard
+ end
+ end
+
+ def test_contacts_are_listed_by_id
+ with_store({"znorth" => ZED, "aiden" => AIDEN}) do |store|
+ assert_equal %w[aiden znorth], store.contacts.map { it.id }
+ end
+ end
+
+ def test_an_empty_store_lists_no_contacts
+ with_store { assert_empty it.contacts }
+ end
+
+ def test_a_missing_contact_is_nil
+ with_store { assert_nil it.contact("nobody") }
+ end
+
+ def test_putting_the_same_id_replaces_the_card
+ with_store({"aiden" => AIDEN}) do |store|
+ updated = AIDEN.sub("Aiden", "Aiden Smith")
+ store.put("aiden", updated)
+
+ assert_equal [updated], store.contacts.map { it.vcard }
+ end
+ end
+
+ def test_the_etag_comes_from_the_card
+ with_store({"aiden" => AIDEN}) do |store|
+ assert_equal ProTacts::Contact.etag_for(AIDEN), store.contact("aiden").etag
+ end
+ end
+
+ # Nothing stores an etag beside a card, so there is no second copy to
+ # fall out of step: change the bytes behind the store's back and the
+ # etag follows them, because it was never anything but their hash.
+ def test_an_etag_cannot_drift_from_its_card
+ with_store({"aiden" => AIDEN}) do |store|
+ refute_includes database(store)[:cards].columns, :etag
+
+ updated = AIDEN.sub("Aiden", "Aiden Smith")
+ database(store)[:cards].where(id: "aiden").update(vcard: updated)
+
+ assert_equal ProTacts::Contact.etag_for(updated), store.contact("aiden").etag
+ end
+ end
+
+ def test_an_id_that_could_not_be_served_is_refused
+ with_store do |store|
+ assert_raises(ArgumentError) { store.put("John Smith", AIDEN) }
+ assert_empty store.contacts
+ end
+ end
+
+ def test_a_card_survives_reopening_the_database
+ Dir.mktmpdir do |dir|
+ path = Pathname.new(dir) / "contacts.db"
+ ProTacts::Store.connect(path) { it.put("aiden", AIDEN) }
+
+ ProTacts::Store.connect(path) do |store|
+ assert_equal AIDEN, store.contact("aiden").vcard
+ end
+ end
+ end
+
+ # An id and a card off the wire are ASCII-8BIT, and Sequel writes
+ # values into UTF-8 SQL: a card with an accent in it raises on the way
+ # in unless the bytes are relabelled first.
+ def test_binary_bytes_off_the_wire_are_stored_as_text
+ accented = AIDEN.sub("Aiden", "Aiden Åberg")
+
+ with_store do |store|
+ store.put("aiden".b, accented.b)
+
+ assert_equal accented, store.contact("aiden".b).vcard
+ assert_equal accented, store.contact("aiden").vcard
+ assert_equal Encoding::UTF_8, store.contact("aiden").vcard.encoding
+ assert_equal ProTacts::Contact.etag_for(accented), store.contact("aiden").etag
+ end
+ end
+
+ ## The collection's content tag
+
+ def test_the_ctag_is_stable_across_reads
+ with_store({"aiden" => AIDEN}) do |store|
+ assert_equal store.ctag, store.ctag
+ end
+ end
+
+ def test_the_ctag_moves_with_a_cards_content
+ with_store({"aiden" => AIDEN}) do |store|
+ before = store.ctag
+ store.put("aiden", AIDEN.sub("Aiden", "Aiden Smith"))
+
+ refute_equal before, store.ctag
+ end
+ end
+
+ def test_the_ctag_moves_with_membership_both_ways
+ with_store({"aiden" => AIDEN}) do |store|
+ alone = store.ctag
+
+ store.put("znorth", ZED)
+ refute_equal alone, store.ctag
+
+ store.delete("znorth")
+ assert_equal alone, store.ctag
+ end
+ end
+
+ ## Timestamps
+
+ def test_a_stored_card_is_stamped
+ with_store({"aiden" => AIDEN}) do |store|
+ row = card_row(store, "aiden")
+
+ assert_match TIMESTAMP, row.fetch(:created_at)
+ assert_equal row.fetch(:created_at), row.fetch(:updated_at)
+ end
+ end
+
+ def test_replacing_a_card_moves_updated_at_and_leaves_created_at
+ with_store({"aiden" => AIDEN}) do |store|
+ first = card_row(store, "aiden")
+ sleep 0.002 # the stamp has millisecond resolution
+ store.put("aiden", AIDEN.sub("Aiden", "Aiden Smith"))
+ second = card_row(store, "aiden")
+
+ assert_equal first.fetch(:created_at), second.fetch(:created_at)
+ assert_operator second.fetch(:updated_at), :>, first.fetch(:updated_at)
+ end
+ end
+
+ # A card written back unchanged is still a write, so it is still
+ # touched: updated_at is when the store last accepted a card, not when
+ # the bytes last differed.
+ def test_storing_the_same_card_again_still_touches_it
+ with_store({"aiden" => AIDEN}) do |store|
+ before = card_row(store, "aiden").fetch(:updated_at)
+ sleep 0.002
+ store.put("aiden", AIDEN)
+
+ assert_operator card_row(store, "aiden").fetch(:updated_at), :>, before
+ end
+ end
+
+ def test_a_change_is_stamped_when_it_is_logged
+ with_store({"aiden" => AIDEN}) do |store|
+ assert_match TIMESTAMP, store.changes.last.created_at
+ end
+ end
+
+ # Relabelling bytes is not converting them, so a card whose bytes are
+ # not UTF-8 at all is refused rather than stored: better than serving
+ # it back as `text/vcard; charset=utf-8` while it is no such thing.
+ # Pinned here so that a later "fix" to Store#text has to be deliberate.
+ def test_a_card_that_is_not_utf_8_is_refused
+ with_store do |store|
+ assert_raises(Sequel::DatabaseError) do
+ store.put("bad", "BEGIN:VCARD\r\nFN:\xFF\xFE\r\nEND:VCARD\r\n".b)
+ end
+
+ assert_empty store.contacts
+ end
+ end
+
+ ## The change log
+
+ def test_a_put_records_the_card_and_its_etag
+ with_store({"aiden" => AIDEN}) do |store|
+ change = store.changes.last
+
+ assert_equal "aiden", change.card_id
+ assert_equal "put", change.action
+ assert_equal ProTacts::Contact.etag_for(AIDEN), change.etag
+ end
+ end
+
+ def test_every_write_is_logged_in_order
+ with_store({"aiden" => AIDEN}) do |store|
+ store.put("aiden", AIDEN.sub("Aiden", "Aiden Smith"))
+ store.put("znorth", ZED)
+ store.delete("aiden")
+
+ assert_equal [%w[aiden put], %w[aiden put], %w[znorth put], %w[aiden delete]],
+ store.changes.map { [it.card_id, it.action] }
+ assert_equal store.changes.map { it.sequence }.sort, store.changes.map { it.sequence }
+ end
+ end
+
+ def test_a_delete_leaves_a_tombstone_behind
+ with_store({"aiden" => AIDEN}) do |store|
+ assert store.delete("aiden")
+ assert_nil store.contact("aiden")
+
+ tombstone = store.changes.last
+ assert_equal %w[aiden delete], [tombstone.card_id, tombstone.action]
+ assert_nil tombstone.etag
+ end
+ end
+
+ def test_deleting_nothing_logs_nothing
+ with_store({"aiden" => AIDEN}) do |store|
+ before = store.changes.length
+
+ refute store.delete("nobody")
+ assert_equal before, store.changes.length
+ end
+ end
+
+ def test_changes_can_be_read_from_a_sequence_on
+ with_store({"aiden" => AIDEN}) do |store|
+ token = store.changes.last.sequence
+ store.put("znorth", ZED)
+
+ assert_equal %w[znorth], store.changes(after: token).map { it.card_id }
+ end
+ end
+
+ # A sequence number is never reused, so a client holding an old token
+ # cannot be handed changes numbered below ones it has already seen.
+ def test_sequence_numbers_are_not_reused_after_a_delete
+ with_store({"aiden" => AIDEN}) do |store|
+ highest = store.changes.last.sequence
+ store.delete("aiden")
+ store.put("znorth", ZED)
+
+ assert_operator store.changes.last.sequence, :>, highest
+ end
+ end
+
+ # The whole reason the cards and the log are one database. Both tests
+ # below fail part way through a write that has already put the card
+ # row in: if the transaction were dropped, the card would survive and a
+ # client's sync token would point at a history that never happened.
+ #
+ # Refusing a bad id does not test this, however it is named: Contact.for
+ # raises before the transaction opens, so nothing was ever attempted.
+ def test_a_write_that_fails_at_the_log_leaves_no_card
+ Dir.mktmpdir do |dir|
+ path = Pathname.new(dir) / "contacts.db"
+
+ FailingLog.connect(path) do |store|
+ assert_raises(RuntimeError) { store.put("aiden", AIDEN) }
+ end
+
+ # Reopened, so this is the file talking and not a cache.
+ ProTacts::Store.connect(path) do |store|
+ assert_empty store.contacts
+ assert_empty store.changes
+ end
+ end
+ end
+
+ # A put joins a transaction already open — the group fan-out this
+ # design needs — so a rollback out there has to take the card, its log
+ # entry and its index rows with it.
+ def test_a_put_inside_a_failed_transaction_is_rolled_back_whole
+ with_store do |store|
+ assert_raises(RuntimeError) do
+ database(store).transaction do
+ store.put("aiden", AIDEN)
+ raise "the fan-out failed"
+ end
+ end
+
+ assert_empty store.contacts
+ assert_empty store.changes
+ assert_empty indexed_names(store, "aiden")
+ end
+ end
+
+ ## The index
+
+ def test_a_card_is_indexed_by_property
+ with_store({"aiden" => AIDEN}) do |store|
+ assert_equal %w[BEGIN VERSION FN UID END], indexed_names(store, "aiden")
+ end
+ end
+
+ def test_parameters_are_indexed_with_their_property
+ with_store({"aiden" => AIDEN.sub("FN:Aiden", "TEL;TYPE=work,voice:+1-555-1234\r\nFN:Aiden")}) do |store|
+ assert_equal [%w[TYPE work], %w[TYPE voice]], indexed_parameters(store, "aiden", "TEL")
+ end
+ end
+
+ def test_replacing_a_card_replaces_its_index_rows
+ with_store({"aiden" => AIDEN}) do |store|
+ store.put("aiden", AIDEN.sub("FN:Aiden\r\n", ""))
+
+ assert_equal %w[BEGIN VERSION UID END], indexed_names(store, "aiden")
+ end
+ end
+
+ def test_deleting_a_card_takes_its_index_rows_with_it
+ with_store({"aiden" => AIDEN}) do |store|
+ store.delete("aiden")
+
+ assert_empty indexed_names(store, "aiden")
+ assert_empty indexed_parameters(store, "aiden", "FN")
+ end
+ end
+
+ # The index is derived and nothing else is authoritative in it, so
+ # throwing it away and deriving it again from the cards alone has to
+ # land in exactly the same place.
+ def test_the_index_can_be_rebuilt_from_the_cards_alone
+ with_store({"aiden" => AIDEN, "znorth" => ZED}) do |store|
+ before = index_rows(store)
+ refute_empty before
+
+ wreck_the_index(store)
+ refute_equal before, index_rows(store)
+
+ assert_empty store.rebuild_index
+ assert_equal before, index_rows(store)
+ end
+ end
+
+ # Rows, not contacts, so that a rebuild which stamped updated_at would
+ # be caught: the index is derived, and deriving it again is not a write
+ # to the card it came from.
+ def test_rebuilding_the_index_leaves_the_cards_and_the_log_alone
+ with_store({"aiden" => AIDEN, "znorth" => ZED}) do |store|
+ cards = database(store)[:cards].order(:id).all
+ changes = store.changes
+
+ sleep 0.002
+ store.rebuild_index
+
+ assert_equal cards, database(store)[:cards].order(:id).all
+ assert_equal changes, store.changes
+ end
+ end
+
+ ## Cards that will not parse
+
+ # Fail open: the bytes are what gets served, so a card the parser
+ # cannot read is still a contact. It just contributes nothing to the
+ # index, and the rebuild names it.
+ def test_an_unparseable_card_is_still_stored_and_served
+ with_store({"broken" => "this is not a vCard\r\n"}) do |store|
+ assert_equal "this is not a vCard\r\n", store.contact("broken").vcard
+ assert_empty indexed_names(store, "broken")
+ end
+ end
+
+ def test_rebuilding_names_the_cards_it_could_not_index
+ with_store({"aiden" => AIDEN, "broken" => "this is not a vCard\r\n"}) do |store|
+ assert_equal %w[broken], store.rebuild_index
+ assert_equal %w[BEGIN VERSION FN UID END], indexed_names(store, "aiden")
+ end
+ end
+
+ private
+
+ # A store whose change-log write fails, after Store#put has already put
+ # the card row in. Minitest 6 ships no stubbing, and a subclass says
+ # what is being broken more plainly than a stub would.
+ class FailingLog < ProTacts::Store
+ private
+
+ def record(card_id, action, etag)
+ raise "the change log is unavailable"
+ end
+ end
+
+ # The index is queried directly on purpose: it exists to be queried,
+ # and a test that went back through the store would not show that it
+ # can be.
+ def database(store)
+ store.instance_variable_get(:@database)
+ end
+
+ def card_row(store, id)
+ database(store)[:cards].where(id:).sole
+ end
+
+ def indexed_names(store, card_id)
+ database(store)[:card_properties].where(card_id:).order(:position).select_map(:name)
+ end
+
+ def indexed_parameters(store, card_id, name)
+ parameter = Sequel[:card_parameters]
+
+ database(store)[:card_parameters]
+ .join(:card_properties, [:card_id, :position])
+ .where(parameter[:card_id] => card_id, Sequel[:card_properties][:name] => name)
+ .order(parameter[:rowid])
+ .select_map([parameter[:name], parameter[:value]])
+ end
+
+ def index_rows(store)
+ [
+ database(store)[:card_properties].order(:card_id, :position).all,
+ database(store)[:card_parameters].order(:card_id, :position, :name, :value).all,
+ ]
+ end
+
+ def wreck_the_index(store)
+ database(store)[:card_parameters].delete
+ database(store)[:card_properties].update(value: "wrong")
+ end
+end
diff --git a/test/pro_tacts/test_vcard.rb b/test/pro_tacts/test_vcard.rb
index ec90f56..f762c54 100644
--- a/test/pro_tacts/test_vcard.rb
+++ b/test/pro_tacts/test_vcard.rb
@@ -1,496 +1,54 @@
-
require_relative "../test_helper"
-require "kdl"
require "hegel"
require "pro_tacts/vcard"
+# The writer's half of the module: nothing serves through escape and
+# fold yet, and a PUT or the web editor will.
class VCardTest < Minitest::Test
include Hegel::Syntax::Methods
- def render(kdl, uid: "test-uid")
- ProTacts::VCard.render(KDL.parse(kdl), uid:)
- end
-
- ## Unit tests
-
- def test_simple_contact
- vcard = render(<<~KDL)
- name "John Smith"
- phone "+1-555-1234" type="mobile"
- email "john@example.com" type="home"
- address type="home" {
- street "123 Main St"
- city "Springfield"
- state "IL"
- zip "62701"
- country "USA"
- }
- KDL
-
- assert_equal <<~VCARD.gsub("\n", "\r\n"), vcard
- BEGIN:VCARD
- VERSION:3.0
- N:Smith;John;;;
- FN:John Smith
- TEL;TYPE=mobile:+1-555-1234
- EMAIL;TYPE=home:john@example.com
- ADR;TYPE=home:;;123 Main St;Springfield;IL;62701;USA
- UID:test-uid
- END:VCARD
- VCARD
- end
-
- def test_single_token_name_gets_empty_given
- vcard = render(<<~KDL)
- name "Cher"
- KDL
-
- assert_includes vcard, "N:Cher;;;;"
- end
-
- def test_components_only_name
- vcard = render(<<~KDL)
- name {
- family "van Beethoven"
- given "Ludwig"
- }
- KDL
-
- assert_includes vcard, "N:van Beethoven;Ludwig;;;"
- assert_includes vcard, "FN:Ludwig van Beethoven"
- end
-
- def test_one_component_leaves_the_rest_empty
- vcard = render(<<~KDL)
- name {
- family "Bach"
- }
- KDL
-
- assert_includes vcard, "N:Bach;;;;"
- assert_includes vcard, "FN:Bach"
- end
-
- def test_fn_joins_components_in_display_order
- vcard = render(<<~KDL)
- name {
- prefix "Dr."
- given "John"
- additional "Jacob"
- family "Smith"
- suffix "Jr."
- }
- KDL
-
- assert_includes vcard, "FN:Dr. John Jacob Smith Jr."
- end
-
- def test_values_are_escaped
- vcard = render(<<~KDL)
- name "semi;colon, comma back\\\\slash"
- KDL
-
- assert_includes vcard, "FN:semi\\;colon\\, comma back\\\\slash"
- end
-
- def test_newlines_escape_as_literal_n
- vcard = render(<<~KDL)
- name "two\\nlines"
- KDL
-
- assert_includes vcard, "FN:two\\nlines"
- end
-
- def test_long_lines_fold_and_unfold_intact
- vcard = render(<<~KDL)
- name "#{"x" * 30}#{("é" * 60)}"
- KDL
-
- physical = vcard.split("\r\n")
- assert_operator physical.length, :>, 1, "expected folding"
- physical.each do
- assert_operator it.bytesize, :<=, 75
- end
-
- logical = unfold(physical)
- assert_equal "FN:#{"x" * 30}#{("é" * 60)}", logical.find { it.start_with?("FN:") }
- end
-
- def test_properties_without_type
- vcard = render(<<~KDL)
- name "John"
- phone "+1-555-1234"
- address {
- street "123 Main St"
- }
- KDL
-
- assert_includes vcard, "TEL:+1-555-1234"
- assert_includes vcard, "ADR:;;123 Main St;;;;"
- end
-
- def test_order_is_preserved
- vcard = render(<<~KDL)
- name "John"
- phone "+1-555-1"
- phone "+1-555-2"
- email "a@example.com"
- phone "+1-555-3"
- KDL
-
- lines = vcard.split("\r\n")
- assert_equal %w[+1-555-1 +1-555-2 +1-555-3], lines.grep(/\ATEL/).map { it.split(":", 2).last }
- assert_equal 1, lines.grep(/\AEMAIL/).length
- end
-
- def test_missing_name_raises
- error = assert_raises(ArgumentError) do
- render(<<~KDL)
- phone "+1-555-1234"
- KDL
- end
-
- assert_equal "contact requires a name", error.message
- end
-
- def test_duplicate_names_raise
- error = assert_raises(ArgumentError) do
- render(<<~KDL)
- name "One"
- name "Two"
- KDL
- end
-
- assert_equal "contact takes a single name", error.message
- end
-
- def test_name_with_both_display_and_components_raises
- error = assert_raises(ArgumentError) do
- render(<<~KDL)
- name "Ludwig van Beethoven" {
- family "van Beethoven"
- }
- KDL
- end
-
- assert_equal "name takes a display string or component children, not both", error.message
- end
-
- def test_name_with_neither_display_nor_components_raises
- error = assert_raises(ArgumentError) do
- render(<<~KDL)
- name
- KDL
- end
-
- assert_equal "name requires a display string or component children", error.message
- end
-
- def test_all_empty_components_raise
- error = assert_raises(ArgumentError) do
- render(<<~KDL)
- name {
- family ""
- }
- KDL
- end
-
- assert_equal "name components cannot all be empty", error.message
- end
-
- def test_property_without_value_raises
- error = assert_raises(ArgumentError) do
- render(<<~KDL)
- name "John"
- phone type="mobile"
- KDL
- end
+ ## Escaping
- assert_equal "phone requires a string argument", error.message
+ # RFC 2426 section 2.4.2. escape and fold are the writer's half of this
+ # module: nothing serves through them yet, and a PUT or the web editor
+ # will.
+ def test_the_separators_and_the_backslash_escape
+ assert_equal "Smith\\, John\\; Jr.", ProTacts::VCard.escape("Smith, John; Jr.")
+ assert_equal "a\\\\b", ProTacts::VCard.escape("a\\b")
end
- def test_unknown_contact_key_raises
- error = assert_raises(ArgumentError) do
- render(<<~KDL)
- name "John"
- emial "john@example.com"
- KDL
- end
-
- assert_equal "unknown key in contact: emial", error.message
+ # A raw line break would end the property line, so it becomes the
+ # escape instead, whichever spelling it arrived in.
+ def test_line_breaks_escape_rather_than_break_the_line
+ assert_equal "a\\nb\\nc\\nd", ProTacts::VCard.escape("a\r\nb\rc\nd")
end
- def test_unknown_name_component_raises
- error = assert_raises(ArgumentError) do
- render(<<~KDL)
- name {
- family "Smith"
- middle "Q"
- }
- KDL
- end
+ def test_a_line_at_the_limit_is_left_alone
+ line = "NOTE:#{"x" * (ProTacts::VCard::LINE_LIMIT - 5)}"
- assert_equal "unknown key in name: middle", error.message
+ assert_equal ProTacts::VCard::LINE_LIMIT, line.bytesize
+ assert_equal line, ProTacts::VCard.fold(line)
end
- def test_unknown_address_key_raises
- error = assert_raises(ArgumentError) do
- render(<<~KDL)
- name "John"
- address {
- street "123 Main St"
- province "IL"
- }
- KDL
- end
-
- assert_equal "unknown key in address: province", error.message
- end
+ def test_folding_never_splits_a_character
+ folded = ProTacts::VCard.fold("NOTE:#{"\u00e9" * 60}")
- def test_unknown_property_raises
- error = assert_raises(ArgumentError) do
- render(<<~KDL)
- name "John"
- phone "+1-555-1234" tpye="mobile"
- KDL
+ folded.split("\r\n").each do |physical|
+ assert_operator physical.bytesize, :<=, ProTacts::VCard::LINE_LIMIT
+ assert physical.valid_encoding?
end
-
- assert_equal "unknown property on phone: tpye", error.message
end
## Property tests
- #
- # The oracles are deliberately independent of the renderer: an
- # unfold/unescape/parse implementation written here in the test, so a
- # renderer bug cannot hide behind shared code.
-
- # The renderer normalizes CRLF and CR to \n before escaping, so the
- # oracle applies the same normalization before comparing.
- def normalize(text)
- text.gsub(/\r\n|\r/, "\n")
- end
-
- # Serializes a string as a KDL quoted string: backslash, quote, and
- # the line breaks are escaped (CRLF and CR normalize to a single \n,
- # matching the renderer's documented normalization), and the C0/DEL
- # control characters that KDL forbids raw inside a string become \u
- # escapes.
- def kdl_string(text)
- escaped = normalize(text).chars.map { |char|
- case char
- when "\\" then "\\\\"
- when '"' then '\\"'
- when "\n", "\r" then "\\n"
- when /[\x00-\x08\x0b\x0c\x0e-\x1f\x7f]/ then format('\u{%02x}', char.ord)
- else char
- end
- }.join
- "\"#{escaped}\""
- end
-
- # Joins physical lines back into logical ones: a line starting with a
- # single space continues the previous one. slice_when starts a new
- # group wherever the next line is not a continuation.
- def logical_lines(vcard)
- unfold(vcard.split("\r\n"))
- end
-
- def unfold(lines)
- lines.slice_when { |_line, next_line| !next_line.start_with?(" ") }
- .map { |group| group.first + group.drop(1).map { it[1..] }.join }
- end
-
- # Reverses RFC 2426 section 2.4.2 escaping.
- def unescape(text)
- text.gsub(/\\(.)/) do
- case Regexp.last_match(1)
- when "n" then "\n"
- when "\\", ";", "," then Regexp.last_match(1)
- else raise "unknown vCard escape: \\#{Regexp.last_match(1)}"
- end
- end
- end
-
- # Splits on a separator that is not itself escaped.
- def split_unescaped(text, separator)
- parts = [+""]
- index = 0
- while index < text.length
- if text[index] == "\\" && index + 1 < text.length
- parts.last << text[index, 2]
- index += 2
- elsif text[index] == separator
- parts << +""
- index += 1
- else
- parts.last << text[index]
- index += 1
- end
- end
- parts
- end
-
- # Minimal independent vCard parser: returns the fields the renderer
- # produces, with values unescaped and components split.
- def parse_vcard(vcard)
- fields = {n: [], tel: [], email: [], adr: []}
- logical_lines(vcard).each do |line|
- head, value = line.split(":", 2)
- name, raw_params = head.split(";", 2)
- type = raw_params&.then { it[/\ATYPE=(.*)\z/, 1] }
- case name
- when "FN" then fields[:fn] = unescape(value)
- when "UID" then fields[:uid] = unescape(value)
- when "N", "ADR"
- components = split_unescaped(value, ";").map { unescape(it) }
- fields[:n] = components if name == "N"
- fields[:adr] << [components, type] if name == "ADR"
- when "TEL" then fields[:tel] << [unescape(value), type]
- when "EMAIL" then fields[:email] << [unescape(value), type]
- end
- end
- fields
- end
- # The expected FN: the display string for a display-only name, or the
- # components joined in display order for a components-only name.
- # Values are normalized (CR/CRLF → LF) the way the renderer's escape
- # step normalizes them before writing.
- def expected_fn(display, components)
- return normalize(display) if components.values.none?
-
- %w[prefix given additional family suffix]
- .map { normalize(components.fetch(it.to_sym) || "") }.reject(&:empty?).join(" ")
- end
-
- # The display-name heuristic, reimplemented: last token family, the
- # rest given.
- def derived_n(display)
- tokens = normalize(display).split
- family = tokens.last || ""
- given = tokens.length > 1 ? tokens.first(tokens.length - 1).join(" ") : ""
- [family, given, "", "", ""]
- end
-
- # Builds the KDL source for a contact from generated field data.
- def kdl_contact(display:, family: nil, given: nil, additional: nil, prefix: nil, suffix: nil,
- phones: [], emails: [], addresses: [])
- name_children = {family:, given:, additional:, prefix:, suffix:}
- .filter_map { |part, value| "#{part} #{kdl_string(value)}" unless value.nil? }
-
- contact = +""
- if name_children.empty?
- contact << " name #{kdl_string(display)}\n"
- else
- contact << " name {\n#{name_children.map { " #{it}\n" }.join} }\n"
- end
- phones.each do |value, type|
- suffix = type ? " type=\"#{type}\"" : ""
- contact << " phone #{kdl_string(value)}#{suffix}\n"
- end
- emails.each do |value, type|
- suffix = type ? " type=\"#{type}\"" : ""
- contact << " email #{kdl_string(value)}#{suffix}\n"
- end
- addresses.each do |parts, type|
- suffix = type ? " type=\"#{type}\"" : ""
- inner = parts.map { |part, value| " #{part} #{kdl_string(value)}\n" }.join
- contact << " address#{suffix} {\n#{inner} }\n"
- end
- contact
- end
-
- def test_escaped_fn_survives_round_trip
+ def test_unfolding_reverses_folding
Hegel.test do |tc|
- display = tc.draw(text(min_size: 1, max_size: 200))
- vcard = ProTacts::VCard.render(
- KDL.parse(kdl_contact(display:)),
- uid: "uid"
- )
-
- fn = parse_vcard(vcard).fetch(:fn)
- raise "FN did not survive escaping" unless fn == normalize(display)
- end
- end
-
- def test_physical_lines_fit_in_75_octets
- Hegel.test do |tc|
- display = tc.draw(text(min_size: 1, max_size: 300))
- uid = tc.draw(text(min_size: 1, max_size: 300))
- vcard = ProTacts::VCard.render(
- KDL.parse(kdl_contact(display:)),
- uid:
- )
-
- physical = vcard.split("\r\n")
- too_long = physical.find { it.bytesize > 75 }
- raise "line exceeds 75 octets: #{too_long&.bytesize}" if too_long
- raise "folded away the terminators" unless physical.first == "BEGIN:VCARD" && physical.last == "END:VCARD"
- raise "UID did not survive folding" unless parse_vcard(vcard).fetch(:uid) == normalize(uid)
- end
- end
-
- def test_contact_fields_survive_round_trip
- type = from_regex("[a-zA-Z0-9]{1,10}", fullmatch: true)
- Hegel.test do |tc|
- display = tc.draw(text(max_size: 30))
- components = {
- family: tc.draw(optional(text(max_size: 20))),
- given: tc.draw(optional(text(max_size: 20))),
- additional: tc.draw(optional(text(max_size: 20))),
- prefix: tc.draw(optional(text(max_size: 20))),
- suffix: tc.draw(optional(text(max_size: 20))),
- }
- phones = tc.draw(arrays(tuples(text(max_size: 30), optional(type)), max_size: 5))
- emails = tc.draw(arrays(tuples(text(max_size: 30), optional(type)), max_size: 5))
- addresses = tc.draw(arrays(
- tuples(
- text(max_size: 20), text(max_size: 20), text(max_size: 20),
- text(max_size: 20), text(max_size: 20), optional(type),
- ),
- max_size: 3,
- ))
- uid = tc.draw(uuids)
- # The renderer rejects a components-only name whose parts are all
- # empty (FN would have nothing to draw from), so the generator
- # honors that contract.
- tc.assume(components.values.any? { !it.nil? && !it.empty? })
-
- kdl = kdl_contact(
- display:,
- **components,
- phones:,
- emails:,
- addresses: addresses.map { |street, city, state, zip, country, addr_type|
- [%w[street city state zip country].zip([street, city, state, zip, country]).to_h, addr_type]
- }
- )
- vcard = ProTacts::VCard.render(KDL.parse(kdl), uid:)
- parsed = parse_vcard(vcard)
-
- raise "FN mismatch" unless parsed.fetch(:fn) == expected_fn(display, components)
- raise "UID mismatch" unless parsed.fetch(:uid) == uid
-
- expected_n = if components.values.none?
- derived_n(display)
- else
- %i[family given additional prefix suffix].map { normalize(components.fetch(it) || "") }
- end
- raise "N mismatch" unless parsed.fetch(:n) == expected_n
-
- expected_phones = phones.map { |value, phone_type| [normalize(value), phone_type] }
- raise "TEL mismatch" unless parsed.fetch(:tel) == expected_phones
-
- expected_emails = emails.map { |value, email_type| [normalize(value), email_type] }
- raise "EMAIL mismatch" unless parsed.fetch(:email) == expected_emails
+ value = tc.draw(text(max_size: 300))
+ line = "NOTE:#{ProTacts::VCard.escape(value)}"
- raise "ADR count mismatch" unless parsed.fetch(:adr).length == addresses.length
- addresses.zip(parsed.fetch(:adr)).each do |(street, city, state, zip, country, addr_type), (got, got_type)|
- raise "ADR components mismatch" unless got == ["", "", normalize(street), normalize(city), normalize(state), normalize(zip), normalize(country)]
- raise "ADR type mismatch" unless got_type == addr_type
- end
+ raise "unfold did not reverse fold" unless ProTacts::VCard.unfold(ProTacts::VCard.fold(line)) == line
end
end
end
diff --git a/test/pro_tacts/test_web.rb b/test/pro_tacts/test_web.rb
index 5bc18db..aefd62e 100644
--- a/test/pro_tacts/test_web.rb
+++ b/test/pro_tacts/test_web.rb
@@ -7,6 +7,7 @@ require "rack/test"
require "tmpdir"
require "pro_tacts/config"
+require "pro_tacts/store"
require "pro_tacts/web"
class WebTest < Minitest::Test
@@ -146,22 +147,28 @@ class WebTest < Minitest::Test
assert_equal "Not Found", last_response.body
end
- # Swaps in a throwaway data directory so the multi-contact routes can
- # be exercised without touching the exchange fixture data.
- def with_contacts(files)
+ # A card as Contacts would send one, so the routes are exercised
+ # against stored bytes rather than anything this test renders.
+ def card(id, name)
+ "BEGIN:VCARD\r\nVERSION:3.0\r\nFN:#{name}\r\nUID:#{id}\r\nEND:VCARD\r\n"
+ end
+
+ # Hands the app a throwaway store so the multi-contact routes can be
+ # exercised without touching the exchange fixture data, and yields it
+ # so a test can change a card mid-request-sequence. Only the store is
+ # swapped: nothing in a request reads configuration.
+ def with_contacts(names)
Dir.mktmpdir do |dir|
- contacts_dir = Pathname.new(dir) / "contacts"
- Dir.mkdir(contacts_dir)
- files.each { |name, content| File.write(contacts_dir / name, content) }
- original = ProTacts.config
- ProTacts.config = ProTacts::Config.new({
- "RACK_ENV" => "test",
- "PRO_TACTS_DATA_DIR" => dir,
- })
- begin
- yield contacts_dir
+ original = ProTacts::Web.store
+
+ ProTacts::Store.connect(Pathname.new(dir) / "contacts.db") do |store|
+ ProTacts::Web.store = store
+ names.each do |id, name|
+ store.put(id, card(id, name))
+ end
+ yield store
ensure
- ProTacts.config = original
+ ProTacts::Web.store = original
end
end
end
@@ -203,11 +210,8 @@ class WebTest < Minitest::Test
XML
end
- def test_listing_and_multiget_serve_every_contact_on_disk
- with_contacts({
- "aiden.kdl" => "name \"Aiden\"",
- "znorth.kdl" => "name \"Zed\"",
- }) do
+ def test_listing_and_multiget_serve_every_stored_contact
+ with_contacts({"aiden" => "Aiden", "znorth" => "Zed"}) do
request "/dav/addressbook/", method: "PROPFIND", "HTTP_DEPTH" => "1", input: etag_only_propfind
assert_equal 207, last_response.status
@@ -225,7 +229,7 @@ class WebTest < Minitest::Test
end
def test_multiget_reports_unknown_hrefs_as_404
- with_contacts({"aiden.kdl" => "name \"Aiden\""}) do
+ with_contacts({"aiden" => "Aiden"}) do
request "/dav/addressbook/", method: "REPORT", input: multiget("aiden", "nope")
assert_equal 207, last_response.status
@@ -236,7 +240,7 @@ class WebTest < Minitest::Test
end
def test_multiget_escapes_vcard_content_for_xml
- with_contacts({"aiden.kdl" => "name \"A & B <Team>\""}) do
+ with_contacts({"aiden" => "A & B <Team>"}) do
request "/dav/addressbook/", method: "REPORT", input: multiget("aiden")
assert_equal 207, last_response.status
@@ -245,7 +249,7 @@ class WebTest < Minitest::Test
end
def test_sync_collection_returns_etags_only
- with_contacts({"aiden.kdl" => "name \"Aiden\""}) do
+ with_contacts({"aiden" => "Aiden"}) do
request "/dav/addressbook/", method: "REPORT", input: <<~XML
<?xml version="1.0" encoding="UTF-8"?>
<A:sync-collection xmlns:A="DAV:">
@@ -272,7 +276,7 @@ class WebTest < Minitest::Test
directory = ProTacts.config.unhandled_dir
FileUtils.rm_rf(directory)
- with_contacts({"aiden.kdl" => "name \"Aiden\""}) do
+ with_contacts({"aiden" => "Aiden"}) do
request "/dav/addressbook/", method: "REPORT", input: addressbook_query
assert_equal 403, last_response.status
@@ -290,7 +294,7 @@ class WebTest < Minitest::Test
end
def test_etags_agree_across_listing_multiget_and_get
- with_contacts({"aiden.kdl" => "name \"Aiden\""}) do
+ with_contacts({"aiden" => "Aiden"}) do
request "/dav/addressbook/", method: "PROPFIND", "HTTP_DEPTH" => "1", input: etag_only_propfind
etag = last_response.body[%r{<d:getetag>(.+)</d:getetag>}, 1]
@@ -303,7 +307,7 @@ class WebTest < Minitest::Test
end
def test_a_changed_card_changes_its_etag_and_the_collection_tags
- with_contacts({"aiden.kdl" => "name \"Aiden\""}) do |contacts_dir|
+ with_contacts({"aiden" => "Aiden"}) do |store|
read_tags = lambda {
request "/dav/addressbook/", method: "PROPFIND"
[last_response.body[%r{<d:getetag>(.+)</d:getetag>}, 1],
@@ -314,7 +318,7 @@ class WebTest < Minitest::Test
before = read_tags.call
assert_equal before, read_tags.call # stable across requests
- File.write(contacts_dir / "aiden.kdl", "name \"Aiden Smith\"")
+ store.put("aiden", card("aiden", "Aiden Smith"))
after = read_tags.call
before.zip(after).each { refute_equal it.first, it.last }
diff --git a/test/pro_tacts/vcard/test_parser.rb b/test/pro_tacts/vcard/test_parser.rb
new file mode 100644
index 0000000..6554027
--- /dev/null
+++ b/test/pro_tacts/vcard/test_parser.rb
@@ -0,0 +1,210 @@
+require_relative "../../test_helper"
+
+require "hegel"
+
+require "pro_tacts/vcard/parser"
+
+class VCardParserTest < Minitest::Test
+ include Hegel::Syntax::Methods
+
+ def parse(card)
+ ProTacts::VCard::Parser.parse(card)
+ end
+
+ # A card as it arrives: CRLF line breaks, trailing break included.
+ def card(*lines)
+ lines.join("\r\n") + "\r\n"
+ end
+
+ ## Content lines
+
+ def test_every_line_comes_back_as_a_property
+ properties = parse(card(
+ "BEGIN:VCARD",
+ "VERSION:3.0",
+ "FN:Test Contact",
+ "END:VCARD",
+ ))
+
+ assert_equal %w[BEGIN VERSION FN END], properties.map { it.name }
+ assert_equal %w[VCARD 3.0], properties.first(2).map { it.value }
+ end
+
+ def test_a_value_may_contain_colons_and_spaces
+ property = parse(card("URL:https://example.com/a:b")).first
+
+ assert_equal "https://example.com/a:b", property.value
+ end
+
+ def test_a_value_may_be_empty
+ property = parse(card("NOTE:")).first
+
+ assert_equal "NOTE", property.name
+ assert_equal "", property.value
+ end
+
+ def test_a_value_keeps_its_escaping
+ property = parse(card("FN:Smith\\, John\; Jr.")).first
+
+ assert_equal "Smith\\, John\; Jr.", property.value
+ end
+
+ def test_a_group_is_split_off_the_name
+ properties = parse(card("item1.X-ABLabel:_$!<Home>!$_", "X-ABADR:us"))
+
+ assert_equal ["item1", nil], properties.map { it.group }
+ assert_equal %w[X-ABLabel X-ABADR], properties.map { it.name }
+ end
+
+ def test_blank_lines_are_skipped
+ assert_equal %w[FN UID], parse("FN:A\r\n\r\nUID:b\r\n").map { it.name }
+ end
+
+ def test_a_final_line_break_is_optional
+ assert_equal %w[END], parse("END:VCARD").map { it.name }
+ end
+
+ # Contacts sends CRLF, but a card that reached the store some other way
+ # should not be unreadable over a line ending.
+ def test_bare_line_feeds_are_accepted
+ assert_equal %w[FN UID], parse("FN:A\nUID:b\n").map { it.name }
+ end
+
+ ## Parameters
+
+ def test_a_parameter_is_a_name_and_a_value
+ property = parse(card("TEL;TYPE=work:+1-555-1234")).first
+
+ assert_equal [%w[TYPE work]], property.parameters
+ assert_equal "+1-555-1234", property.value
+ end
+
+ # RFC 2426 section 3.3.1 permits both spellings, so both parse to the
+ # same pairs. Contacts writes the first (see
+ # docs/plans/2026-08-24-corrections-from-the-first-write.md).
+ def test_a_repeated_parameter_and_a_value_list_agree
+ repeated = parse(card("ADR;type=HOME;type=pref:;;1 Main St;;;;")).first
+ listed = parse(card("ADR;type=HOME,pref:;;1 Main St;;;;")).first
+
+ assert_equal [%w[type HOME], %w[type pref]], repeated.parameters
+ assert_equal repeated.parameters, listed.parameters
+ end
+
+ # A quoted value is how a parameter carries the delimiters that would
+ # otherwise end it.
+ def test_a_quoted_parameter_value_keeps_its_delimiters
+ property = parse(card('PHOTO;X-NOTE="a;b:c,d":data')).first
+
+ assert_equal [["X-NOTE", "a;b:c,d"]], property.parameters
+ assert_equal "data", property.value
+ end
+
+ def test_an_empty_parameter_value_is_allowed
+ assert_equal [["TYPE", ""]], parse(card("TEL;TYPE=:+1-555-1234")).first.parameters
+ end
+
+ ## Folding
+
+ # Unfolding takes the line break and the one whitespace character after
+ # it, and nothing else: a fold falls between two octets, not between
+ # two words, so a space either side of it is content.
+ def test_a_folded_line_is_one_property
+ property = parse(card("NOTE:supercalifragilistic", " expialidocious")).first
+
+ assert_equal "supercalifragilisticexpialidocious", property.value
+ end
+
+ # A fold lands wherever 75 octets falls, which is not always in the
+ # value: Contacts does not look at what it is splitting.
+ def test_a_fold_inside_the_parameters_is_unfolded_too
+ property = parse(card("TEL;TY", " PE=work:+1-555-1234")).first
+
+ assert_equal [%w[TYPE work]], property.parameters
+ assert_equal "+1-555-1234", property.value
+ end
+
+ # A line of nothing but whitespace is a fold by the letter of RFC 2426
+ # section 2.6 — a line break followed by a space — so it joins the line
+ # above rather than standing alone, and whatever whitespace is left
+ # over becomes part of that value.
+ def test_a_whitespace_only_line_folds_into_the_one_above_it
+ properties = parse(card("FN:A", " ", "UID:b"))
+
+ assert_equal %w[FN UID], properties.map { it.name }
+ assert_equal "A ", properties.first.value
+ end
+
+ # With no line above it to fold into, the same whitespace is simply not
+ # a content line. One stray line at the top of a card therefore costs
+ # the whole card its index entry — it is still served, from its bytes.
+ def test_a_leading_whitespace_only_line_raises
+ assert_raises(ProTacts::VCard::ParseError) { parse(card(" ", "FN:A")) }
+ end
+
+ def test_a_continuation_may_start_with_a_tab
+ assert_equal "abcd", parse(card("NOTE:ab", "\tcd")).first.value
+ end
+
+ ## Refusals
+
+ def test_a_line_without_a_colon_raises
+ error = assert_raises(ProTacts::VCard::ParseError) { parse(card("FN")) }
+
+ assert_match(/expected ':'/, error.message)
+ end
+
+ def test_a_line_with_no_name_raises
+ assert_raises(ProTacts::VCard::ParseError) { parse(card(":value")) }
+ end
+
+ # vCard 2.1 allowed `TEL;HOME:...`; the 3.0 grammar does not, and
+ # guessing which parameter was meant would invent data.
+ def test_a_parameter_without_a_value_raises
+ error = assert_raises(ProTacts::VCard::ParseError) { parse(card("TEL;HOME:+1-555-1234")) }
+
+ assert_match(/HOME has no value/, error.message)
+ end
+
+ def test_bytes_that_are_not_a_card_raise
+ assert_raises(ProTacts::VCard::ParseError) { parse("{\"json\": true}\r\n") }
+ end
+
+ ## Property tests
+
+ # An independent unescaper, so a bug shared with the escaper cannot
+ # hide: reverses RFC 2426 section 2.4.2.
+ def unescape(text)
+ text.gsub(/\\(.)/) { ::Regexp.last_match(1) == "n" ? "\n" : ::Regexp.last_match(1) }
+ end
+
+ # The renderer normalizes CRLF and CR to \n before escaping, so the
+ # oracle applies the same normalization before comparing.
+ def normalize(text)
+ text.gsub(/\r\n|\r/, "\n")
+ end
+
+ def test_an_escaped_and_folded_value_parses_back
+ Hegel.test do |tc|
+ value = tc.draw(text(max_size: 300))
+ line = ProTacts::VCard.fold("NOTE:#{ProTacts::VCard.escape(value)}")
+
+ properties = ProTacts::VCard::Parser.parse("#{line}\r\n")
+ raise "expected one property, got #{properties.length}" unless properties.length == 1
+ raise "value did not survive" unless unescape(properties.first.value) == normalize(value)
+ end
+ end
+
+ # Folding and unfolding are inverses on a rendered line, which is what
+ # makes an index rebuilt from stored cards agree with what was served.
+ def test_a_parameter_value_survives_a_round_trip
+ token = from_regex("[A-Za-z0-9-]{1,20}", fullmatch: true)
+ Hegel.test do |tc|
+ name = tc.draw(token)
+ values = tc.draw(arrays(token, min_size: 1, max_size: 4))
+ line = "TEL#{values.map { ";#{name}=#{it}" }.join}:+1-555-1234"
+
+ property = ProTacts::VCard::Parser.parse("#{line}\r\n").first
+ raise "parameters did not survive" unless property.parameters == values.map { [name, it] }
+ end
+ end
+end
diff --git a/test/sequel/extensions/test_sole.rb b/test/sequel/extensions/test_sole.rb
new file mode 100644
index 0000000..16c852a
--- /dev/null
+++ b/test/sequel/extensions/test_sole.rb
@@ -0,0 +1,85 @@
+require_relative "../../test_helper"
+
+require "sequel"
+require "tmpdir"
+
+require "sequel/extensions/sole"
+
+class SoleTest < Minitest::Test
+ def with_rows(ids)
+ Dir.mktmpdir do |dir|
+ database = Sequel.sqlite("#{dir}/t.db")
+ database.extension(:sole)
+ database.create_table(:things) do
+ String :id
+ String :body
+ end
+ ids.each do |id|
+ database[:things].insert(id:, body: "body of #{id}")
+ end
+ yield database[:things]
+ end
+ end
+
+ def test_the_one_row_comes_back
+ with_rows(%w[a b]) do |things|
+ assert_equal "body of a", things.where(id: "a").sole.fetch(:body)
+ end
+ end
+
+ # Sequel's own `first!` raises this for an empty dataset, so a caller
+ # for whom no row is ordinary has one error to name.
+ def test_no_rows_raises
+ with_rows(%w[a]) do |things|
+ assert_raises(Sequel::NoMatchingRow) { things.where(id: "nobody").sole }
+ end
+ end
+
+ # The half `first` gets wrong: it would answer with one of the two.
+ def test_more_than_one_row_raises
+ with_rows(%w[a a]) do |things|
+ assert_raises(Sequel::Sole::TooManyRows) { things.where(id: "a").sole }
+ end
+ end
+
+ # An exception message is one of the things that reaches Sentry, and a
+ # filter can carry card content, so the message names the table only.
+ def test_the_message_names_the_table_and_not_the_query
+ with_rows(%w[a a]) do |things|
+ error = assert_raises(Sequel::Sole::TooManyRows) { things.where(body: "body of a").sole }
+
+ assert_equal "more than one row in things", error.message
+ refute_includes error.message, "body of a"
+ end
+ end
+
+ def test_it_is_a_sequel_error
+ assert_operator Sequel::Sole::TooManyRows, :<, Sequel::Error
+ end
+
+ # Everything Sequel logs, which is how the query itself is inspected.
+ class QueryLog
+ attr_reader :queries
+
+ def initialize
+ @queries = []
+ end
+
+ def info(message)
+ @queries << message
+ end
+ alias_method :error, :info
+ end
+
+ # A filter that turned out to match every row must not drag them all
+ # back to prove it.
+ def test_it_reads_two_rows_at_most
+ with_rows(%w[a a a a a]) do |things|
+ log = QueryLog.new
+ things.db.loggers << log
+
+ assert_raises(Sequel::Sole::TooManyRows) { things.sole }
+ assert(log.queries.any? { it.include?("LIMIT 2") }, log.queries.inspect)
+ end
+ end
+end
diff --git a/test/test_helper.rb b/test/test_helper.rb
index a30fcd8..80a7269 100644
--- a/test/test_helper.rb
+++ b/test/test_helper.rb
@@ -1,14 +1,25 @@
-# The fixtures directory serves as the data directory: its contacts/
-# holds the card the recorded macOS exchange asked for, so the replay
-# resolves the same hrefs the client did. Nothing else is set up:
-# requiring the app has no side effects, and config.ru never runs here.
+# A throwaway data directory holds the database the tests run against,
+# seeded from test/fixtures/cards so the recorded macOS exchange resolves
+# the same hrefs the real client did.
+#
+# The environment is set before the app is required rather than after it,
+# so the requires cannot all sit at the top: the app reads configuration
+# as it loads, because the unhandled-request middleware is given its
+# directory at class-definition time. Set it afterwards and the captures
+# land in log/ instead of tmp/.
require "pathname"
-ENV["PRO_TACTS_DATA_DIR"] = (Pathname.new(__dir__) / "fixtures").to_s
+ENV["PRO_TACTS_DATA_DIR"] = (Pathname.new(__dir__).parent / "tmp" / "test-data").to_s
# Several tests provoke 404s. Keep the captures out of log/ and out of the
# fixtures; UnhandledRequestsTest points the middleware at its own tmpdir.
ENV["PRO_TACTS_UNHANDLED_DIR"] = (Pathname.new(__dir__).parent / "tmp" / "test-unhandled").to_s
+require_relative "fixture_data"
+require "pro_tacts/web"
require "minitest/autorun"
+
+# Standing in for config.ru, which never runs here: build the store and
+# hand it to the app.
+ProTacts::Web.store = FixtureData.install(ENV.fetch("PRO_TACTS_DATA_DIR"))