search: a key's fields are lists, its comments half as long
The key joined each field into one string, which needed a paragraph
on why a word cannot span the separator. Its fields are now lists,
stored as one JSON column, so matching reads like the walk it was;
Key parses and writes itself, a group's key adds to a member's with
`+`, and the store hands back keys by id rather than a KeyedContact
wrapper. The arguments the plan already makes are cited, not
repeated. Migration 014 is edited in place: it has only run on
preview deploys, which reseed from the fixtures.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EQvcDxcd8ucY7iVaS6cJB8
diff --git a/db/migrations/014_search_keys.rb b/db/migrations/014_search_keys.rb
index 5783d9c..a84aa42 100644
--- a/db/migrations/014_search_keys.rb
+++ b/db/migrations/014_search_keys.rb
@@ -1,17 +1,8 @@
-# Each card's search key: its values folded the way the search compares
-# them, worked out when the card is written so that a search reads
-# these rows rather than building every contact to read them off
-# (docs/plans/2026-10-17-search-keys.md). Derived from the stored card
-# like card_properties, so Store#rebuild_index makes it again and a
-# card's delete cascades it away.
-#
-# Filled here through Admin::Search.key, the app's own rule, rather
-# than a copy spelled out against VCard the way 012 is: this is index,
-# not data, so what it must agree with is the search reading it today,
-# and a later change to what is keyed re-derives it the same way.
-#
-# `text: true` on every column because this is a STRICT table, which
-# refuses Sequel's plain String.
+# Each card's search key (Admin::Search::Key) as JSON, so a search
+# reads these rather than building every contact
+# (docs/plans/2026-10-17-search-keys.md). Index, like card_properties:
+# filled here through the app's own Search.key, since what it must
+# agree with is the search that reads it.
require "pro_tacts/admin/search"
require "pro_tacts/contact"
require "pro_tacts/vcard"
@@ -20,15 +11,13 @@ Sequel.migration do
up do
create_table(:search_keys, strict: true) do
foreign_key :card_id, :cards, type: String, text: true, primary_key: true, null: false, on_delete: :cascade
- String :text, text: true, null: false
- String :phones, text: true, null: false
- String :names, text: true, null: false
+ String :key, text: true, null: false
end
self[:cards].order(:id).each do |row|
id = row.fetch(:id)
contact = ProTacts::Contact.new(id:, stored: ProTacts::VCard.new(row.fetch(:vcard)), birthday: nil, inherited: [])
- self[:search_keys].insert(card_id: id, **ProTacts::Admin::Search.key(contact).to_h)
+ self[:search_keys].insert(card_id: id, key: ProTacts::Admin::Search.key(contact).to_json)
end
end
diff --git a/docs/plans/2026-10-17-search-keys.md b/docs/plans/2026-10-17-search-keys.md
index 6bc5597..7611720 100644
--- a/docs/plans/2026-10-17-search-keys.md
+++ b/docs/plans/2026-10-17-search-keys.md
@@ -28,11 +28,14 @@ list costs, and a cap would be a separate decision.
## Shape
-- `search_keys` holds one row per card: `text` (each text field
- folded, one per line), `phones` (each number's digits) and `names`
- (the name and nickname, for ranking). `Store#reindex` writes the row
+- `search_keys` holds one row per card, an `Admin::Search::Key` as
+ JSON: the folded text fields, each number's digits, and the words of
+ the name and nickname, for ranking. `Store#reindex` writes the row
next to the card's other index rows, and `#rebuild_index` derives it
again. Migration 014 fills it for the cards that are already stored.
+ The migration uses the app's own `Search.key`, unlike 012, which
+ spells its rule out so it cannot drift. This table is index, not
+ data, so what it has to agree with is the search that reads it today.
- `Admin::Search` is still the one place that decides a match. It
builds the key (`Search.key`) and reads it. The matching rules have
not changed.
diff --git a/lib/pro_tacts/admin/search.rb b/lib/pro_tacts/admin/search.rb
index d4d1273..b5d474d 100644
--- a/lib/pro_tacts/admin/search.rb
+++ b/lib/pro_tacts/admin/search.rb
@@ -1,3 +1,5 @@
+require "json"
+
require "pro_tacts/admin/format"
module ProTacts
@@ -11,43 +13,46 @@ module ProTacts
# compares digits alone, since a number is spelled with whatever
# punctuation its writer liked.
#
- # Matching stays substring-generous rather than typo-tolerant: a
- # family's book is small enough to walk whole, and SQLite FTS5 was
- # weighed and set aside for it, twice (ranger pnrqpyvk, and
- # docs/plans/2026-10-17-search-keys.md). What is walked is each
- # contact's Key, folded when its card is written, rather than the
- # contacts themselves.
+ # Matching stays substring-generous rather than typo-tolerant or
+ # indexed: a family's book is small enough to walk whole, as Keys
+ # folded when each card is written
+ # (docs/plans/2026-10-17-search-keys.md).
module Search
# The punctuation a phone number is written with, which a word made
# of nothing else (and at least one digit) is a number spelled.
PHONE_WORD = /\A[\d().+-]*\d[\d().+-]*\z/ #: Regexp
private_constant :PHONE_WORD
- # A contact's values folded for matching, worked out once when its
- # card is written (Store#reindex) rather than on every search, so
- # a search reads three short columns and builds only the contacts
- # it finds. `text` is every text field (#text_fields) folded and
- # one to a line, `phones` each number's digits, a space between,
- # and `names` the name and nickname a name hit is read off. A word
- # is split on whitespace, so it cannot span the separator between
- # two values: a word found in the joined string is a word found in
- # one of them.
- #
- # A group's values are keyed the same way, at search time, from
- # what it lends (Group#reading) and its label: a contact is found
- # by those too, and keying them into each member would leave every
- # member's key to rewrite on a rename or a change to a lent line.
+ # What a word is matched against, already folded: the text fields,
+ # each number's digits, and the words of the name and nickname a
+ # name hit is read off. Stored per card (Store#reindex); a group's
+ # is made at search time, and added to each member's.
# @rbs skip
- Key = Data.define(:text, :phones, :names)
+ Key = Data.define(:texts, :phones, :names)
+
+ # Reopened rather than defined in the block above, for the reason
+ # CardDiff is.
+ class Key
+ #: (String json) -> Key
+ def self.parse(json)
+ fields = JSON.parse(json)
+ new(texts: fields.fetch("texts"), phones: fields.fetch("phones"), names: fields.fetch("names"))
+ end
+
+ #: (*untyped) -> String
+ def to_json(*) = JSON.generate({texts:, phones:, names:})
- # A contact's key, read off its own card alone: what its groups
- # lend is matched as the group's (Key).
+ #: (Key other) -> Key
+ def +(other) = with(texts: texts + other.texts, phones: phones + other.phones)
+ end
+
+ # A contact's key, off its own card alone.
#: (Contact | ListedContact contact) -> Key
def self.key(contact)
Key.new(
- text: text_fields(contact).map { Format.search_key(it) }.join("\n"),
- phones: contact.phones.map { digits(it.value) }.join(" "),
- names: [contact.name, contact.nickname].compact.map { Format.search_key(it) }.join(" "),
+ texts: text_fields(contact).map { Format.search_key(it) },
+ phones: contact.phones.map { digits(it.value) },
+ names: [contact.name, contact.nickname].compact.flat_map { Format.search_key(it).split },
)
end
@@ -55,20 +60,18 @@ module ProTacts
# is read: a hit on the name first — every word the start of a
# word of the name or nickname, "ada" or "love" for Ada Lovelace —
# then the hits anywhere else, recency keeping its order within
- # each. The rows arrive in recency order, and partition keeps it.
- # The groups arrive whole because a contact is found by the names
- # of the groups it is in and by what they lend it.
- #: (Array[Store::KeyedContact] rows, query: String, groups: Array[Group]) -> Array[String]
- def self.contacts(rows, query:, groups:)
+ # each. The keys arrive newest first, and partition keeps it.
+ #: (Hash[String, Key] keys, query: String, groups: Array[Group]) -> Array[String]
+ def self.contacts(keys, query:, groups:)
words = words(query)
lent = {} #: Hash[String, Array[Key]]
groups.each { |group|
key = group_key(group)
group.members.each { (lent[it] ||= []) << key }
}
- matched = rows.select { |row| matches?([row.key, *lent.fetch(row.id, [])], words) }
- names, rest = matched.partition { name_hit?(it.key, words) }
- (names + rest).map(&:id)
+ matched = keys.select { |id, key| matches?(lent.fetch(id, []).reduce(key) { |all, more| all + more }, words) }
+ names, rest = matched.keys.partition { name_hit?(keys.fetch(it), words) }
+ names + rest
end
# The groups whose names the query matches, every word somewhere
@@ -88,21 +91,20 @@ module ProTacts
end
private_class_method :words
- # What a group adds to each member's key: the lines it lends, read
- # as a card, and its label. A name it does not lend, so its key's
- # names stay its own and no member's name hit comes from a group.
+ # The lines a group lends, read as a card, and its label. Lent
+ # lines hold no name, so a name hit is always the contact's own.
#: (Group group) -> Key
def self.group_key(group)
lent = key(group.reading)
- lent.with(text: [lent.text, Format.search_key(group.label)].join("\n"))
+ lent.with(texts: [*lent.texts, Format.search_key(group.label)])
end
private_class_method :group_key
- #: (Array[Key] keys, Array[String] words) -> bool
- def self.matches?(keys, words)
+ #: (Key key, Array[String] words) -> bool
+ def self.matches?(key, words)
words.all? { |word|
- keys.any? { it.text.include?(word) } ||
- (word.match?(PHONE_WORD) && keys.any? { it.phones.include?(digits(word)) })
+ key.texts.any? { it.include?(word) } ||
+ (word.match?(PHONE_WORD) && key.phones.any? { it.include?(digits(word)) })
}
end
private_class_method :matches?
@@ -126,8 +128,7 @@ module ProTacts
#: (Key key, Array[String] words) -> bool
def self.name_hit?(key, words)
- name_words = key.names.split
- words.all? { |word| name_words.any? { it.start_with?(word) } }
+ words.all? { |word| key.names.any? { it.start_with?(word) } }
end
private_class_method :name_hit?
diff --git a/lib/pro_tacts/store.rb b/lib/pro_tacts/store.rb
index cb79d9d..87d0805 100644
--- a/lib/pro_tacts/store.rb
+++ b/lib/pro_tacts/store.rb
@@ -91,12 +91,6 @@ module ProTacts
# @rbs skip
RecentContact = Data.define(:contact, :updated_at)
- # A contact's id beside the key a search matches it by
- # (Admin::Search::Key), with no contact built: what a search walks
- # the whole book as, building only the contacts it finds.
- # @rbs skip
- KeyedContact = Data.define(:id, :key)
-
# A contact paired with the calendar day its birthday next lands
# on — the read behind the admin's upcoming-birthdays list, where
# the order is the coming year's, not the stored components'.
@@ -272,8 +266,8 @@ module ProTacts
# Every contact as a list shows it, paired with its card's
# updated_at, newest first — the read behind any "recently updated"
# surface, ordered by the column SQLite already stamps rather than
- # by id. The ids narrow it to the contacts a search found, and the
- # limit to the newest few, and either way only those are built.
+ # by id. Only the contacts of these ids, or the newest few, are
+ # built.
#: (?Array[String]? ids, ?limit: Integer?) -> Array[RecentContact]
def contacts_by_recency(ids = nil, limit: nil)
rows = (ids ? cards.where(id: ids) : cards).order(Sequel.desc(:updated_at)).select(:id, :updated_at)
@@ -284,15 +278,13 @@ module ProTacts
}
end
- # Every contact's search key, newest first like
- # #contacts_by_recency, which a search hands the ids it finds back
- # to (Web#search_results).
- #: () -> Array[KeyedContact]
- def keyed_by_recency
+ # Every card's search key by its id, newest first like
+ # #contacts_by_recency.
+ #: () -> Hash[String, Admin::Search::Key]
+ def search_keys_by_recency
search_keys.join(:cards, id: :card_id).order(Sequel.desc(Sequel[:cards][:updated_at]))
- .select(Sequel[:search_keys][:card_id], :text, :phones, :names).map {
- key = Admin::Search::Key.new(text: it.fetch(:text).to_s, phones: it.fetch(:phones).to_s, names: it.fetch(:names).to_s)
- KeyedContact.new(id: it.fetch(:card_id).to_s, key:)
+ .select_map([Sequel[:search_keys][:card_id], :key]).to_h { |id, key|
+ [id.to_s, Admin::Search::Key.parse(key.to_s)] #: [String, Admin::Search::Key]
}
end
@@ -1206,8 +1198,7 @@ module ProTacts
# way, and there is no version of this write that fixes a line the
# parser cannot read. Takes the stored card — with the birthday
# already subtracted — so the index never sees a BDAY no stored
- # card carries. The card's search key goes with its rows, read off
- # the stored card alone for the reason Admin::Search::Key gives.
+ # card carries.
#: (String id, VCard vcard) -> void
def reindex(id, vcard)
# Before the parse, so that a card which has stopped parsing does
@@ -1227,7 +1218,8 @@ module ProTacts
card_parameters.insert(card_id: id, position:, name:, value:)
end
end
- search_keys.insert(card_id: id, **Admin::Search.key(Contact.new(id:, stored: vcard, birthday: nil, inherited: [])).to_h)
+ key = Admin::Search.key(Contact.new(id:, stored: vcard, birthday: nil, inherited: []))
+ search_keys.insert(card_id: id, key: key.to_json)
end
# Applies whatever migrations the database has not seen. On one it is
diff --git a/lib/pro_tacts/web/search.rb b/lib/pro_tacts/web/search.rb
index a27e7b6..76812ab 100644
--- a/lib/pro_tacts/web/search.rb
+++ b/lib/pro_tacts/web/search.rb
@@ -36,8 +36,7 @@ module ProTacts
# A blank query answers the most recently updated contacts, the
# dashboard's own list, so the search page is never empty before a
- # word is typed. A query walks every contact's search key and builds
- # only the contacts it finds (docs/plans/2026-10-17-search-keys.md).
+ # word is typed. A query builds only the contacts it finds.
#: (String query) -> Admin::SearchResults
def search_results(query)
if query.empty?
@@ -47,10 +46,10 @@ module ProTacts
return Admin::SearchResults.new(contacts: recent, groups: [], recent: true)
end
- keyed = span("search.keyed_by_recency") { |step| store.keyed_by_recency.tap { step.set_attribute("rows", it.length) } }
+ keys = span("search.search_keys_by_recency") { |step| store.search_keys_by_recency.tap { step.set_attribute("rows", it.length) } }
groups = span("search.all_groups") { |step| store.all_groups.tap { step.set_attribute("rows", it.length) } }
ids = span("search.match") { |step|
- Admin::Search.contacts(keyed, query:, groups:).tap {
+ Admin::Search.contacts(keys, query:, groups:).tap {
step.set_attribute("words", query.split.length)
step.set_attribute("contacts", it.length)
}
diff --git a/sig/pro_tacts/admin/search.rbs b/sig/pro_tacts/admin/search.rbs
index e8dd9f9..34f31d9 100644
--- a/sig/pro_tacts/admin/search.rbs
+++ b/sig/pro_tacts/admin/search.rbs
@@ -1,17 +1,16 @@
# Admin::Search::Key is a Data class, whose members the inline parser
-# cannot read (see sig/pro_tacts/contact.rbs). Search itself is a plain
-# module and carries its signatures inline.
+# cannot read (see sig/pro_tacts/contact.rbs); the methods reopened
+# onto it carry their own inline types.
module ProTacts
module Admin
module Search
class Key
- attr_reader text: String
- attr_reader phones: String
- attr_reader names: String
+ attr_reader texts: Array[String]
+ attr_reader phones: Array[String]
+ attr_reader names: Array[String]
- def self.new: (text: String, phones: String, names: String) -> instance
- def with: (?text: String, ?phones: String, ?names: String) -> instance
- def to_h: () -> { text: String, phones: String, names: String }
+ def self.new: (texts: Array[String], phones: Array[String], names: Array[String]) -> instance
+ def with: (?texts: Array[String], ?phones: Array[String], ?names: Array[String]) -> instance
end
end
end
diff --git a/sig/pro_tacts/store.rbs b/sig/pro_tacts/store.rbs
index f501402..1b23018 100644
--- a/sig/pro_tacts/store.rbs
+++ b/sig/pro_tacts/store.rbs
@@ -1,7 +1,7 @@
-# Change, GroupChange, RecentContact, KeyedContact, UpcomingBirthday,
-# GroupChoice, and Snapshot are Data classes, whose members the inline
-# parser cannot read (see sig/pro_tacts/contact.rbs). Store itself is an
-# ordinary class and its types are inline in store.rb.
+# Change, GroupChange, RecentContact, UpcomingBirthday, GroupChoice, and
+# Snapshot are Data classes, whose members the inline parser cannot read
+# (see sig/pro_tacts/contact.rbs). Store itself is an ordinary class and
+# its types are inline in store.rb.
module ProTacts
class Store
class Change
@@ -32,13 +32,6 @@ module ProTacts
def self.new: (contact: ListedContact, updated_at: String) -> instance
end
- class KeyedContact
- attr_reader id: String
- attr_reader key: Admin::Search::Key
-
- def self.new: (id: String, key: Admin::Search::Key) -> instance
- end
-
class UpcomingBirthday
attr_reader contact: ListedContact
attr_reader occurs_on: Date
diff --git a/test/pro_tacts/admin/test_contacts_pages.rb b/test/pro_tacts/admin/test_contacts_pages.rb
index b3726f5..762d1d0 100644
--- a/test/pro_tacts/admin/test_contacts_pages.rb
+++ b/test/pro_tacts/admin/test_contacts_pages.rb
@@ -403,7 +403,7 @@ class AdminContactsPagesTest < Minitest::Test
assert_equal 1, transactions.length
transaction = transactions.first
spans = transaction.spans.select { it[:op].start_with?("search.") }
- assert_equal %w[search.keyed_by_recency search.all_groups search.match search.contacts_by_recency search.render], spans.map { it[:op] }
+ assert_equal %w[search.search_keys_by_recency search.all_groups search.match search.contacts_by_recency search.render], spans.map { it[:op] }
assert_equal({"otel.kind" => :internal, "words" => 1, "contacts" => 1}, spans.find { it[:op] == "search.match" }[:data])
refute_includes JSON.generate(transaction.to_h), "lovelace"
end
diff --git a/test/pro_tacts/admin/test_search.rb b/test/pro_tacts/admin/test_search.rb
index ed75e70..e1c3830 100644
--- a/test/pro_tacts/admin/test_search.rb
+++ b/test/pro_tacts/admin/test_search.rb
@@ -7,17 +7,17 @@ require "pro_tacts/store"
require "pro_tacts/vcard"
class SearchTest < Minitest::Test
- # A card from its extra lines, keyed as the store keys it: the rows
- # arrive most recent first, the way Store#keyed_by_recency hands them
- # over.
+ # A card from its extra lines, keyed as the store keys it, beside its
+ # id. The rows are listed most recent first, the way
+ # Store#search_keys_by_recency hands them over.
def row(id, name, *lines)
bytes = ["BEGIN:VCARD", "VERSION:3.0", "FN:#{name}", "UID:#{id}", *lines, "END:VCARD", ""].join("\r\n")
contact = ProTacts::Contact.new(id:, stored: ProTacts::VCard.new(bytes), birthday: nil, inherited: [])
- ProTacts::Store::KeyedContact.new(id:, key: ProTacts::Admin::Search.key(contact))
+ [id, ProTacts::Admin::Search.key(contact)]
end
def search(rows, query, groups: [])
- ProTacts::Admin::Search.contacts(rows, query:, groups:)
+ ProTacts::Admin::Search.contacts(rows.to_h, query:, groups:)
end
# Each word finds its own field: neither "ada" nor "london" is
diff --git a/test/pro_tacts/test_store.rb b/test/pro_tacts/test_store.rb
index 60c8627..a05a818 100644
--- a/test/pro_tacts/test_store.rb
+++ b/test/pro_tacts/test_store.rb
@@ -536,7 +536,7 @@ class StoreTest < Minitest::Test
assert_equal [new_id], store.group(group).members
assert_equal new_id, store.card_id_with_uid(new_id)
assert database(store)[:card_properties].where(card_id: UUID).empty?
- assert_equal [new_id], store.keyed_by_recency.map(&:id)
+ assert_equal [new_id], store.search_keys_by_recency.keys
end
end
@@ -714,10 +714,10 @@ class StoreTest < Minitest::Test
store.add_member(group, "aiden")
store.set_group_lines(group, [HOUSEHOLD_ADDRESS])
- key = store.keyed_by_recency.fetch(0).key
- assert_equal "aiden\nade\nplays chess", key.text
- assert_equal "15551234567", key.phones
- assert_equal "aiden ade", key.names
+ key = store.search_keys_by_recency.fetch("aiden")
+ assert_equal ["aiden", "ade", "plays chess"], key.texts
+ assert_equal ["15551234567"], key.phones
+ assert_equal ["aiden", "ade"], key.names
end
end
@@ -726,7 +726,7 @@ class StoreTest < Minitest::Test
sleep 0.002
store.put("znorth", vcard(ZED))
- assert_equal %w[znorth aiden], store.keyed_by_recency.map(&:id)
+ assert_equal %w[znorth aiden], store.search_keys_by_recency.keys
end
end
@@ -2507,7 +2507,7 @@ class StoreTest < Minitest::Test
end
ProTacts::Store.connect(path) do |store|
- assert_equal({"aiden" => "aiden", "znorth" => "zed"}, store.keyed_by_recency.to_h { [it.id, it.key.names] })
+ assert_equal({"aiden" => ["aiden"], "znorth" => ["zed"]}, store.search_keys_by_recency.transform_values(&:names))
end
end
end