import: stage a file per card and the list's rows at upload
A walk screen read and parsed every staged card to draw the list down its side and to match the one card it showed. The upload now splits the file into a card per file and saves each row's label, sort key and losses beside them, rewritten only for the card a Save changes; the match reads the book's listings and reads only the best match whole. docs/plans/2026-10-14-lists-read-the-index.md records why. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GNVLyiMrZJ7sZctEYq6WCM
diff --git a/docs/plans/2026-10-14-lists-read-the-index.md b/docs/plans/2026-10-14-lists-read-the-index.md
new file mode 100644
index 0000000..a10d730
--- /dev/null
+++ b/docs/plans/2026-10-14-lists-read-the-index.md
@@ -0,0 +1,46 @@
+# Lists read the index
+
+2026-10-14. A screen that lists contacts reads them from the
+`card_properties` and `card_parameters` index rather than parsing every
+stored card, and the import's walk saves its list's rows once, at the
+upload, rather than reading every staged card on every screen. Both
+came out of measuring allocations per request on a 600-card book with
+120 photos of about 80 KB each (PR #28).
+
+## Why
+
+A list shows a name, a phone or two, and an avatar, but the card it
+read them from carries the whole base64 picture. On that book
+`/contacts`, `/search` and a group's members each allocated over 200 MB
+a request, almost all of it the PHOTO lines of cards whose pictures the
+list draws as a small avatar, or not at all.
+
+## The listing type
+
+`ListedContact` is what a row, a sort, or a search reads: the
+`Contact` readings (name, nickname, organization, birthday, phones,
+emails, addresses, notes) over a card written back out of the index's
+rows with the PHOTO left out (`Store#listed_contacts`). Its readings
+are Contact's own, inherited lines and the composed birthday included,
+so a row and the contact's page cannot disagree.
+
+That card is not the stored one, so a `ListedContact` answers no bytes
+and no etag: nothing that writes, guards a write, or serves a client
+takes one. A screen that does any of those reads the contact whole
+(`Store#contact`).
+
+Whether there is a picture is read from the first bytes the index holds
+of the card's first PHOTO (`Contact::PHOTO_HEAD`, `Contact.photo_head?`),
+enough to know its type, which is all the avatar chooses between a
+picture and initials on.
+
+## The import
+
+An upload is split into a file per card under `imports/<id>/original/`
+and `revised/`, named by its place in the file, so a screen reads the
+one card it shows. The list down the walk's side is a `rows.json` of a
+label, a sort key and a count of dropped lines per card, read once at
+the upload and again for the one card a Save rewrites
+(`Web#import_row`). Matching a card against the book reads the
+listings, then reads the best match whole, since that is the one a
+merge writes into.
diff --git a/lib/pro_tacts/admin/import_sidebar.rb b/lib/pro_tacts/admin/import_sidebar.rb
index c97f29c..238d54e 100644
--- a/lib/pro_tacts/admin/import_sidebar.rb
+++ b/lib/pro_tacts/admin/import_sidebar.rb
@@ -15,7 +15,7 @@ module ProTacts
# (Admin::ContactsShow#diff_lines, .diff-removed in admin.css).
class ImportSidebar < Phlex::HTML
# @rbs @upload: String
- # @rbs @rows: Array[[::ProTacts::Contact, Integer, Integer]]
+ # @rbs @rows: Array[[String, Integer, Integer]]
# @rbs @saved: Hash[String, String]
# @rbs @current: Integer?
@@ -25,7 +25,7 @@ module ProTacts
# (Web#walk_order); `saved` the contact each row that has come in
# was written as, by the row's place in the file; `current` the
# row whose screen is open.
- #: (upload: String, rows: Array[[::ProTacts::Contact, Integer, Integer]], saved: Hash[String, String], current: Integer) -> void
+ #: (upload: String, rows: Array[[String, Integer, Integer]], saved: Hash[String, String], current: Integer) -> void
def initialize(upload:, rows:, saved:, current:)
@upload = upload
@rows = rows
@@ -38,8 +38,8 @@ module ProTacts
def view_template
nav(class: "record walk-list") do
ul(class: "card") do
- @rows.each do |(contact, dropped, index)|
- row(contact, dropped, index)
+ @rows.each do |(label, dropped, index)|
+ row(label, dropped, index)
end
end
end
@@ -47,8 +47,8 @@ module ProTacts
private
- #: (::ProTacts::Contact contact, Integer dropped, Integer index) -> void
- def row(contact, dropped, index)
+ #: (String label, Integer dropped, Integer index) -> void
+ def row(label, dropped, index)
stored = @saved[index.to_s]
li do
# Every row opens in the walk, a saved one included, as the
@@ -61,7 +61,7 @@ module ProTacts
aria_current: ("page" if index == @current),
data: {state: stored ? "saved" : "unsaved"}) do
div(style: "flex: 1; min-width: 0;") do
- div(style: "font-weight: 550;") { Format.name_label(contact) }
+ div(style: "font-weight: 550;") { label }
unless stored
span(class: "type-label diff-removed") { "-#{dropped}" } if dropped.positive?
end
diff --git a/lib/pro_tacts/import/match.rb b/lib/pro_tacts/import/match.rb
index b8d1aca..0c9b89d 100644
--- a/lib/pro_tacts/import/match.rb
+++ b/lib/pro_tacts/import/match.rb
@@ -25,13 +25,13 @@ module ProTacts
PHONE_DIGITS = 10 #: Integer
# The contacts `arriving` might be, most alike first.
- #: (Contact arriving, Array[Contact] contacts) -> Array[Contact]
+ #: (Contact arriving, Array[ListedContact] contacts) -> Array[ListedContact]
def self.candidates(arriving, contacts)
scored = contacts.filter_map { |contact|
fields = likeness(arriving, contact)
next if (fields.max || 0.0) < THRESHOLD
- pair = [contact, fields.sum(0.0)] #: [Contact, Float]
+ pair = [contact, fields.sum(0.0)] #: [ListedContact, Float]
pair
}
scored.sort_by { -it[1] }.first(LIMIT).map { it[0] }
@@ -39,7 +39,7 @@ module ProTacts
# How alike the two are in each of the three, from 0 for nothing
# alike to 1 for a value they share.
- #: (Contact arriving, Contact stored) -> Array[Float]
+ #: (Contact arriving, ListedContact stored) -> Array[Float]
def self.likeness(arriving, stored)
[
closest(names(arriving), names(stored)),
@@ -90,13 +90,13 @@ module ProTacts
# The name as its words, sorted: "Booles, Jane" and "Jane
# Booles" are one name written two ways, and an edit distance
# over the two spellings would call them strangers.
- #: (Contact contact) -> Array[String]
+ #: (Contact | ListedContact contact) -> Array[String]
def self.names(contact)
[contact.name.to_s.downcase.scan(/[[:alnum:]]+/).sort.join(" ")].reject(&:empty?)
end
# An address's case is no part of it.
- #: (Contact contact) -> Array[String]
+ #: (Contact | ListedContact contact) -> Array[String]
def self.emails(contact)
contact.emails.map { it.value.strip.downcase }.reject(&:empty?)
end
@@ -113,7 +113,7 @@ module ProTacts
}.max || 0.0
end
- #: (Contact contact) -> Array[String]
+ #: (Contact | ListedContact contact) -> Array[String]
def self.phones(contact)
contact.phones.map { phone(it.value) }.reject(&:empty?)
end
diff --git a/lib/pro_tacts/import/staged.rb b/lib/pro_tacts/import/staged.rb
index a7b1f36..e935551 100644
--- a/lib/pro_tacts/import/staged.rb
+++ b/lib/pro_tacts/import/staged.rb
@@ -8,10 +8,12 @@ require "pro_tacts/change_id"
module ProTacts
module Import
# An import in progress, on disk between the requests that make it
- # up: its three slots, and why on disk, are
- # docs/plans/2026-09-21-import-a-vcf.md, "The import waits on the
- # server". Under the data directory, which is the one place this
- # app writes (Config#data_dir), and not the system temp dir: a
+ # up: why on disk is docs/plans/2026-09-21-import-a-vcf.md, "The
+ # import waits on the server", and why a file per card with the
+ # list's rows beside them is
+ # docs/plans/2026-10-14-lists-read-the-index.md, "The import".
+ # Under the data directory, which is the one place this app writes
+ # (Config#data_dir), and not the system temp dir: a
# half-finished import is this server's own state, short-lived
# without being anyone else's.
module Staged
@@ -23,7 +25,23 @@ module ProTacts
ORIGINAL = "original" #: String
REVISED = "revised" #: String
SAVED = "saved" #: String
- SLOTS = [ORIGINAL, REVISED, SAVED].freeze #: Array[String]
+ ROWS = "rows" #: String
+ # The walk's notes, a JSON file each.
+ SLOTS = [SAVED, ROWS].freeze #: Array[String]
+
+ # The cards' two slots, each a directory holding a file per card
+ # named by its place in the file, so a screen reads the one card
+ # it shows rather than every card the upload held.
+ CARDS = [ORIGINAL, REVISED].freeze #: Array[String]
+
+ # One row of the walk's list as it was last read: what the row is
+ # called and sorted by, off the card as it will be written, and
+ # how many of its original's lines are not coming with it
+ # (Web#walk_sidebar). Read once at the upload and again for the
+ # one card a Save rewrites, rather than off every card on every
+ # screen.
+ # @rbs skip
+ Row = Data.define(:label, :sort_key, :losses)
# The two halves of the saved slot: the name of the group this
# import files its arrivals under, and the contact each saved
@@ -39,14 +57,16 @@ module ProTacts
# it carries. Minted, so the id a link or a form hands back is
# checked against the one shape this writes (#path) rather than
# trusted as a filename.
- #: (original: String, revised: String, group: String) -> String
- def self.open(original:, revised:, group:)
+ #: (originals: Array[String], revised: Array[String], rows: Array[Row], group: String) -> String
+ def self.open(originals:, revised:, rows:, group:)
sweep
id = ChangeId.mint(ChangeId::CONTACT_LENGTH)
directory = root / id
- FileUtils.mkdir_p(directory)
- File.binwrite(directory / file_name(ORIGINAL), original)
- File.binwrite(directory / file_name(REVISED), revised)
+ {ORIGINAL => originals, REVISED => revised}.each do |slot, cards|
+ FileUtils.mkdir_p(directory / slot)
+ cards.each_with_index { |bytes, index| File.binwrite(directory / slot / "#{index}.vcf", bytes) }
+ end
+ File.binwrite(directory / file_name(ROWS), JSON.generate(rows.map(&:to_h)))
# The group is settled here, and never renamed (the plan's
# "Nothing waits for the end"). Nothing saved yet, and
# empty rather than absent, so every slot of an import that
@@ -73,19 +93,45 @@ module ProTacts
File.read(file.to_s, encoding: Encoding::UTF_8)
end
- # The cards as they will be written, rewritten: the editor's
- # save, and one of the two writes this takes after the import
- # opens. Whole rather than a card at a time, because the file is
- # the unit a re-read splits (Vcf.cards) and a card's index in it
- # is how a screen names one.
- #
- # Still rewritten for a card that was just saved into the store,
- # rather than dropped from the file: the walk's list renders
- # every row from these bytes, and a row that has come in is one
- # it still has to be able to name (Admin::ImportSidebar).
- #: (String id, String revised) -> void
- def self.update(id, revised)
- write(id, REVISED, revised)
+ # One card's bytes from either cards slot, by its place in the
+ # file, or none for an import that is gone or a place past its
+ # last card.
+ #: (String id, String slot, Integer index) -> String?
+ def self.card(id, slot, index)
+ file = card_path(id, slot, index)
+ return nil if file.nil? || !file.file?
+
+ # #read's encoding, for #read's reason.
+ File.read(file.to_s, encoding: Encoding::UTF_8)
+ end
+
+ # The walk's list, a Row per card in file order, or none for an
+ # import that is gone. A slot that will not read back as rows is
+ # #saved's broken assumption, and JSON or Row says so.
+ #: (String id) -> Array[Row]?
+ def self.rows(id)
+ raw = read(id, ROWS)
+ return nil if raw.nil?
+
+ parsed = JSON.parse(raw, symbolize_names: true) #: Array[Hash[Symbol, untyped]]
+ parsed.map { Row.new(label: it.fetch(:label), sort_key: it.fetch(:sort_key), losses: it.fetch(:losses)) }
+ end
+
+ # One card as it will be written, rewritten, with its row: the
+ # editor's save, and one of the writes this takes after the
+ # import opens. Still rewritten for a card that was just saved
+ # into the store, rather than dropped: the walk's list names
+ # every row, and a row that has come in is one it still has to
+ # be able to name (Admin::ImportSidebar).
+ #: (String id, Integer index, String bytes, Row row) -> void
+ def self.update(id, index, bytes, row)
+ file = card_path(id, REVISED, index) or raise ArgumentError, "#{id} is not an import id"
+ raise ArgumentError, "#{id} has no card #{index}" unless file.file?
+
+ File.binwrite(file.to_s, bytes)
+ rows = rows(id) or raise ArgumentError, "#{id} is not an import in progress"
+ rows[index] = row
+ write(id, ROWS, JSON.generate(rows.map(&:to_h)))
end
# What this import has done: the group it files under, and the
@@ -165,6 +211,16 @@ module ProTacts
directory / file_name(slot)
end
+ # One card's file in a cards slot, checked as #path checks: the
+ # place is an Integer the route matched, and never below zero.
+ #: (String id, String slot, Integer index) -> Pathname?
+ def self.card_path(id, slot, index)
+ directory = import_root(id)
+ return nil if directory.nil? || !CARDS.include?(slot) || index.negative?
+
+ directory / slot / "#{index}.vcf"
+ end
+
# One slot, rewritten. Both refusals name which of the two
# things went wrong: an id this never minted, or an import that
# is no longer here.
@@ -187,13 +243,12 @@ module ProTacts
ids
end
- # The cards are a .vcf, and the walk's notes beside them are not.
#: (String slot) -> String
def self.file_name(slot)
- slot == ORIGINAL || slot == REVISED ? "#{slot}.vcf" : "#{slot}.json"
+ "#{slot}.json"
end
- private_class_method :root, :import_root, :path, :file_name, :write, :ids
+ private_class_method :root, :import_root, :path, :card_path, :file_name, :write, :ids
end
end
end
diff --git a/lib/pro_tacts/web/import.rb b/lib/pro_tacts/web/import.rb
index 0203347..d417988 100644
--- a/lib/pro_tacts/web/import.rb
+++ b/lib/pro_tacts/web/import.rb
@@ -23,9 +23,10 @@ module ProTacts
# (docs/plans/2026-09-21-import-a-vcf.md, "The walk" through
# "Nothing waits for the end").
#
- # Every screen re-reads the whole staged file, so each step that
- # grows with it is a span (Web#span) counting cards, never their
- # content.
+ # Every screen reads the one card it shows and the walk's list,
+ # staged as rows at the upload (#import_row); each step that grows
+ # with the file or the book is a span (Web#span) counting cards,
+ # never their content.
#
# Under the same identity gate as every other route (web.rb). A
# card this creates is ticked into everyone's book rather than the
@@ -80,9 +81,10 @@ module ProTacts
# The upload's own POST: the file — a .vcf, or a Monica export
# read as one — read and judged whole before anything is staged,
# so a source that will not read brings in nothing. What it
- # stages is both readings —
- # the file as it arrived, and the cards pared to what this book
- # shows — and then it is the walk's first card.
+ # stages is both readings — the file as it arrived, and the cards
+ # pared to what this book shows — a card at a time, with the
+ # walk's list read off them once, and then it is the walk's first
+ # card.
#: (untyped r) -> untyped
def stage_upload(r)
upload = file_in(r.params["vcf"])
@@ -104,11 +106,17 @@ module ProTacts
return import_screen(notice: error.message)
end
- revised = span("import.pare") { cards.map { Import::Vcf.read(it).card } }
+ readings = span("import.pare") { cards.map { Import::Vcf.read(it) } }
+ rows = span("import.rows") {
+ readings.each_with_index.map { |reading, index|
+ import_row(reading.card, index, Import::Vcf.losses(reading.dropped).length)
+ }
+ }
id = span("import.stage") {
Import::Staged.open(
- original: bytes,
- revised: joined(revised),
+ originals: cards.map(&:to_s),
+ revised: readings.map { it.card.to_s },
+ rows:,
# Named now and never renamed (the plan's "Nothing waits
# for the end"); its membership is ticked a contact at a time
# (#import_picker).
@@ -119,23 +127,32 @@ module ProTacts
# list's first row (#walk_order), not the file's first card. A
# 303, the other writes' answer, because every screen of the
# walk is a GET a back button can revisit.
- r.redirect "/import/#{id}/#{walk_order(import_contacts(revised)).fetch(0)}", 303
+ r.redirect "/import/#{id}/#{walk_order(rows).fetch(0)}", 303
end
# The walk's list of contacts, as every screen of it shows them:
- # the contact each card will be written as, how many of its
+ # the name each card will be written under, how many of its
# original's lines are not coming with it, and which rows are in
# the book already. `current` is the row whose screen is open.
- #: (String upload, Array[VCard] originals, Array[VCard] revised, Hash[String, String] saved, current: Integer) -> Admin::ImportSidebar
- def walk_sidebar(upload, originals, revised, saved, current:)
- rows = span("import.sidebar") {
- contacts = import_contacts(revised)
- walk_order(contacts).map { |index|
- dropped = Import::Vcf.read(originals.fetch(index)).dropped
- [contacts.fetch(index), Import::Vcf.losses(dropped).length, index] #: [Contact, Integer, Integer]
- }
+ # Drawn from the staged rows (#import_row), not the cards.
+ #: (String upload, Array[Import::Staged::Row] rows, Hash[String, String] saved, current: Integer) -> Admin::ImportSidebar
+ def walk_sidebar(upload, rows, saved, current:)
+ listed = walk_order(rows).map { |index|
+ row = rows.fetch(index)
+ [row.label, row.losses, index] #: [String, Integer, Integer]
}
- Admin::ImportSidebar.new(upload:, rows:, saved:, current:)
+ Admin::ImportSidebar.new(upload:, rows: listed, saved:, current:)
+ end
+
+ # One card's row in the walk's list, read off the contact it will
+ # be written as: once per card at the upload, and again for the
+ # card a Save rewrites, so no screen reads every card to draw the
+ # list (docs/plans/2026-10-14-lists-read-the-index.md, "The
+ # import"). The losses are the original's, which no Save moves.
+ #: (VCard card, Integer index, Integer losses) -> Import::Staged::Row
+ def import_row(card, index, losses)
+ contact = import_contact(card, index)
+ Import::Staged::Row.new(label: Admin::Format.name_label(contact), sort_key: Admin::Format.sort_key(contact), losses:)
end
# The rows' places in the file, in the order the list shows them:
@@ -144,16 +161,13 @@ module ProTacts
# where the book's id would, as a number: a row's id is that place
# spelled as a string, and "10" sorts before "2". A row is still
# named by its place (the Integer route), which a save cannot
- # move; its position in the list is read off the revised cards
- # afresh on every screen, so a family name changed on the way in
- # moves its row the way it would move the contact's. The contacts
- # rather than the cards, so the sidebar sorts the ones it shows
- # instead of reading every card a second time.
- #: (Array[Contact] contacts) -> Array[Integer]
- def walk_order(contacts)
- span("import.walk_order") {
- contacts.each_index.sort_by { |index| [Admin::Format.sort_key(contacts.fetch(index)), index] }
- }
+ # move; its position in the list is its row's sort key, which the
+ # Save that rewrites a card writes again (#import_row), so a family
+ # name changed on the way in moves its row the way it would move
+ # the contact's.
+ #: (Array[Import::Staged::Row] rows) -> Array[Integer]
+ def walk_order(rows)
+ rows.each_index.sort_by { |index| [rows.fetch(index).sort_key, index] }
end
# A row opened: the card it arrived as beside the contact editor
@@ -180,12 +194,11 @@ module ProTacts
# plan's "Writing them in").
#: (String upload, Integer index, ?into: String?, ?notice: String?, ?joined: Array[String]?, ?named: Array[String]?) -> String?
def card_screen(upload, index, into: "", notice: nil, joined: nil, named: nil)
- staged = staged_cards(upload)
- return expired_screen if staged.nil?
+ rows = Import::Staged.rows(upload)
+ return expired_screen if rows.nil?
- originals, revised = staged
- original = originals[index]
- card = revised[index]
+ original = staged_card(upload, Import::Staged::ORIGINAL, index)
+ card = staged_card(upload, Import::Staged::REVISED, index)
# An index past the end of the file is the empty-body 404 the
# not_found handler fills in, the same as a contact id nobody
# has.
@@ -195,7 +208,7 @@ module ProTacts
stored = saved[index.to_s]
contact = stored ? store.contact(stored) : nil
dropped = Import::Vcf.read(original).dropped
- sidebar = walk_sidebar(upload, originals, revised, saved, current: index)
+ sidebar = walk_sidebar(upload, rows, saved, current: index)
response["Content-Type"] = "text/html; charset=utf-8"
return span("import.render") {
@@ -213,13 +226,15 @@ module ProTacts
} if contact
arriving = import_contact(card, index)
- book = span("import.contacts") { |step| store.contacts.tap { step.set_attribute("rows", it.length) } }
+ book = span("import.contacts") { |step| store.listed_contacts.tap { step.set_attribute("rows", it.length) } }
matches = span("import.match") { Import::Match.candidates(arriving, book) }
target = into.nil? || into.empty? ? nil : store.contact(into)
# The row opened with no choice made is the first choice
# offered, the best match
- # (docs/plans/2026-09-23-update-is-the-default.md).
- target ||= matches.first if into.nil?
+ # (docs/plans/2026-09-23-update-is-the-default.md), read whole
+ # now that it is the one being folded into.
+ best = matches.first
+ target ||= store.contact(best.id) if into.nil? && best
merge = target && span("import.merge") { Import::Merge.new(target, card) }
span("import.render") {
Admin::ContactsEdit.call(
@@ -248,12 +263,13 @@ module ProTacts
# them — a toggle has to show the side it is on, and a contact
# edited since it was offered can have stopped looking like the
# card.
- #: (String upload, Integer index, Array[Contact] matches, Contact? target) -> Admin::ImportTarget?
+ #: (String upload, Integer index, Array[ListedContact] matches, Contact? target) -> Admin::ImportTarget?
def import_target(upload, index, matches, target)
- matches = [target, *matches] if target && matches.none? { it.id == target.id }
- return nil if matches.empty?
+ offered = matches #: Array[Contact | ListedContact]
+ offered = [target, *matches] if target && matches.none? { it.id == target.id }
+ return nil if offered.empty?
- Admin::ImportTarget.new(row: "/import/#{upload}/#{index}", matches:, into: target)
+ Admin::ImportTarget.new(row: "/import/#{upload}/#{index}", matches: offered, into: target)
end
# The editor over a stored row's contact, reached from that row's
@@ -266,11 +282,10 @@ module ProTacts
# goes back to the row screen.
#: (untyped r, String upload, Integer index, ?contact: Contact?, ?notice: String?) -> untyped
def edit_card_screen(r, upload, index, contact: nil, notice: nil)
- staged = staged_cards(upload)
- return expired_screen if staged.nil?
+ rows = Import::Staged.rows(upload)
+ return expired_screen if rows.nil?
- originals, revised = staged
- original = originals[index]
+ original = staged_card(upload, Import::Staged::ORIGINAL, index)
return nil if original.nil?
_group, saved = Import::Staged.saved(upload)
@@ -286,7 +301,7 @@ module ProTacts
action: "/import/#{upload}/#{index}",
save: "Save",
aside: Admin::ImportOriginal.new(card: original, dropped: Import::Vcf.read(original).dropped),
- sidebar: walk_sidebar(upload, originals, revised, saved, current: index),
+ sidebar: walk_sidebar(upload, rows, saved, current: index),
)
end
@@ -325,11 +340,10 @@ module ProTacts
# card and its groups going in together.
#: (untyped r, String upload, Integer index) -> untyped
def save_card(r, upload, index)
- staged = staged_cards(upload)
- return expired_screen if staged.nil?
+ rows = Import::Staged.rows(upload)
+ return expired_screen if rows.nil?
- _originals, revised = staged
- card = revised[index]
+ card = staged_card(upload, Import::Staged::REVISED, index)
return nil if card.nil?
_group, saved = Import::Staged.saved(upload)
@@ -427,9 +441,8 @@ module ProTacts
}
# Staged after the write, unlike a new contact's below
# (docs/plans/2026-09-23-merging-on-import.md, "The Save").
- revised[index] = edited
- span("import.restage") { Import::Staged.update(upload, joined(revised)) }
- return walk_on(r, upload, index, written, saved, revised)
+ span("import.restage") { restage(upload, index, edited, rows) }
+ return walk_on(r, upload, index, written, saved, rows)
end
# The birthday goes back into the card, an import having no
@@ -454,24 +467,33 @@ module ProTacts
# Staged before it is stored, and kept afterwards: the list
# renders every row from these bytes, so a row that has come in
# is still a row the list has to be able to name.
- revised[index] = edited
- span("import.restage") { Import::Staged.update(upload, joined(revised)) }
+ span("import.restage") { restage(upload, index, edited, rows) }
written = span("import.write") { Import::Write.call(store, edited, joins: ticked, named: wanted, everyone: syncing) }
- walk_on(r, upload, index, written, saved, revised)
+ walk_on(r, upload, index, written, saved, rows)
+ end
+
+ # A Save's card staged in place of the one it rewrote, and its row
+ # read again off it (#import_row): `rows` comes back holding the new
+ # row, for the walk on from it.
+ #: (String upload, Integer index, VCard edited, Array[Import::Staged::Row] rows) -> void
+ def restage(upload, index, edited, rows)
+ row = import_row(edited, index, rows.fetch(index).losses)
+ Import::Staged.update(upload, index, edited.to_s, row)
+ rows[index] = row
end
# A row's Save landed, as a new contact or an updated one: the row
# noted as saved, and the walk on from it. `saved` is the rows
- # saved before this one, and `revised` the file's cards with this
+ # saved before this one, and `rows` the walk's list with this
# row's save staged among them.
- #: (untyped r, String upload, Integer index, Contact written, Hash[String, String] saved, Array[VCard] revised) -> untyped
- def walk_on(r, upload, index, written, saved, revised)
+ #: (untyped r, String upload, Integer index, Contact written, Hash[String, String] saved, Array[Import::Staged::Row] rows) -> untyped
+ def walk_on(r, upload, index, written, saved, rows)
Import::Staged.record(upload, index.to_s, written.id)
# The last one: there is nothing left to come back to, so the
# walk is over rather than a list of rows that all say the same
# thing.
- return close_walk(r, upload) if saved.length + 1 == revised.length
+ return close_walk(r, upload) if saved.length + 1 == rows.length
# Onto the next row still to save below this one, in the list's
# order (#walk_order). Below rather than first, so a row
@@ -482,7 +504,7 @@ module ProTacts
# `first` because empty is the case the line above took, and
# reaching it is a broken assumption rather than a screen to
# render.
- order = walk_order(import_contacts(revised))
+ order = walk_order(rows)
remaining = order.reject { |i| saved.key?(i.to_s) || i == index }
below = order.drop(order.index(index).to_i + 1)
r.redirect "/import/#{upload}/#{below.find { remaining.include?(it) } || remaining.fetch(0)}", 303
@@ -505,25 +527,14 @@ module ProTacts
r.redirect(filed ? "/groups/#{filed.id}" : "/contacts", 303)
end
- # The import's two readings, or none when it is gone
- # (Import::Staged.read).
- #
- # The re-read parses files this already parsed, which is not a
- # second judgment of them: the bytes are what was staged, and this
- # is the only way back to the cards. A file that read on the way
- # in and will not read now is a broken assumption and raises.
- #: (String upload) -> [Array[VCard], Array[VCard]]?
- def staged_cards(upload)
- original = Import::Staged.read(upload, Import::Staged::ORIGINAL)
- revised = Import::Staged.read(upload, Import::Staged::REVISED)
- return nil if original.nil? || revised.nil?
-
- span("import.staged") { |step|
- step.set_attribute("bytes", original.bytesize + revised.bytesize)
- cards = [Import::Vcf.cards(original), Import::Vcf.cards(revised)] #: [Array[VCard], Array[VCard]]
- step.set_attribute("cards", cards[0].length)
- cards
- }
+ # One staged card, as it arrived or as it will be written, or none
+ # for a place past the file's last card (Import::Staged.card). Its
+ # bytes are what the upload split and judged (#stage_upload), so
+ # this is not a second judgment of them.
+ #: (String upload, String slot, Integer index) -> VCard?
+ def staged_card(upload, slot, index)
+ bytes = Import::Staged.card(upload, slot, index)
+ bytes && VCard.new(bytes)
end
# A staged card read as the contact the editor edits. The id is
@@ -541,29 +552,11 @@ module ProTacts
Contact.new(id: index.to_s, stored: birthday ? rest : card, birthday:, inherited: [])
end
- # Every staged card as #import_contact reads it, in file order.
- #: (Array[VCard] revised) -> Array[Contact]
- def import_contacts(revised)
- revised.each_with_index.map { |card, index| import_contact(card, index) }
- end
-
#: (Contact contact) -> bool
def carries_own_bday?(contact)
contact.stored.lines.any? { it.names?("BDAY") }
end
- # Cards back into a file, each keeping its own bytes. A break
- # after any that did not carry one: a file whose last card ends
- # without a newline is still a file, and two cards glued at
- # END:VCARDBEGIN:VCARD is not.
- #: (Array[VCard] cards) -> String
- def joined(cards)
- cards.map { |card|
- bytes = card.to_s
- bytes.end_with?("\n") ? bytes : "#{bytes}\r\n"
- }.join
- end
-
# The file part the form sent, and none for a POST carrying no
# file: #ids_in's shape over an upload, and its reason for being a
# method — the empty case needs a type, and the signature is where
diff --git a/sig/pro_tacts/admin.rbs b/sig/pro_tacts/admin.rbs
index 528a813..22df518 100644
--- a/sig/pro_tacts/admin.rbs
+++ b/sig/pro_tacts/admin.rbs
@@ -70,22 +70,23 @@ module ProTacts
end
class ImportTarget
- def self.new: (row: String, matches: Array[Contact], into: Contact?) -> instance
+ def self.new: (row: String, matches: Array[Contact | ListedContact], into: Contact?) -> instance
end
class ImportSidebar
- def self.new: (upload: String, rows: Array[[Contact, Integer, Integer]], saved: Hash[String, String], current: Integer) -> instance
+ def self.new: (upload: String, rows: Array[[String, Integer, Integer]], saved: Hash[String, String], current: Integer) -> instance
end
class ImportSaved
def self.call: (contact: Contact, login: String, groups: Array[Group], all_groups: Array[Store::GroupChoice], aside: untyped, sidebar: untyped, edit: String, land: String, ?notice: String?) -> String
end
- # Its sort_key orders the import's list the way the listings of
- # contacts are ordered (Web#walk_order), and its search_key is the
- # fold Search matches on.
+ # Its sort_key and name_label are the import's list rows'
+ # (Web#import_row), ordered the way the listings of contacts are, and
+ # its search_key is the fold Search matches on.
module Format
def self.sort_key: (Contact | ListedContact contact) -> String
+ def self.name_label: (Contact | ListedContact contact) -> String
def self.search_key: (String value) -> String
end
diff --git a/sig/pro_tacts/import.rbs b/sig/pro_tacts/import.rbs
index 8177611..93947d0 100644
--- a/sig/pro_tacts/import.rbs
+++ b/sig/pro_tacts/import.rbs
@@ -1,6 +1,7 @@
-# Import::Vcf::Unknown and ::Reading are Data classes, whose members the
-# inline parser cannot read (see sig/pro_tacts/contact.rbs). Each is
-# marked `@rbs skip` where it is defined and signed here.
+# Import::Vcf::Unknown and ::Reading, and Import::Staged::Row, are Data
+# classes, whose members the inline parser cannot read (see
+# sig/pro_tacts/contact.rbs). Each is marked `@rbs skip` where it is
+# defined and signed here.
module ProTacts
module Import
module Vcf
@@ -19,5 +20,16 @@ module ProTacts
def self.new: (card: ::ProTacts::VCard, dropped: Array[::ProTacts::VCard::Parser::Line]) -> instance
end
end
+
+ module Staged
+ class Row
+ attr_reader label: String
+ attr_reader sort_key: String
+ attr_reader losses: Integer
+
+ def self.new: (label: String, sort_key: String, losses: Integer) -> instance
+ def to_h: () -> Hash[Symbol, untyped]
+ end
+ end
end
end
diff --git a/test/pro_tacts/admin/test_import_pages.rb b/test/pro_tacts/admin/test_import_pages.rb
index 902c544..187ab0a 100644
--- a/test/pro_tacts/admin/test_import_pages.rb
+++ b/test/pro_tacts/admin/test_import_pages.rb
@@ -985,7 +985,7 @@ class AdminImportPagesTest < Minitest::Test
transactions = sentry_events.grep(Sentry::TransactionEvent)
ops = transactions.flat_map { |transaction| transaction.spans.map { it[:op] } }.grep(/\Aimport\./).uniq.sort
- assert_equal %w[contacts match merge pare render restage sidebar split stage staged walk_order write].map { "import.#{it}" }, ops
+ assert_equal %w[contacts match merge pare render restage rows split stage write].map { "import.#{it}" }, ops
transactions.each { refute_match(/Booles|Jane/, JSON.generate(it.to_h)) }
end
ensure