Upsert
External integrators replay the same payload — webhook redeliveries, file imports, retry loops after a network drop. The right primitive for “make the row look like this, whether or not it already exists” is upsert, keyed on a stable identifier the producer owns. CrateStack exposes it as.upsert(input)
on every model whose primary key is client-supplied.
When to use it
- Idempotent ingestion — an external producer (payment processor, CSV import, message-queue consumer) sends events with stable IDs that you want to converge to, not duplicate
- Cache rehydration — re-deriving a projection from a source-of-truth stream where each event already carries the resulting row state
- CRDT-style materializations — when the input fully describes the desired state and you don’t care whether the row was new
.create(...) when you want a duplicate-key error to surface a bug.
Use .update(...) when “row must already exist” is a precondition.
Eligibility
.upsert(...) is generated only on models whose @id field is
client-supplied — i.e. has no @default(...). Calling .upsert(...)
on a model with a server-generated PK (@id @default(cuid()),
@id @default(uuid_v7()), etc.) is a compile error, not a runtime
“not supported.”
.on_conflict(ConflictTarget::Columns(&["col1", "col2"])) lets the upsert target any column tuple backed by a UNIQUE
constraint or index — including a composite @@unique([...])
constraint. This is a real, currently-working mechanism (ConflictTarget::{PrimaryKey, Columns}),
available on both the server (cratestack-sqlx) upsert builder and the
embedded (cratestack-rusqlite) upsert builder symmetrically:
UNIQUE constraint/index on the
target table — the database enforces this and surfaces a clear error if
not — and the input must carry a value for every column in the target
tuple, since the conflict probe (SELECT … FOR UPDATE) filters on it.
Composite-constraint-by-name (ON CONFLICT ON CONSTRAINT my_unique_idx)
isn’t exposed; pass the matching column tuple via ConflictTarget::Columns
instead.
Programmatic use
The input shape is the sameCreate<Model>Input struct you already use
for .create(...). The runtime decides at call time whether the call
becomes an INSERT or an UPDATE.
.do_nothing(): converge without overwriting
.upsert(input).run(ctx) above always does ON CONFLICT DO UPDATE — a conflicting row gets
overwritten with the new input’s values. That’s wrong for the idempotent-ingestion case this guide
opens on whenever the retry’s payload is incomplete, not a full re-statement of the desired row: a
cash-in claim that inserts a PENDING row and treats a conflict as “already in flight” must never let
a retry’s blank values overwrite a row a downstream process has since moved to COMPLETED.
.do_nothing() (crates/cratestack-sqlx/src/query/write/upsert.rs, cratestack#487) switches the
conflict branch to a real ON CONFLICT DO NOTHING — the existing row is returned completely
untouched, not merged:
.do_nothing() returns a distinct builder (UpsertRecordDoNothing), because the return type
genuinely changes: a real DO NOTHING returns nothing at all for the conflicting row (Postgres only
RETURNINGs rows a statement actually touched), so Result<M, CratestackError> can’t express
“inserted vs. already there” — Result<UpsertOutcome<M>, CratestackError> can.
UpsertOutcome<M>::{Inserted(M), Existing(M)} exposes .was_inserted() -> bool and
.into_record() -> M/.record() -> &M for callers that only need the row. .on_conflict(...) chains
the same way as the plain path, before or after .do_nothing(). This is purely additive — existing
.upsert(...).run(...) call sites keep their current Result<M, CratestackError> signature and DO
UPDATE behavior unchanged.
Server-only. .do_nothing() exists on cratestack-sqlx’s builder; there is no
cratestack-rusqlite (embedded) equivalent — see Embedded semantics below.
Server semantics
The server (cratestack-sqlx) path is always transactional and follows a
deliberate, banking-friendly sequence:
- Validate input — schema-derived validators (
@length,@regex, …) run before any SQL - Apply create defaults —
@default(auth().*)and@default(...)columns are filled in - Evaluate create policies —
@@allow(create, …)and@@deny(create, …)must permit the call, against the input values plus defaults - Begin transaction, ensure outbox / audit tables exist
- Probe with
SELECT … FOR UPDATEon the primary key — this both predicts insert vs. update and serializes concurrent upserts on the same key - If the probe found a row → evaluate the update policy against the
live row, capture the
beforesnapshot, and runDO UPDATE. Denial is indistinguishable from a missing row, matching ordinary.update(...)semantics. - If the probe found no row → execute
INSERT … ON CONFLICT (<target>) DO NOTHING RETURNING …, so the database itself answers “did I actually insert?” - A returned row means a genuine insert —
Created,AuditOperation::Create, nobeforesnapshot, one statement, no extra round trip - No returned row means the probe lost a race, and the winning row
has not been touched yet. The runtime re-enters the update branch from
the top, re-running the same probe — which blocks until the winning
transaction commits, so it reads that transaction’s final data:
- Re-probe finds the winner (the ordinary case) → run the update
policy gate, capture a real
beforesnapshot, thenDO UPDATE. Outcome isUpdated/AuditOperation::Updatewith the winner’s row asbefore - Re-probe still finds nothing → the conflict is real but
invisible to the probe.
DO UPDATEruns without the update policy gate and the outcome is reported asInserted/Createdwith nobefore. Two causes: the winning row was deleted again between the two statements (in which case the statement really did insert, andInsertedis correct), or a soft-delete tombstone sits at the conflict target — see below, where that second case is a known defect rather than the correct answer
- Re-probe finds the winner (the ordinary case) → run the update
policy gate, capture a real
- Enqueue the event and the audit entry reflecting what the database actually did, then commit and drain the outbox
Why the database decides, not the probe (#745).
Before 0.8.14 the Created-vs-Updated decision came from the pre-statement
probe and was never reconciled against what actually happened. When a
concurrent transaction committed a conflicting row in the gap, Postgres
serialized on the unique index and performed a genuine UPDATE — but
the runtime still emitted a
Created event and wrote
AuditOperation::Create with a null before-snapshot, and skipped the
update-policy gate entirely. The returned row was correct; the audit
trail and event stream described something that never happened.Nothing changed off the race path: the uncontended insert and the
probe-predicted update emit exactly what they always did, and
UpsertOutcome’s public shape is unchanged.This is deliberately not implemented with RETURNING (xmax = 0),
which classifies correctly but only after the prior row has been
overwritten and is unrecoverable — and is a Postgres storage detail with
no counterpart elsewhere, where ON CONFLICT DO NOTHING … RETURNING is
documented behaviour SQLite mirrors verbatim.SELECT … FOR UPDATE is the price of clean
event / audit semantics without leaning on Postgres xmax — keeping the
rusqlite mirror trivial. Upsert is not a hot read path; callers who need
raw insert/update throughput should use .create(...) / .update(...)
directly.
Policies: both must allow
Upsert evaluates both create and update policies at call time, before the runtime knows which branch will actually fire. This is stricter than “evaluate the path that runs,” but it’s the only choice we can make without leaking row existence to the caller (pre-flighting a read just to pick the policy slot would tell denied callers whether the row exists). In practice this means:- write
@@allow(create, …)and@@allow(update, …)so the intersection of permitted callers is exactly the set you want to be able to upsert - don’t reach for
.upsert(...)on models where create and update audiences are deliberately disjoint — that’s a sign the operation wants to be split into separate create / update routes
@version is bumped, but if_match isn’t honored
Models with @version get the same monotonic guarantee as .update(...):
the update branch emits version = <table>.version + 1 in the same
statement, so concurrent upserts converge to a coherent version number.
if_match is not supported on upsert. The semantics — “update only if
version = N, otherwise insert” — is rarely what callers actually want; if
you really need that conditional, the right shape is an explicit
transaction with find_unique → update.if_match(N). Adding if_match
to the upsert builder is on the deferred list and will require a clear
use case.
.do_nothing()’s policy, event, and concurrency contract
.do_nothing() reuses the same SELECT … FOR UPDATE probe as the DO UPDATE path — the row lock is
what makes “return what the probe found, untouched” safe without a second statement:
- Create policy still gates the insert branch unconditionally, same as
.create()and the DO UPDATE path —.do_nothing()still performs a realINSERTwhen no conflicting row exists. - The update policy is still evaluated against an existing row, even though it’s never mutated.
Skipping this check would let a caller with only create authorization use
.do_nothing()to probe for a row’s existence and read its current contents — exactly the leak the DO UPDATE path’s “both policies must allow” rule already exists to close (see Policies: both must allow above). Denial surfaces the identical"update policy denied this upsert"error either way. - Only the
Insertedbranch emits anything. ACreatedevent and audit entry fire exactly like.create(...)’s.Existingemits neither — the row genuinely didn’t change, so there’s nothing to record. - The insert branch races honestly, not naively. The probe finding no row doesn’t itself lock
anything, so a concurrent transaction can still commit a conflicting row in the gap before this
transaction’s own
INSERT … ON CONFLICT DO NOTHINGruns. When that race is lost, the runtime performs one more locked read and hands back the winning transaction’s row asExisting— never a phantomExistingbuilt from stale data. In the doubly-unlikely case that that row is deleted before the fallback read completes, the call returnsCratestackError::Conflictrather than inventing a result; retry the call.
Soft-deleted rows and the two paths
Models with@@soft_delete treat tombstoned rows as “not present” for
the probe step, and the two upsert paths diverge from there.
Auth-derived defaults are insert-only
Columns marked@default(auth().*) (e.g. ownership_id derived from the
caller’s principal) are excluded from the DO UPDATE clause. They’re
identity bindings, not column values; clobbering them on an update would
turn upsert into “take ownership of any row I name,” which is exactly the
attack we’re not interested in shipping.
The descriptor exposes the exact set of columns the update branch is
allowed to overwrite as ModelDescriptor::upsert_update_columns. Today
the rule is scalar columns − {primary key, @version, @readonly, @server_only, @default(...) }.
Embedded semantics
The on-device (cratestack-rusqlite) path is deliberately thinner:
- no policy enforcement (the embedded backend is single-user and trusts its caller)
- no transactional probe — the upsert is a single statement
- no event outbox or audit log to discriminate
INSERT … ON CONFLICT (<pk-or-columns>) DO UPDATE SET … with the same upsert_update_columns rule, and @version
is bumped via <table>.<col> + 1 so concurrent on-device writers
converge. .on_conflict(ConflictTarget::Columns(&[...])) works here too
— composite-key upsert isn’t a server-only capability, it’s available on
the embedded delegate symmetrically. Use this path when you’re processing
inbound sync messages from a server-of-truth and want each message to be
a self-describing convergence step.
HTTP
Upsert is ORM-only at v1. There is noPUT /<model>/<id> route
generated today; that’s deferred until the precondition story (If-Match,
If-None-Match: *) is wired through the upsert builder. The route shape
when it lands will be canonical REST:
If-* header → either branch is allowed, matching the current ORM
behavior. There is no POST /<model>/upsert and no verb-in-path
alternative; the conflict target lives in the URL.
Comparison with idempotency
IdempotencyLayer and .upsert(...) solve complementary
problems and compose cleanly:
Use both when ingesting from a high-retry producer: the layer protects
against duplicate handler execution, the primitive protects against
duplicate rows even when two distinct requests carry the same payload.