Scrypath.Loader (scrypath v0.3.17)

Copy Markdown View Source

Loads the current source records for search document ids.

Oban sync jobs carry document ids, not document data. When a job runs, it loads the records as they are now, so a retried or delayed job never sends an older copy over newer index data.

By default the loader queries the schema by its document_id field with the configured :repo, then applies the schema's declared preload::

use Scrypath,
  fields: [:name, :category_name],
  preload: [:category]

A schema that needs full control, for example tenant-scoped repos, soft deletes, or document ids that are not a column, defines search_records/2. It receives the document ids and a context map with :repo, and returns the records that still exist and should be searchable:

def search_records(ids, %{repo: repo}) do
  from(p in __MODULE__, where: p.id in ^ids, preload: :category)
  |> repo.all(skip_tenant_id: true)
end

Ids with no returned record are removed from the index.

Summary

Functions

check_loadable(schema_module, config)

Checks that records for schema_module can be loaded by document id.

custom_loader?(schema_module)

Whether the schema defines its own search_records/2 loader.

ensure_loadable!(schema_module, config)

Raises when records for schema_module cannot be loaded by document id.

load(schema_module, document_ids, config)

Returns {:ok, records, missing_ids} for the given document ids.

Functions

check_loadable(schema_module, config)

@spec check_loadable(
  module(),
  keyword()
) :: :ok | {:error, atom()}

Checks that records for schema_module can be loaded by document id.

Returns {:error, :search_records_required} for schemas with custom document ids and no search_records/2, and {:error, :repo_required} when no :repo is configured.

custom_loader?(schema_module)

@spec custom_loader?(module()) :: boolean()

Whether the schema defines its own search_records/2 loader.

ensure_loadable!(schema_module, config)

@spec ensure_loadable!(
  module(),
  keyword()
) :: :ok

Raises when records for schema_module cannot be loaded by document id.

Called before an Oban job is queued so a missing repo or an unmappable custom document id fails at the call site instead of inside the worker.

load(schema_module, document_ids, config)

@spec load(module(), [term()], keyword()) ::
  {:ok, [struct()], [term()]} | {:error, term()}

Returns {:ok, records, missing_ids} for the given document ids.

missing_ids keeps the caller's id values for documents whose records no longer exist.