Purchase Document Ingestion

Source-backed module reference for apps/backend/src/purchase-document-ingestion/.

This page covers architecture, domain rules, and the merchant workflow. For extractor contracts, see Purchase Document Extractor Port. For incident handling, see Purchase Document Ingestion Troubleshooting.

PWA screens live at /forms/purchaseDocumentIngestions (PurchaseDocumentsPage and PurchaseDocumentDetailPage).

Intent

Turn supplier invoices (Guatemala FEL XML, optional matching PDF, or PDF-only) into a searchable AP inbox, then optionally materialize a draft purchase. Stock, WAC, and accounts-payable posting still happen on the existing purchase submit path — this module does not move inventory itself.

Two intakes converge on the same draft:

authenticated multipart upload
SendGrid Inbound Parse email to a per-business inbox-{token}@ingest.flowpos.app address

Hexagonal map

domain/
  purchase-document-extractor.port.ts
  purchase-document-ingestion-repository.domain.ts
  purchase-document-ingestion-line-repository.domain.ts
  supplier-product-mapping-repository.domain.ts
  description-key.ts
  posted-purchase-payment-detail.ts
  errors/

application/
  upload-purchase-documents.use-case.ts
  extract-purchase-document.use-case.ts
  ingest-email-attachment.use-case.ts
  resolve-purchase-document-line.use-case.ts
  post-purchase-document.use-case.ts
  retry-*.use-case.ts
  ...

infrastructure/
  extractors/fel-xml-extractor.adapter.ts
  extractors/purchase-document-extractor.registry.ts
  jobs/extract-purchase-document.processor.ts
  jobs/retry-stalled-extractions.processor.ts
  purchase-document-storage.service.ts
  *.repository.ts

interfaces/
  purchase-document-ingestion.controller.ts
  webhooks/purchase-document-email-intake.controller.ts

Layer	Responsibility
Domain	Extracted-document shape, description-key matching, posting payment-detail construction, repository ports, domain errors
Application	Intake grouping/dedup, extraction orchestration, line resolution, posting to a draft purchase, retry/sweep
Infrastructure	FEL XML parse, GCS private-bucket I/O, SHA-256 hashing, BullMQ jobs, Kysely adapters
Interfaces	REST under `/purchase-document-ingestions`; public SendGrid webhook

Do not describe GCS keys, MIME sniffing, or BullMQ job IDs as domain behavior. Those are adapter concerns.

Domain concepts

Ingestion vs document status

Two independent statuses on purchase_document_ingestion:

Field	Values	Meaning
`ingestionStatus`	`received`, `extracted`, `needs_review`, `unparsed`, `failed`	Capture/parse pipeline
`documentStatus`	`draft`, `posted`	Whether a purchase row has been materialized

Only ingestionStatus = extracted can post. needs_review must be cleared first (dismiss remaining flags).

Line treatment

Each purchase_document_ingestion_line is classified as:

inventory — becomes a purchase item; requires a catalog product (and variant when product.hasVariants)
expense — skipped at post time (not written onto the purchase)
ignore — skipped at post time

Posting requires at least one inventory line. Expense-only invoices cannot post through this path.

Description key

normalizeDescriptionKey trims, collapses whitespace, and uppercases the raw line description. Matching is exact on that key — never fuzzy.

When a merchant maps an inventory line, the same key is:

copied to sibling lines on the same invoice that are still unmapped or already inventory
upserted on supplier_product_mapping for later invoices from the same supplier

Recognized DTE types

Intake stores: FACT, FCAM, NCRE, NDEB. Anything else fails loudly (UnsupportedDocumentTypeError → failed).

Only FACT and FCAM may post as a purchase. Credit/debit notes remain labeled drafts.

Workflows

1. Upload → extract

POST /purchase-document-ingestions/upload
  → group files by filename stem
  → reject exact file-hash duplicates
  → attach late-arriving PDF to existing draft (stem = felAuthorization)
  → store objects, create row (ingestionStatus=received)
  → enqueue Bull job extract on queue purchase-document-extraction
  → ExtractPurchaseDocumentUseCase (never in the HTTP path)

Limits (same as data-import per-file cap): 10 MB per file, 50 files per batch. Only .xml and .pdf. PDFs are MIME-sniffed; XML is validated at parse time.

XML + PDF that share a filename stem (extension stripped, lowercased) become one draft. Unrelated stems in the same batch become separate drafts.

2. Extraction outcomes

Outcome	Status	Notes
XML parsed, no flags	`extracted`	Supplier matched by tax ID only; unmatched supplier stays `supplierId = null`
XML parsed, undismissed flags	`needs_review`	Arithmetic mismatch, unverified email sender, or possible duplicate
PDF-only (no XML)	`unparsed`	Terminal, not a failure
Receiver NIT ≠ business tax ID	row deleted	Never appears in the merchant list
Malformed XML / unrecognized DTE	`failed`	No Bull retries — deterministic

Possible-duplicate flags fire when another draft from the same supplier, issued within 3 days, matches document number or total. Merchants can dismiss flags; when none remain, status reverts needs_review → extracted.

Exact file-hash duplicates are rejected at upload (duplicates[].reason = file_hash). Fiscal UUID identity is only known after XML parse; a late PDF whose stem equals an existing felAuthorization attaches to that draft instead of creating a second one.

3. Email intake

POST /webhooks/sendgrid/inbound-purchase-documents  (@IsPublic)
  → extract inbox-{token}@ from the To header
  → resolve business by purchaseDocumentInboxToken
  → reuse UploadPurchaseDocumentsUseCase (source=email)

There is no SendGrid signature header for Inbound Parse. The unguessable 32-hex-char token in the local part is the auth boundary. Unknown tokens and emails with no XML/PDF return HTTP 200 and no-op (never leak whether an address exists). Processing errors are logged and swallowed so SendGrid does not retry-storm.

Unverified senders (From domain does not match a supplier email for that business) still create a draft; extraction adds an unverified_sender review flag.

Inbox address: GET /purchase-document-ingestions/inbox-address?businessId= lazily creates business.purchaseDocumentInboxToken and returns inbox-{token}@ingest.flowpos.app.

4. Resolve lines and post

PATCH  /purchase-document-ingestions/:id                 set supplierId / locationId
PATCH  /purchase-document-ingestions/:id/lines/:lineId   classify + map product
POST   /purchase-document-ingestions/:id/post            create purchase (status=draft)

Posting (PostPurchaseDocumentUseCase):

Requires signed-in db_user_id, extracted status, postable DTE, supplier, location, currency, and ≥1 inventory line.
Resolves each line from the line itself, then same-invoice sibling key, then supplier_product_mapping.
Builds paymentDetail with the built-in accountsPayable payment method (generateElectronicTaxDocument: false, requireSupplierAccount: true).
Calls PurchasesService.createPurchase with status: draft.
Sets documentStatus=posted, materializedEntityType=purchase, materializedEntityId.

Inventory still does not move until a user submits that purchase. Re-posting the same ingestion returns 409.

Public HTTP surface

Base: PurchaseDocumentIngestionController at /purchase-document-ingestions.

Auth: global Firebase AuthGuard + RolesGuard with PolicyResource.Purchase. @ResolveBusinessIdFromTable("purchaseDocumentIngestion") resolves tenant context from the row for :id routes. List/upload/inbox still require businessId query.

Method	Path	Permission	Purpose
`GET`	`/`	Read	Paginated drafts (`search`, `status`, `supplierId`, `taxId`, `documentNumber`, `locationId`, `dateFrom`/`dateTo`)
`GET`	`/inbox-address`	Read	Get or create email intake address
`POST`	`/upload`	Create	Multipart `files` field
`GET`	`/:id`	Read	Header + lines + signed file URLs (15 min). GCS keys never leave the server.
`GET`	`/:id/files/:kind`	Read	Authenticated preview stream (`xml` or `pdf`). XML served as `text/plain`.
`PATCH`	`/:id`	Create	Set `supplierId` and/or `locationId`
`PATCH`	`/:id/lines/:lineId`	Create	Line treatment + product/variant + `conversionFactor`
`POST`	`/:id/post`	Create	Materialize draft purchase
`POST`	`/:id/retry-extraction`	Create	Re-queue a `failed` draft against stored files
`POST`	`/:id/flags/:flagIndex/dismiss`	Create	Dismiss one review flag

Cross-tenant and missing IDs both return 404 Purchase document not found (no existence leak).

Webhook (separate controller, excluded from Swagger):

Method	Path	Auth
`POST`	`/webhooks/sendgrid/inbound-purchase-documents`	`@IsPublic()` + inbox token in `To`

Background jobs

Queue purchase-document-extraction, job name extract, jobId = ingestionId.

Default job options: 3 attempts, exponential backoff from 2000 ms, removeOnComplete: 100, removeOnFail: 500.

Deterministic parse failures are marked failed inside the use case so Bull does not burn retries on XML that will never parse differently.

Stalled sweep: RetryStalledExtractionsProcessor runs @Cron(EVERY_10_MINUTES). Rows still received for more than 10 minutes are claimed with FOR UPDATE SKIP LOCKED. Within 24 hours of receivedAt they are re-enqueued; after 24 hours they become failed with a Spanish timeout message.

Merchant retry: POST :id/retry-extraction only accepts failed (not unparsed / extracted). It resets status to received, removes any leftover Bull job with that jobId (removeOnFail: 500 would otherwise no-op a re-add), and enqueues again. Posted ingestions cannot be retried.

Storage

PurchaseDocumentStorageService uses GCS_PRIVATE_DOCUMENTS_BUCKET — not the public product-photo bucket. Object keys:

purchase-documents/{businessId}/{uuid}/{originalFilename}

Upload metadata sets contentDisposition: attachment. In-app preview uses the authenticated stream endpoint; downloads use the short-lived signed URL from GET :id.

If the bucket env is unset, uploads throw: GCS_PRIVATE_DOCUMENTS_BUCKET is not configured.

packages/global/enums/purchase-document-ingestion.enums.ts
apps/backend/src/purchases/ — draft purchase created on post
apps/frontend-pwa/src/pages/purchase-documents/
Specs: specs/048-purchase-document-ingestion/, specs/049-purchase-document-posting/

Intent​

Hexagonal map​

Domain concepts​

Ingestion vs document status​

Line treatment​

Description key​

Recognized DTE types​

Workflows​

1. Upload → extract​

2. Extraction outcomes​

3. Email intake​

4. Resolve lines and post​

Public HTTP surface​

Background jobs​

Storage​

Related codepaths​