.do
Complete Examples

Complete Invoice Processing Workflow Service

End-to-end implementation of a multi-step invoice processing workflow with OCR, validation, approval, and payment

This comprehensive guide demonstrates how to build a production-ready invoice processing workflow service using the .do platform. The service automates the entire accounts payable process from invoice receipt through payment, incorporating OCR extraction, intelligent validation, multi-level approval workflows, and payment processing integrations.

Service Overview

Modern businesses process thousands of invoices monthly, each requiring multiple manual steps: data extraction, validation against purchase orders, approval routing, and payment execution. This workflow automation service eliminates these manual touchpoints while maintaining necessary controls and audit trails.

Our invoice processing workflow implements a sophisticated state machine that orchestrates multiple stages: OCR extraction using AI vision models, intelligent validation against business rules and purchase order data, conditional approval routing based on amount thresholds and department budgets, payment processing through integrated financial systems, and comprehensive error recovery mechanisms.

The service targets three primary use cases: accounts payable automation for mid-market and enterprise businesses processing 500+ invoices monthly, expense management systems requiring policy enforcement and approval workflows, and supplier payment portals needing automated invoice-to-payment cycles. The workflow handles both simple straight-through processing for standard invoices and complex multi-approver scenarios for exceptional cases.

Key features include intelligent OCR extraction with 98%+ accuracy using state-of-the-art vision models, flexible approval routing supporting unlimited approval levels with configurable thresholds, real-time integration with ERP systems like QuickBooks and NetSuite, automatic purchase order matching with three-way reconciliation, duplicate invoice detection preventing payment errors, and comprehensive audit logging tracking every state transition and data modification.

The service implements human-in-the-loop patterns for exception handling, allowing approvers to review flagged invoices through web interfaces or email notifications. Automated retry logic handles transient failures in external integrations, while sophisticated error recovery mechanisms ensure no invoice is lost or double-processed. Performance monitoring tracks processing times, approval latencies, and success rates across the entire workflow.

The platform's pricing model aligns with customer value, charging per successfully processed invoice on a sliding scale from $2 for simple auto-approved invoices to $8 for complex multi-approval workflows requiring extensive validation and multiple system integrations. This usage-based pricing ensures customers only pay for actual processing while the service automatically scales to handle volume fluctuations.

Service Definition

The workflow service begins with a comprehensive service definition that establishes the invoice processing capabilities, pricing tiers, state machine configuration, and integration requirements. This definition serves as the contract between the service and its consumers.

Basic Service Configuration

import $, { ai, db, on, send } from 'sdk.do'

// Define the invoice processing workflow service
const invoiceWorkflowService = await $.Service.create({
  name: 'invoice-processing-workflow',
  type: $.ServiceType.WorkflowAutomation,
  description: 'End-to-end invoice processing with OCR, validation, approval, and payment',
  version: '1.0.0',

  // Pricing model based on invoice complexity
  pricing: {
    model: $.PricingModel.Usage,
    tiers: [
      {
        name: 'simple',
        description: 'Auto-approved invoices under $500',
        basePrice: 2.0,
        currency: 'USD',
        unit: 'invoice',
        conditions: {
          amount: { max: 500 },
          requiresApproval: false,
          hasPurchaseOrder: true,
        },
      },
      {
        name: 'standard',
        description: 'Single-approval invoices $500-$5,000',
        basePrice: 4.0,
        currency: 'USD',
        unit: 'invoice',
        conditions: {
          amount: { min: 500, max: 5000 },
          approvalLevels: 1,
        },
      },
      {
        name: 'complex',
        description: 'Multi-approval invoices over $5,000',
        basePrice: 8.0,
        currency: 'USD',
        unit: 'invoice',
        conditions: {
          amount: { min: 5000 },
          approvalLevels: { min: 2 },
        },
      },
    ],
  },

  // Service capabilities and features
  capabilities: [
    'ocr-extraction',
    'ai-validation',
    'approval-routing',
    'payment-processing',
    'po-matching',
    'duplicate-detection',
    'multi-currency',
    'audit-logging',
  ],

  // External system integrations
  integrations: [
    {
      name: 'stripe',
      type: 'payment-processor',
      required: true,
      credentials: ['api_key'],
    },
    {
      name: 'quickbooks',
      type: 'erp-system',
      required: false,
      credentials: ['oauth_token'],
    },
    {
      name: 'sendgrid',
      type: 'email-service',
      required: true,
      credentials: ['api_key'],
    },
  ],

  // Service-level agreements
  sla: {
    availability: 99.9,
    responseTime: {
      extraction: 30, // seconds
      validation: 10,
      approvalNotification: 60,
      payment: 120,
    },
    successRate: 98.5,
  },
})

// Store service configuration
await db.services.insert(invoiceWorkflowService)

State Machine Definition

The workflow implements a comprehensive state machine that models the complete invoice lifecycle from initial receipt through final payment or rejection. Each state represents a distinct processing stage with defined transitions and actions.

// Define the workflow state machine
const workflowStateMachine = {
  name: 'invoice-processing-workflow',
  initialState: 'received',

  states: {
    // Initial state when invoice is received
    received: {
      description: 'Invoice uploaded and queued for processing',
      type: 'initial',
      actions: ['validateFormat', 'assignWorkflowId'],
      transitions: ['extracting', 'invalid'],
    },

    // OCR extraction in progress
    extracting: {
      description: 'Extracting data from invoice using OCR',
      type: 'processing',
      actions: ['performOCR', 'structureData'],
      transitions: ['validating', 'extraction_failed'],
      timeout: 60, // seconds
      retryPolicy: {
        maxAttempts: 3,
        backoff: 'exponential',
        initialDelay: 5,
      },
    },

    // Data validation stage
    validating: {
      description: 'Validating extracted data against business rules',
      type: 'processing',
      actions: ['validateFields', 'checkDuplicates', 'matchPurchaseOrder'],
      transitions: ['pending_approval', 'auto_approved', 'validation_failed'],
      rules: [
        {
          name: 'required_fields',
          check: 'vendor_name && invoice_number && amount && date',
        },
        {
          name: 'duplicate_check',
          check: 'not exists in processed_invoices',
        },
        {
          name: 'amount_limit',
          check: 'amount <= company.max_invoice_amount',
        },
      ],
    },

    // Awaiting approval from authorized personnel
    pending_approval: {
      description: 'Waiting for approval from assigned approvers',
      type: 'waiting',
      actions: ['routeToApprovers', 'sendNotifications', 'startSLAClock'],
      transitions: ['approved', 'rejected', 'approval_timeout'],
      timeout: 172800, // 48 hours
      escalationPolicy: {
        levels: [
          { after: 86400, notifyManager: true },
          { after: 129600, notifyDirector: true },
        ],
      },
    },

    // Auto-approved for low-value invoices
    auto_approved: {
      description: 'Automatically approved based on business rules',
      type: 'decision',
      actions: ['logAutoApproval', 'applyApprovalReason'],
      transitions: ['processing_payment'],
      conditions: {
        amount: { max: 500 },
        hasValidPO: true,
        vendorTrusted: true,
      },
    },

    // Manual approval granted
    approved: {
      description: 'Invoice approved by authorized personnel',
      type: 'decision',
      actions: ['recordApproval', 'notifyRequestor'],
      transitions: ['processing_payment'],
    },

    // Processing payment through financial systems
    processing_payment: {
      description: 'Initiating payment through integrated systems',
      type: 'processing',
      actions: ['createPayment', 'updateERP', 'notifyVendor'],
      transitions: ['paid', 'payment_failed'],
      timeout: 300,
      retryPolicy: {
        maxAttempts: 5,
        backoff: 'exponential',
        initialDelay: 10,
      },
    },

    // Successfully completed
    paid: {
      description: 'Invoice successfully paid',
      type: 'final',
      actions: ['recordPayment', 'archiveInvoice', 'updateMetrics'],
      transitions: [],
    },

    // Rejected by approver
    rejected: {
      description: 'Invoice rejected during approval',
      type: 'final',
      actions: ['recordRejection', 'notifySubmitter', 'archiveInvoice'],
      transitions: [],
    },

    // Various failure states
    invalid: {
      description: 'Invalid invoice format or corrupted file',
      type: 'error',
      actions: ['logError', 'notifySubmitter'],
      transitions: [],
    },

    extraction_failed: {
      description: 'OCR extraction failed after retries',
      type: 'error',
      actions: ['logError', 'routeToManualReview'],
      transitions: ['manual_review'],
    },

    validation_failed: {
      description: 'Data validation failed',
      type: 'error',
      actions: ['logValidationErrors', 'notifySubmitter'],
      transitions: ['manual_review'],
    },

    payment_failed: {
      description: 'Payment processing failed',
      type: 'error',
      actions: ['logPaymentError', 'notifyFinance'],
      transitions: ['manual_review', 'processing_payment'],
    },

    approval_timeout: {
      description: 'Approval not received within SLA',
      type: 'error',
      actions: ['escalateToManager', 'extendTimeout'],
      transitions: ['pending_approval', 'rejected'],
    },

    manual_review: {
      description: 'Requires manual intervention',
      type: 'waiting',
      actions: ['createTicket', 'notifyOperations'],
      transitions: ['validating', 'rejected'],
    },
  },

  // Global event handlers
  events: {
    onStateChange: 'logStateTransition',
    onError: 'handleWorkflowError',
    onTimeout: 'handleTimeout',
  },

  // Workflow metadata storage
  metadata: {
    version: '1.0.0',
    createdAt: new Date().toISOString(),
    trackMetrics: true,
    auditEnabled: true,
  },
}

// Register the state machine
await $.StateMachine.register(workflowStateMachine)

Implementation

The implementation section covers the complete workflow logic, including OCR extraction, validation rules, approval routing, payment processing, and comprehensive error handling. Each component is designed for reliability, performance, and maintainability.

State Machine Implementation

import $, { ai, db, on, send } from 'sdk.do'

// Core state machine executor
class InvoiceWorkflowStateMachine {
  private workflowId: string
  private currentState: string
  private context: Record<string, any>
  private history: Array<{ state: string; timestamp: Date; data?: any }>

  constructor(workflowId: string, initialContext: Record<string, any> = {}) {
    this.workflowId = workflowId
    this.currentState = 'received'
    this.context = initialContext
    this.history = []
  }

  // Execute state transition
  async transition(targetState: string, data?: any): Promise<boolean> {
    const stateDef = workflowStateMachine.states[this.currentState]

    // Validate transition is allowed
    if (!stateDef.transitions.includes(targetState)) {
      throw new Error(`Invalid transition from ${this.currentState} to ${targetState}`)
    }

    // Record transition in history
    this.history.push({
      state: this.currentState,
      timestamp: new Date(),
      data,
    })

    // Execute exit actions for current state
    await this.executeStateActions(stateDef, 'exit')

    // Update state
    const previousState = this.currentState
    this.currentState = targetState

    // Execute entry actions for new state
    const newStateDef = workflowStateMachine.states[targetState]
    await this.executeStateActions(newStateDef, 'entry')

    // Persist state change
    await this.persistState()

    // Emit state change event
    await send(
      $.Event.create({
        type: 'workflow.state.changed',
        subject: this.workflowId,
        data: {
          from: previousState,
          to: targetState,
          timestamp: new Date().toISOString(),
          context: this.context,
        },
      })
    )

    // Handle final states
    if (newStateDef.type === 'final') {
      await this.completeWorkflow(targetState)
    }

    return true
  }

  // Execute state-specific actions
  private async executeStateActions(stateDef: any, phase: 'entry' | 'exit'): Promise<void> {
    const actions = stateDef.actions || []

    for (const action of actions) {
      try {
        await this.executeAction(action)
      } catch (error) {
        console.error(`Action ${action} failed:`, error)

        // Handle critical failures
        if (stateDef.type === 'processing') {
          await this.handleActionFailure(action, error)
        }
      }
    }
  }

  // Execute individual action
  private async executeAction(action: string): Promise<void> {
    switch (action) {
      case 'validateFormat':
        await this.validateInvoiceFormat()
        break
      case 'assignWorkflowId':
        this.context.workflowId = this.workflowId
        break
      case 'performOCR':
        this.context.extractedData = await this.performOCR()
        break
      case 'structureData':
        this.context.structuredData = await this.structureExtractedData()
        break
      case 'validateFields':
        await this.validateRequiredFields()
        break
      case 'checkDuplicates':
        await this.checkForDuplicates()
        break
      case 'matchPurchaseOrder':
        await this.matchPurchaseOrder()
        break
      case 'routeToApprovers':
        await this.routeToApprovers()
        break
      case 'sendNotifications':
        await this.sendApprovalNotifications()
        break
      case 'createPayment':
        await this.createPayment()
        break
      case 'updateERP':
        await this.updateERPSystem()
        break
      case 'recordPayment':
        await this.recordPaymentCompletion()
        break
      case 'logError':
        await this.logWorkflowError()
        break
      default:
        console.warn(`Unknown action: ${action}`)
    }
  }

  // Validate invoice format
  private async validateInvoiceFormat(): Promise<void> {
    const invoice = this.context.invoice

    if (!invoice.fileUrl) {
      throw new Error('Invoice file URL is required')
    }

    // Check file type
    const validTypes = ['application/pdf', 'image/png', 'image/jpeg']
    if (!validTypes.includes(invoice.mimeType)) {
      throw new Error(`Invalid file type: ${invoice.mimeType}`)
    }

    // Check file size (max 10MB)
    if (invoice.fileSize > 10 * 1024 * 1024) {
      throw new Error('Invoice file too large (max 10MB)')
    }
  }

  // Check if should auto-approve
  async shouldAutoApprove(): Promise<boolean> {
    const { amount, vendor, hasPurchaseOrder } = this.context.structuredData

    // Auto-approve if all conditions met
    return amount <= 500 && hasPurchaseOrder && vendor.trusted === true && !this.context.validationWarnings?.length
  }

  // Persist current state to database
  private async persistState(): Promise<void> {
    await db.workflows.update(this.workflowId, {
      currentState: this.currentState,
      context: this.context,
      history: this.history,
      updatedAt: new Date(),
    })
  }

  // Complete workflow execution
  private async completeWorkflow(finalState: string): Promise<void> {
    const success = ['paid', 'auto_approved'].includes(finalState)

    await db.workflows.update(this.workflowId, {
      status: 'completed',
      completedAt: new Date(),
      success,
      finalState,
    })

    // Update metrics
    await this.recordWorkflowMetrics(success)
  }

  // Handle action failures with retry logic
  private async handleActionFailure(action: string, error: any): Promise<void> {
    const retryCount = this.context.retries?.[action] || 0
    const maxRetries = 3

    if (retryCount < maxRetries) {
      // Increment retry counter
      this.context.retries = {
        ...this.context.retries,
        [action]: retryCount + 1,
      }

      // Exponential backoff
      const delay = Math.pow(2, retryCount) * 1000
      await new Promise((resolve) => setTimeout(resolve, delay))

      // Retry action
      await this.executeAction(action)
    } else {
      // Max retries exceeded, transition to error state
      await this.transition('extraction_failed', { error: error.message })
    }
  }

  // Record workflow metrics
  private async recordWorkflowMetrics(success: boolean): Promise<void> {
    const duration = Date.now() - this.context.startTime

    await $.Metric.record({
      name: 'invoice_workflow_completed',
      value: 1,
      tags: {
        success: success.toString(),
        finalState: this.currentState,
        duration: Math.round(duration / 1000),
      },
    })
  }
}

OCR Extraction Implementation

// OCR extraction using AI vision models
async function performOCR(invoiceFile: { url: string; mimeType: string }): Promise<any> {
  // Use AI vision model for OCR
  const extraction = await ai.vision.extract({
    imageUrl: invoiceFile.url,
    model: 'gpt-5-vision',
    schema: {
      vendorName: { type: 'string', required: true },
      vendorAddress: { type: 'string' },
      vendorTaxId: { type: 'string' },
      invoiceNumber: { type: 'string', required: true },
      invoiceDate: { type: 'date', required: true },
      dueDate: { type: 'date' },
      purchaseOrderNumber: { type: 'string' },
      lineItems: {
        type: 'array',
        items: {
          description: { type: 'string', required: true },
          quantity: { type: 'number', required: true },
          unitPrice: { type: 'number', required: true },
          amount: { type: 'number', required: true },
          taxRate: { type: 'number' },
        },
      },
      subtotal: { type: 'number', required: true },
      taxAmount: { type: 'number' },
      totalAmount: { type: 'number', required: true },
      currency: { type: 'string', default: 'USD' },
      paymentTerms: { type: 'string' },
      bankDetails: {
        accountNumber: { type: 'string' },
        routingNumber: { type: 'string' },
        swiftCode: { type: 'string' },
      },
    },
    prompt: `
      Extract all invoice details from this document.
      Pay special attention to:
      - Exact invoice number (critical for duplicate detection)
      - Line items with quantities and prices
      - Total amounts including tax
      - Payment terms and bank details

      Return structured data matching the schema.
    `,
    confidence: 0.85, // Minimum confidence threshold
  })

  // Validate extraction confidence
  if (extraction.confidence < 0.85) {
    throw new Error(`OCR extraction confidence too low: ${extraction.confidence}`)
  }

  // Apply post-processing corrections
  const corrected = await applyOCRCorrections(extraction.data)

  return {
    raw: extraction.data,
    corrected,
    confidence: extraction.confidence,
    model: extraction.model,
    extractedAt: new Date().toISOString(),
  }
}

// Apply common OCR corrections
async function applyOCRCorrections(data: any): Promise<any> {
  const corrected = { ...data }

  // Fix common OCR errors
  if (corrected.invoiceNumber) {
    // Remove common OCR mistakes (O vs 0, I vs 1)
    corrected.invoiceNumber = corrected.invoiceNumber.replace(/O/g, '0').replace(/I/g, '1').toUpperCase()
  }

  // Validate and parse dates
  if (corrected.invoiceDate) {
    corrected.invoiceDate = new Date(corrected.invoiceDate).toISOString()
  }

  // Ensure amounts are numbers
  if (typeof corrected.totalAmount === 'string') {
    corrected.totalAmount = parseFloat(corrected.totalAmount.replace(/[^0-9.]/g, ''))
  }

  // Calculate line item totals
  if (corrected.lineItems) {
    corrected.lineItems = corrected.lineItems.map((item: any) => ({
      ...item,
      amount: item.quantity * item.unitPrice,
    }))
  }

  // Verify subtotal matches line items
  const calculatedSubtotal = corrected.lineItems?.reduce((sum: number, item: any) => sum + item.amount, 0)

  if (Math.abs(calculatedSubtotal - corrected.subtotal) > 0.01) {
    corrected.warnings = corrected.warnings || []
    corrected.warnings.push({
      field: 'subtotal',
      message: 'Subtotal does not match line items',
      expected: calculatedSubtotal,
      actual: corrected.subtotal,
    })
  }

  return corrected
}

// Structure extracted data into standard format
async function structureExtractedData(extractedData: any): Promise<any> {
  return {
    vendor: {
      name: extractedData.vendorName,
      address: extractedData.vendorAddress,
      taxId: extractedData.vendorTaxId,
      trusted: await isVendorTrusted(extractedData.vendorName),
    },
    invoice: {
      number: extractedData.invoiceNumber,
      date: extractedData.invoiceDate,
      dueDate: extractedData.dueDate,
      purchaseOrder: extractedData.purchaseOrderNumber,
    },
    lineItems: extractedData.lineItems,
    amounts: {
      subtotal: extractedData.subtotal,
      tax: extractedData.taxAmount,
      total: extractedData.totalAmount,
      currency: extractedData.currency,
    },
    payment: {
      terms: extractedData.paymentTerms,
      bankDetails: extractedData.bankDetails,
    },
    metadata: {
      confidence: extractedData.confidence,
      extractedAt: extractedData.extractedAt,
      warnings: extractedData.warnings || [],
    },
  }
}

// Check if vendor is trusted
async function isVendorTrusted(vendorName: string): Promise<boolean> {
  const vendor = await db.vendors.findOne({
    name: vendorName,
  })

  return vendor?.trusted === true && vendor?.successfulPayments >= 5
}

Data Validation Implementation

// Comprehensive invoice validation
async function validateInvoiceData(workflow: InvoiceWorkflowStateMachine): Promise<void> {
  const data = workflow.context.structuredData
  const errors: string[] = []
  const warnings: string[] = []

  // Required field validation
  if (!data.vendor?.name) {
    errors.push('Vendor name is required')
  }

  if (!data.invoice?.number) {
    errors.push('Invoice number is required')
  }

  if (!data.invoice?.date) {
    errors.push('Invoice date is required')
  }

  if (!data.amounts?.total || data.amounts.total <= 0) {
    errors.push('Invalid invoice total amount')
  }

  // Business rule validations
  if (data.amounts.total > 100000) {
    warnings.push('Invoice amount exceeds $100,000 - requires executive approval')
  }

  // Date validations
  const invoiceDate = new Date(data.invoice.date)
  const today = new Date()
  const daysOld = Math.floor((today.getTime() - invoiceDate.getTime()) / (1000 * 60 * 60 * 24))

  if (daysOld > 90) {
    warnings.push(`Invoice is ${daysOld} days old - may be stale`)
  }

  if (invoiceDate > today) {
    errors.push('Invoice date cannot be in the future')
  }

  // Duplicate check
  const duplicate = await db.invoices.findOne({
    vendorName: data.vendor.name,
    invoiceNumber: data.invoice.number,
    status: { $in: ['processing', 'paid'] },
  })

  if (duplicate) {
    errors.push(`Duplicate invoice detected: ${data.invoice.number} already processed`)
  }

  // Purchase order matching
  if (data.invoice.purchaseOrder) {
    const po = await db.purchaseOrders.findOne({
      number: data.invoice.purchaseOrder,
      vendor: data.vendor.name,
    })

    if (!po) {
      warnings.push(`Purchase order ${data.invoice.purchaseOrder} not found`)
    } else {
      // Three-way match: PO vs Invoice vs Receipt
      const match = await performThreeWayMatch(data, po)

      if (!match.success) {
        warnings.push(...match.warnings)

        if (match.critical) {
          errors.push(...match.errors)
        }
      }
    }
  } else if (data.amounts.total > 1000) {
    warnings.push('Invoice over $1,000 missing purchase order')
  }

  // Line item validation
  if (data.lineItems?.length) {
    const calculatedSubtotal = data.lineItems.reduce((sum: number, item: any) => sum + item.amount, 0)

    const discrepancy = Math.abs(calculatedSubtotal - data.amounts.subtotal)

    if (discrepancy > 0.01) {
      warnings.push(`Line items total ($${calculatedSubtotal}) does not match subtotal ($${data.amounts.subtotal})`)
    }
  }

  // Tax calculation validation
  if (data.amounts.tax) {
    const expectedTax = data.amounts.subtotal * 0.0875 // Example 8.75% rate
    const taxDiscrepancy = Math.abs(data.amounts.tax - expectedTax)

    if (taxDiscrepancy > data.amounts.subtotal * 0.02) {
      warnings.push('Tax amount appears incorrect')
    }
  }

  // Store validation results
  workflow.context.validation = {
    errors,
    warnings,
    valid: errors.length === 0,
    validatedAt: new Date().toISOString(),
  }

  // Throw error if validation failed
  if (errors.length > 0) {
    throw new Error(`Invoice validation failed: ${errors.join(', ')}`)
  }
}

// Perform three-way match between PO, Invoice, and Receipt
async function performThreeWayMatch(
  invoiceData: any,
  purchaseOrder: any
): Promise<{
  success: boolean
  warnings: string[]
  errors: string[]
  critical: boolean
}> {
  const warnings: string[] = []
  const errors: string[] = []
  let critical = false

  // Match vendor
  if (invoiceData.vendor.name !== purchaseOrder.vendor) {
    errors.push('Invoice vendor does not match purchase order')
    critical = true
  }

  // Match amounts
  const amountDiscrepancy = Math.abs(invoiceData.amounts.total - purchaseOrder.amount)
  const percentageDiscrepancy = (amountDiscrepancy / purchaseOrder.amount) * 100

  if (percentageDiscrepancy > 10) {
    errors.push(`Invoice amount ($${invoiceData.amounts.total}) differs from PO amount ($${purchaseOrder.amount}) by ${percentageDiscrepancy.toFixed(1)}%`)
    critical = true
  } else if (percentageDiscrepancy > 5) {
    warnings.push(`Invoice amount differs from PO amount by ${percentageDiscrepancy.toFixed(1)}%`)
  }

  // Match line items
  if (invoiceData.lineItems?.length && purchaseOrder.items?.length) {
    const itemMatches = invoiceData.lineItems.map((invItem: any) => {
      const poItem = purchaseOrder.items.find((item: any) => item.description === invItem.description)

      if (!poItem) {
        return {
          matched: false,
          item: invItem.description,
          reason: 'Not found in PO',
        }
      }

      if (Math.abs(invItem.quantity - poItem.quantity) > 0) {
        return {
          matched: false,
          item: invItem.description,
          reason: `Quantity mismatch: Invoice ${invItem.quantity} vs PO ${poItem.quantity}`,
        }
      }

      return { matched: true, item: invItem.description }
    })

    const unmatchedItems = itemMatches.filter((m: any) => !m.matched)

    if (unmatchedItems.length > 0) {
      warnings.push(`${unmatchedItems.length} line items do not match PO: ${unmatchedItems.map((i: any) => i.reason).join(', ')}`)
    }
  }

  // Check if goods received
  if (purchaseOrder.receiptRequired && !purchaseOrder.receivedDate) {
    warnings.push('Goods receipt not recorded for this purchase order')
  }

  return {
    success: errors.length === 0,
    warnings,
    errors,
    critical,
  }
}

Approval Workflow Implementation

// Route invoice to appropriate approvers
async function routeToApprovers(workflow: InvoiceWorkflowStateMachine): Promise<void> {
  const data = workflow.context.structuredData
  const amount = data.amounts.total

  // Determine approval levels based on amount
  const approvalLevels = []

  // Level 1: Department manager (all invoices over $500)
  if (amount > 500) {
    approvalLevels.push({
      level: 1,
      role: 'department_manager',
      approvers: await findDepartmentApprovers(data.invoice.department),
    })
  }

  // Level 2: Finance director ($5,000+)
  if (amount > 5000) {
    approvalLevels.push({
      level: 2,
      role: 'finance_director',
      approvers: await findApproversByRole('finance_director'),
    })
  }

  // Level 3: CFO ($25,000+)
  if (amount > 25000) {
    approvalLevels.push({
      level: 3,
      role: 'cfo',
      approvers: await findApproversByRole('cfo'),
    })
  }

  // Level 4: CEO ($100,000+)
  if (amount > 100000) {
    approvalLevels.push({
      level: 4,
      role: 'ceo',
      approvers: await findApproversByRole('ceo'),
    })
  }

  // Store approval requirements
  workflow.context.approvals = {
    levels: approvalLevels,
    currentLevel: 0,
    completed: [],
    pending: approvalLevels[0],
  }

  // Send notifications to first level approvers
  await sendApprovalNotifications(workflow)
}

// Send approval request notifications
async function sendApprovalNotifications(workflow: InvoiceWorkflowStateMachine): Promise<void> {
  const { approvals, structuredData } = workflow.context
  const currentLevel = approvals.levels[approvals.currentLevel]

  if (!currentLevel) {
    return // No more approvals needed
  }

  // Generate approval link
  const approvalUrl = `https://platform.do/approvals/${workflow.workflowId}`

  // Send email to each approver
  for (const approver of currentLevel.approvers) {
    await send(
      $.Email.create({
        to: approver.email,
        subject: `Invoice Approval Required: ${structuredData.vendor.name} - $${structuredData.amounts.total}`,
        template: 'invoice-approval-request',
        data: {
          approverName: approver.name,
          vendorName: structuredData.vendor.name,
          invoiceNumber: structuredData.invoice.number,
          invoiceDate: structuredData.invoice.date,
          amount: structuredData.amounts.total,
          currency: structuredData.amounts.currency,
          lineItems: structuredData.lineItems,
          approvalLevel: currentLevel.level,
          approvalUrl,
          dueDate: new Date(Date.now() + 48 * 60 * 60 * 1000).toISOString(),
        },
      })
    )
  }

  // Create in-app notifications
  for (const approver of currentLevel.approvers) {
    await db.notifications.insert({
      userId: approver.id,
      type: 'approval_required',
      title: 'Invoice Approval Required',
      message: `${structuredData.vendor.name} invoice for $${structuredData.amounts.total} requires your approval`,
      link: approvalUrl,
      priority: structuredData.amounts.total > 10000 ? 'high' : 'normal',
      createdAt: new Date(),
    })
  }
}

// Handle approval decision
async function handleApprovalDecision(workflowId: string, approverId: string, decision: 'approve' | 'reject', comments?: string): Promise<void> {
  const workflowData = await db.workflows.findOne({ id: workflowId })
  const workflow = new InvoiceWorkflowStateMachine(workflowId, workflowData.context)

  // Record approval decision
  const approval = {
    approverId,
    decision,
    comments,
    timestamp: new Date().toISOString(),
    level: workflow.context.approvals.currentLevel,
  }

  workflow.context.approvals.completed.push(approval)

  if (decision === 'reject') {
    // Invoice rejected - transition to rejected state
    await workflow.transition('rejected', { rejection: approval })

    // Notify submitter
    await send(
      $.Email.create({
        to: workflow.context.submitterEmail,
        subject: `Invoice Rejected: ${workflow.context.structuredData.invoice.number}`,
        template: 'invoice-rejected',
        data: {
          invoiceNumber: workflow.context.structuredData.invoice.number,
          rejectionReason: comments,
          rejectedBy: approverId,
          rejectedAt: approval.timestamp,
        },
      })
    )

    return
  }

  // Check if more approvals needed
  const nextLevel = workflow.context.approvals.currentLevel + 1

  if (nextLevel < workflow.context.approvals.levels.length) {
    // More approvals required
    workflow.context.approvals.currentLevel = nextLevel
    workflow.context.approvals.pending = workflow.context.approvals.levels[nextLevel]

    await workflow.persistState()
    await sendApprovalNotifications(workflow)
  } else {
    // All approvals complete - transition to payment processing
    await workflow.transition('approved', { approvals: workflow.context.approvals })
  }
}

// Find approvers by role
async function findApproversByRole(role: string): Promise<any[]> {
  return await db.users.find({
    roles: { $contains: role },
    active: true,
    approvalEnabled: true,
  })
}

// Find department-specific approvers
async function findDepartmentApprovers(department: string): Promise<any[]> {
  return await db.users.find({
    department,
    roles: { $contains: 'approver' },
    active: true,
  })
}

Payment Processing Implementation

// Process payment through integrated systems
async function createPayment(workflow: InvoiceWorkflowStateMachine): Promise<void> {
  const { structuredData } = workflow.context

  // Create payment record
  const payment = {
    invoiceId: workflow.workflowId,
    vendor: structuredData.vendor.name,
    amount: structuredData.amounts.total,
    currency: structuredData.amounts.currency,
    method: determinePaymentMethod(structuredData),
    scheduledDate: calculatePaymentDate(structuredData.invoice.dueDate),
    status: 'pending',
  }

  // Process through Stripe
  if (payment.method === 'ach' || payment.method === 'wire') {
    payment.transactionId = await processStripePayment(payment)
  }

  // Update ERP system
  await updateERPSystem(workflow)

  // Store payment record
  workflow.context.payment = payment
  await db.payments.insert(payment)

  // Transition to paid state
  await workflow.transition('paid', { payment })
}

// Process payment through Stripe
async function processStripePayment(payment: any): Promise<string> {
  const stripe = await $.Integration.get('stripe')

  // Create payout to vendor
  const payout = await stripe.payouts.create({
    amount: Math.round(payment.amount * 100), // Convert to cents
    currency: payment.currency.toLowerCase(),
    method: payment.method === 'ach' ? 'instant' : 'standard',
    metadata: {
      invoiceId: payment.invoiceId,
      vendor: payment.vendor,
    },
  })

  return payout.id
}

// Update ERP system with invoice and payment
async function updateERPSystem(workflow: InvoiceWorkflowStateMachine): Promise<void> {
  const quickbooks = await $.Integration.get('quickbooks')

  if (!quickbooks) {
    console.warn('QuickBooks integration not configured')
    return
  }

  const { structuredData } = workflow.context

  // Create vendor if not exists
  let vendor = await quickbooks.vendors.findByName(structuredData.vendor.name)

  if (!vendor) {
    vendor = await quickbooks.vendors.create({
      displayName: structuredData.vendor.name,
      companyName: structuredData.vendor.name,
      taxIdentifier: structuredData.vendor.taxId,
    })
  }

  // Create bill (invoice) in QuickBooks
  const bill = await quickbooks.bills.create({
    vendorRef: { value: vendor.id },
    txnDate: structuredData.invoice.date,
    dueDate: structuredData.invoice.dueDate,
    docNumber: structuredData.invoice.number,
    line: structuredData.lineItems.map((item: any) => ({
      description: item.description,
      amount: item.amount,
      detailType: 'AccountBasedExpenseLineDetail',
      accountBasedExpenseLineDetail: {
        accountRef: { value: '1' }, // Expense account
      },
    })),
    totalAmt: structuredData.amounts.total,
  })

  // Create bill payment
  const billPayment = await quickbooks.billPayments.create({
    vendorRef: { value: vendor.id },
    payType: 'Check',
    totalAmt: structuredData.amounts.total,
    line: [
      {
        amount: structuredData.amounts.total,
        linkedTxn: [
          {
            txnId: bill.id,
            txnType: 'Bill',
          },
        ],
      },
    ],
  })

  workflow.context.erpRecords = {
    vendorId: vendor.id,
    billId: bill.id,
    paymentId: billPayment.id,
  }
}

// Determine payment method based on invoice data
function determinePaymentMethod(data: any): string {
  // Default to ACH for domestic payments
  if (data.amounts.currency === 'USD' && data.payment.bankDetails?.routingNumber) {
    return 'ach'
  }

  // International payments use wire
  if (data.payment.bankDetails?.swiftCode) {
    return 'wire'
  }

  // Fallback to check
  return 'check'
}

// Calculate payment date based on terms
function calculatePaymentDate(dueDate: string): string {
  const due = new Date(dueDate)
  const today = new Date()

  // Pay early if we get discount
  const earlyPayDate = new Date(due.getTime() - 10 * 24 * 60 * 60 * 1000)

  if (earlyPayDate > today) {
    return earlyPayDate.toISOString()
  }

  return due.toISOString()
}

Complete Event Handler

// Main workflow event handler orchestrating all steps
on(
  $.Event.pattern({
    type: 'invoice.uploaded',
    subject: '$.Invoice.*',
  }),
  async (event) => {
    // Extract invoice data from event
    const invoice = {
      fileUrl: event.data.fileUrl,
      mimeType: event.data.mimeType,
      fileSize: event.data.fileSize,
      uploadedBy: event.data.userId,
      uploadedAt: event.data.timestamp,
    }

    // Create workflow instance
    const workflowId = $.generateId('workflow')
    const workflow = new InvoiceWorkflowStateMachine(workflowId, {
      invoice,
      startTime: Date.now(),
      submitterEmail: event.data.userEmail,
    })

    try {
      // Persist initial workflow state
      await db.workflows.insert({
        id: workflowId,
        type: 'invoice-processing',
        status: 'processing',
        currentState: workflow.currentState,
        context: workflow.context,
        createdAt: new Date(),
      })

      // Step 1: Transition to extracting state
      await workflow.transition('extracting')

      // Step 2: Perform OCR extraction
      const extractedData = await performOCR(invoice)
      workflow.context.extractedData = extractedData

      // Step 3: Structure data
      const structuredData = await structureExtractedData(extractedData.corrected)
      workflow.context.structuredData = structuredData

      // Step 4: Transition to validation
      await workflow.transition('validating')

      // Step 5: Validate invoice data
      await validateInvoiceData(workflow)

      // Step 6: Decide next step based on validation
      if (await workflow.shouldAutoApprove()) {
        // Auto-approve low-value invoices
        await workflow.transition('auto_approved')
        await workflow.transition('processing_payment')
      } else {
        // Require manual approval
        await workflow.transition('pending_approval')
        await routeToApprovers(workflow)
      }
    } catch (error) {
      console.error('Workflow error:', error)

      // Determine appropriate error state
      if (workflow.currentState === 'extracting') {
        await workflow.transition('extraction_failed', { error: error.message })
      } else if (workflow.currentState === 'validating') {
        await workflow.transition('validation_failed', { error: error.message })
      } else {
        // Generic error handling
        await workflow.transition('manual_review', { error: error.message })
      }
    }
  }
)

// Handle approval responses
on(
  $.Event.pattern({
    type: 'invoice.approval.response',
    subject: '$.Approval.*',
  }),
  async (event) => {
    await handleApprovalDecision(event.data.workflowId, event.data.approverId, event.data.decision, event.data.comments)
  }
)

// Handle timeout escalations
on(
  $.Event.pattern({
    type: 'workflow.timeout',
    subject: '$.Workflow.*',
  }),
  async (event) => {
    const workflow = await db.workflows.findOne({ id: event.data.workflowId })
    const wf = new InvoiceWorkflowStateMachine(event.data.workflowId, workflow.context)

    if (wf.currentState === 'pending_approval') {
      // Escalate to next level or reject
      const currentLevel = wf.context.approvals.currentLevel
      const maxLevel = wf.context.approvals.levels.length

      if (currentLevel + 1 < maxLevel) {
        // Escalate to next approval level
        wf.context.approvals.currentLevel++
        await wf.persistState()
        await sendApprovalNotifications(wf)
      } else {
        // No more escalation levels - transition to timeout state
        await wf.transition('approval_timeout')
      }
    }
  }
)

// Handle payment completion
on(
  $.Event.pattern({
    type: 'payment.completed',
    subject: '$.Payment.*',
  }),
  async (event) => {
    const workflow = await db.workflows.findOne({
      'context.payment.transactionId': event.data.transactionId,
    })

    if (workflow) {
      const wf = new InvoiceWorkflowStateMachine(workflow.id, workflow.context)
      await wf.transition('paid', { payment: event.data })
    }
  }
)

Testing

Comprehensive testing ensures the workflow operates reliably under all conditions, including edge cases, failures, and concurrent processing scenarios.

State Machine Tests

import { describe, it, expect, beforeEach } from 'vitest'
import { InvoiceWorkflowStateMachine } from './workflow'

describe('InvoiceWorkflowStateMachine', () => {
  let workflow: InvoiceWorkflowStateMachine

  beforeEach(() => {
    workflow = new InvoiceWorkflowStateMachine('test-workflow-001', {
      invoice: {
        fileUrl: 'https://example.com/invoice.pdf',
        mimeType: 'application/pdf',
        fileSize: 125000,
      },
    })
  })

  it('should initialize in received state', () => {
    expect(workflow.currentState).toBe('received')
  })

  it('should transition from received to extracting', async () => {
    const success = await workflow.transition('extracting')
    expect(success).toBe(true)
    expect(workflow.currentState).toBe('extracting')
  })

  it('should reject invalid transitions', async () => {
    await expect(workflow.transition('paid')).rejects.toThrow('Invalid transition')
  })

  it('should track state history', async () => {
    await workflow.transition('extracting')
    await workflow.transition('validating')

    expect(workflow.history).toHaveLength(2)
    expect(workflow.history[0].state).toBe('received')
    expect(workflow.history[1].state).toBe('extracting')
  })

  it('should execute state actions', async () => {
    await workflow.transition('extracting')

    // Verify OCR was performed
    expect(workflow.context.extractedData).toBeDefined()
  })

  it('should handle auto-approval for low amounts', async () => {
    workflow.context.structuredData = {
      amounts: { total: 250 },
      vendor: { trusted: true },
      invoice: { purchaseOrder: 'PO-12345' },
    }

    const shouldAutoApprove = await workflow.shouldAutoApprove()
    expect(shouldAutoApprove).toBe(true)
  })

  it('should require approval for high amounts', async () => {
    workflow.context.structuredData = {
      amounts: { total: 5000 },
      vendor: { trusted: true },
      invoice: { purchaseOrder: 'PO-12345' },
    }

    const shouldAutoApprove = await workflow.shouldAutoApprove()
    expect(shouldAutoApprove).toBe(false)
  })

  it('should handle retry logic on failures', async () => {
    workflow.context.retries = {}

    // Simulate failure
    await workflow.handleActionFailure('performOCR', new Error('API timeout'))

    expect(workflow.context.retries.performOCR).toBe(1)
  })

  it('should transition to error state after max retries', async () => {
    workflow.context.retries = { performOCR: 3 }

    await workflow.handleActionFailure('performOCR', new Error('API timeout'))

    expect(workflow.currentState).toBe('extraction_failed')
  })
})

Integration Tests

import { describe, it, expect, beforeAll, afterAll } from 'vitest'
import $ from 'sdk.do'

describe('Invoice Workflow Integration', () => {
  let testInvoiceUrl: string

  beforeAll(async () => {
    // Upload test invoice
    testInvoiceUrl = await uploadTestInvoice()
  })

  afterAll(async () => {
    // Cleanup test data
    await cleanupTestData()
  })

  it('should process invoice end-to-end for auto-approval', async () => {
    // Trigger workflow
    await send(
      $.Event.create({
        type: 'invoice.uploaded',
        subject: '$.Invoice.test001',
        data: {
          fileUrl: testInvoiceUrl,
          mimeType: 'application/pdf',
          fileSize: 125000,
          userId: 'test-user',
          userEmail: '[email protected]',
        },
      })
    )

    // Wait for workflow completion (with timeout)
    const workflow = await waitForWorkflowCompletion('test-workflow-001', 60000)

    expect(workflow.status).toBe('completed')
    expect(workflow.success).toBe(true)
    expect(workflow.finalState).toBe('paid')

    // Verify data extraction
    expect(workflow.context.structuredData.vendor.name).toBeDefined()
    expect(workflow.context.structuredData.amounts.total).toBeGreaterThan(0)

    // Verify payment created
    expect(workflow.context.payment.transactionId).toBeDefined()
  })

  it('should route to approvers for high-value invoices', async () => {
    // Upload high-value invoice
    const highValueInvoiceUrl = await uploadTestInvoice({ amount: 10000 })

    await send(
      $.Event.create({
        type: 'invoice.uploaded',
        subject: '$.Invoice.test002',
        data: {
          fileUrl: highValueInvoiceUrl,
          mimeType: 'application/pdf',
          fileSize: 145000,
          userId: 'test-user',
          userEmail: '[email protected]',
        },
      })
    )

    // Wait for routing
    await new Promise((resolve) => setTimeout(resolve, 5000))

    const workflow = await db.workflows.findOne({ id: 'test-workflow-002' })

    expect(workflow.currentState).toBe('pending_approval')
    expect(workflow.context.approvals.levels.length).toBeGreaterThan(1)

    // Verify notifications sent
    const notifications = await db.notifications.find({
      type: 'approval_required',
    })

    expect(notifications.length).toBeGreaterThan(0)
  })

  it('should handle duplicate invoice detection', async () => {
    // Upload same invoice twice
    await send(
      $.Event.create({
        type: 'invoice.uploaded',
        subject: '$.Invoice.duplicate001',
        data: {
          fileUrl: testInvoiceUrl,
          mimeType: 'application/pdf',
          fileSize: 125000,
          userId: 'test-user',
          userEmail: '[email protected]',
        },
      })
    )

    // Wait for first to complete
    await waitForWorkflowCompletion('duplicate-workflow-001', 60000)

    // Upload duplicate
    await send(
      $.Event.create({
        type: 'invoice.uploaded',
        subject: '$.Invoice.duplicate002',
        data: {
          fileUrl: testInvoiceUrl,
          mimeType: 'application/pdf',
          fileSize: 125000,
          userId: 'test-user',
          userEmail: '[email protected]',
        },
      })
    )

    // Wait for validation
    await new Promise((resolve) => setTimeout(resolve, 5000))

    const workflow = await db.workflows.findOne({ id: 'duplicate-workflow-002' })

    expect(workflow.currentState).toBe('validation_failed')
    expect(workflow.context.validation.errors).toContain('Duplicate invoice detected')
  })

  it('should handle OCR extraction failures gracefully', async () => {
    // Upload corrupted invoice
    const corruptedUrl = await uploadCorruptedInvoice()

    await send(
      $.Event.create({
        type: 'invoice.uploaded',
        subject: '$.Invoice.corrupted001',
        data: {
          fileUrl: corruptedUrl,
          mimeType: 'application/pdf',
          fileSize: 50000,
          userId: 'test-user',
          userEmail: '[email protected]',
        },
      })
    )

    // Wait for error handling
    await new Promise((resolve) => setTimeout(resolve, 10000))

    const workflow = await db.workflows.findOne({ id: 'corrupted-workflow-001' })

    expect(['extraction_failed', 'manual_review']).toContain(workflow.currentState)
  })
})

// Helper functions
async function waitForWorkflowCompletion(workflowId: string, timeout: number): Promise<any> {
  const startTime = Date.now()

  while (Date.now() - startTime < timeout) {
    const workflow = await db.workflows.findOne({ id: workflowId })

    if (workflow?.status === 'completed') {
      return workflow
    }

    await new Promise((resolve) => setTimeout(resolve, 1000))
  }

  throw new Error(`Workflow ${workflowId} did not complete within ${timeout}ms`)
}

Deployment

Production deployment requires careful configuration of queues, webhooks, monitoring, and error handling to ensure reliable operation at scale.

Production Configuration

import $, { ai, db, on, send } from 'sdk.do'

// Production workflow service deployment
const productionConfig = {
  service: 'invoice-processing-workflow',
  environment: 'production',

  // Queue configuration for async processing
  queues: {
    invoiceProcessing: {
      name: 'invoice-processing',
      concurrency: 10, // Process 10 invoices concurrently
      retryAttempts: 3,
      retryDelay: 5000,
      deadLetterQueue: 'invoice-processing-dlq',
    },
    approvalNotifications: {
      name: 'approval-notifications',
      concurrency: 50,
      retryAttempts: 5,
      retryDelay: 2000,
    },
    paymentProcessing: {
      name: 'payment-processing',
      concurrency: 5,
      retryAttempts: 5,
      retryDelay: 10000,
      deadLetterQueue: 'payment-processing-dlq',
    },
  },

  // Webhook endpoints
  webhooks: {
    approvalResponse: {
      path: '/webhooks/approval-response',
      method: 'POST',
      authentication: 'bearer',
      rateLimit: {
        requests: 100,
        window: 60, // per minute
      },
    },
    stripePaymentStatus: {
      path: '/webhooks/stripe/payment-status',
      method: 'POST',
      authentication: 'stripe-signature',
      rateLimit: {
        requests: 1000,
        window: 60,
      },
    },
  },

  // External service configuration
  integrations: {
    stripe: {
      apiKey: process.env.STRIPE_API_KEY,
      webhookSecret: process.env.STRIPE_WEBHOOK_SECRET,
      timeout: 30000,
    },
    quickbooks: {
      oauthToken: process.env.QUICKBOOKS_OAUTH_TOKEN,
      realmId: process.env.QUICKBOOKS_REALM_ID,
      timeout: 30000,
    },
    sendgrid: {
      apiKey: process.env.SENDGRID_API_KEY,
      fromEmail: '[email protected]',
      timeout: 10000,
    },
  },

  // AI model configuration
  ai: {
    vision: {
      model: 'gpt-5-vision',
      maxTokens: 4096,
      temperature: 0.1, // Low temperature for accurate extraction
      timeout: 60000,
    },
  },

  // Monitoring and alerting
  monitoring: {
    metrics: {
      enabled: true,
      interval: 60000, // Report every minute
      aggregations: ['count', 'avg', 'p95', 'p99'],
    },
    alerts: {
      workflowFailureRate: {
        threshold: 5, // Alert if failure rate > 5%
        window: 3600, // Over 1 hour
        channels: ['email', 'slack'],
      },
      approvalTimeout: {
        threshold: 10, // Alert if 10+ approvals timeout
        window: 86400, // Over 24 hours
        channels: ['email'],
      },
      paymentFailureRate: {
        threshold: 2, // Alert if failure rate > 2%
        window: 3600,
        channels: ['email', 'slack', 'pagerduty'],
      },
    },
  },

  // Performance tuning
  performance: {
    caching: {
      enabled: true,
      ttl: {
        vendors: 3600,
        purchaseOrders: 1800,
        approvers: 600,
      },
    },
    batchProcessing: {
      enabled: true,
      batchSize: 50,
      flushInterval: 30000,
    },
  },

  // Security configuration
  security: {
    encryption: {
      atRest: true,
      inTransit: true,
      algorithm: 'AES-256-GCM',
    },
    auditLogging: {
      enabled: true,
      events: ['all'],
      retention: 2555, // 7 years
    },
    accessControl: {
      approvalAuthentication: 'mfa-required',
      paymentAuthentication: 'mfa-required',
    },
  },
}

// Deploy service with configuration
await $.Service.deploy({
  name: 'invoice-processing-workflow',
  version: '1.0.0',
  config: productionConfig,
  regions: ['us-east-1', 'us-west-2', 'eu-west-1'],
  scaling: {
    min: 2,
    max: 50,
    targetCPU: 70,
    targetMemory: 80,
  },
})

Monitoring

Comprehensive monitoring tracks workflow performance, identifies bottlenecks, and alerts operations teams to issues requiring attention.

Workflow Metrics Dashboard

// Define workflow metrics collection
const workflowMetrics = {
  // Processing metrics
  processingTime: {
    name: 'invoice_processing_time_seconds',
    type: 'histogram',
    description: 'Time to process invoice from upload to payment',
    labels: ['state', 'auto_approved', 'amount_tier'],
    buckets: [10, 30, 60, 300, 600, 1800, 3600, 86400],
  },

  // Approval metrics
  approvalTime: {
    name: 'invoice_approval_time_seconds',
    type: 'histogram',
    description: 'Time from approval request to decision',
    labels: ['level', 'decision', 'escalated'],
    buckets: [60, 300, 1800, 3600, 86400, 172800],
  },

  approvalRate: {
    name: 'invoice_approval_rate',
    type: 'gauge',
    description: 'Percentage of invoices approved vs rejected',
    labels: ['level', 'amount_tier'],
  },

  // Success metrics
  successRate: {
    name: 'invoice_workflow_success_rate',
    type: 'gauge',
    description: 'Percentage of successfully completed workflows',
    labels: ['auto_approved'],
  },

  // Error metrics
  errorRate: {
    name: 'invoice_workflow_error_rate',
    type: 'counter',
    description: 'Count of workflow errors by type',
    labels: ['error_type', 'state'],
  },

  // State transition metrics
  stateTransitions: {
    name: 'invoice_state_transitions_total',
    type: 'counter',
    description: 'Count of state transitions',
    labels: ['from_state', 'to_state'],
  },

  // Business metrics
  invoiceVolume: {
    name: 'invoice_volume_total',
    type: 'counter',
    description: 'Total invoices processed',
    labels: ['auto_approved', 'amount_tier'],
  },

  invoiceValue: {
    name: 'invoice_value_total',
    type: 'counter',
    description: 'Total value of invoices processed',
    labels: ['currency', 'status'],
  },
}

// Record metrics throughout workflow
on(
  $.Event.pattern({
    type: 'workflow.state.changed',
    subject: '$.Workflow.*',
  }),
  async (event) => {
    // Record state transition
    await $.Metric.increment('invoice_state_transitions_total', {
      from_state: event.data.from,
      to_state: event.data.to,
    })

    // Calculate and record processing time for final states
    if (['paid', 'rejected', 'validation_failed'].includes(event.data.to)) {
      const workflow = await db.workflows.findOne({ id: event.subject })
      const duration = Date.now() - workflow.context.startTime

      await $.Metric.histogram('invoice_processing_time_seconds', duration / 1000, {
        state: event.data.to,
        auto_approved: workflow.context.autoApproved?.toString() || 'false',
        amount_tier: getAmountTier(workflow.context.structuredData.amounts.total),
      })
    }
  }
)

// Track approval metrics
on(
  $.Event.pattern({
    type: 'invoice.approval.response',
    subject: '$.Approval.*',
  }),
  async (event) => {
    const workflow = await db.workflows.findOne({ id: event.data.workflowId })
    const approvalRequested = new Date(workflow.context.approvals.requestedAt)
    const approvalDuration = Date.now() - approvalRequested.getTime()

    await $.Metric.histogram('invoice_approval_time_seconds', approvalDuration / 1000, {
      level: event.data.level.toString(),
      decision: event.data.decision,
      escalated: event.data.escalated?.toString() || 'false',
    })
  }
)

// Helper to categorize invoice amounts
function getAmountTier(amount: number): string {
  if (amount <= 500) return 'small'
  if (amount <= 5000) return 'medium'
  if (amount <= 25000) return 'large'
  return 'xlarge'
}

Alert Configuration

// Configure alerting rules
const alertRules = [
  {
    name: 'high_workflow_failure_rate',
    condition: 'invoice_workflow_error_rate > 5% over 1h',
    severity: 'critical',
    channels: ['email', 'slack', 'pagerduty'],
    message: 'Invoice workflow failure rate exceeds 5% - immediate attention required',
    runbook: 'https://docs.company.com/runbooks/invoice-workflow-failures',
  },
  {
    name: 'stuck_approvals',
    condition: 'count(workflows where state=pending_approval AND age>48h) > 10',
    severity: 'warning',
    channels: ['email', 'slack'],
    message: '10+ invoices stuck in approval for over 48 hours',
    runbook: 'https://docs.company.com/runbooks/stuck-approvals',
  },
  {
    name: 'payment_processing_failures',
    condition: 'invoice_workflow_error_rate where state=processing_payment > 2% over 1h',
    severity: 'critical',
    channels: ['email', 'slack', 'pagerduty'],
    message: 'Payment processing failure rate exceeds 2% - check integrations',
    runbook: 'https://docs.company.com/runbooks/payment-failures',
  },
  {
    name: 'ocr_extraction_degradation',
    condition: 'avg(ocr_confidence) < 0.85 over 30m',
    severity: 'warning',
    channels: ['email'],
    message: 'OCR extraction confidence declining - review recent invoices',
    runbook: 'https://docs.company.com/runbooks/ocr-issues',
  },
  {
    name: 'approval_sla_breach',
    condition: 'p95(invoice_approval_time_seconds) > 172800 over 24h',
    severity: 'warning',
    channels: ['email'],
    message: '95th percentile approval time exceeds 48 hours',
    runbook: 'https://docs.company.com/runbooks/approval-sla',
  },
]

// Register alert rules
for (const rule of alertRules) {
  await $.Alert.create(rule)
}

Usage Examples

Real-world usage examples demonstrate how businesses integrate the workflow service into their accounts payable operations.

Simple Invoice Processing

import $, { send } from 'sdk.do'

// Example 1: Process simple invoice that auto-approves
async function processSimpleInvoice() {
  // Upload invoice file
  const file = await fetch('https://vendor.com/invoices/INV-2024-001.pdf')
  const blob = await file.blob()

  const uploadedFile = await $.Storage.upload({
    file: blob,
    path: 'invoices/2024/01/INV-2024-001.pdf',
    contentType: 'application/pdf',
  })

  // Trigger workflow
  await send(
    $.Event.create({
      type: 'invoice.uploaded',
      subject: '$.Invoice.INV-2024-001',
      data: {
        fileUrl: uploadedFile.url,
        mimeType: 'application/pdf',
        fileSize: blob.size,
        userId: 'user-123',
        userEmail: '[email protected]',
        metadata: {
          vendor: 'Office Supplies Co',
          expectedAmount: 347.5,
          purchaseOrder: 'PO-2024-0042',
        },
      },
    })
  )

  // Monitor workflow progress
  const workflowId = 'INV-2024-001'

  // Poll for completion
  let workflow = await db.workflows.findOne({ id: workflowId })

  while (workflow.status === 'processing') {
    await new Promise((resolve) => setTimeout(resolve, 2000))
    workflow = await db.workflows.findOne({ id: workflowId })
  }

  if (workflow.success) {
    console.log(`Invoice ${workflowId} processed successfully`)
    console.log(`Payment ID: ${workflow.context.payment.transactionId}`)
    console.log(`Amount: ${workflow.context.structuredData.amounts.total}`)
  } else {
    console.error(`Invoice ${workflowId} failed: ${workflow.error}`)
  }
}

Complex Multi-Approval Invoice

// Example 2: High-value invoice requiring multiple approvals
async function processComplexInvoice() {
  const invoiceData = {
    fileUrl: 'https://storage.company.com/invoices/INV-2024-002.pdf',
    mimeType: 'application/pdf',
    fileSize: 245000,
    userId: 'user-456',
    userEmail: '[email protected]',
    metadata: {
      vendor: 'Enterprise Software Inc',
      expectedAmount: 125000.0,
      category: 'software-licenses',
      department: 'engineering',
    },
  }

  // Trigger workflow
  await send(
    $.Event.create({
      type: 'invoice.uploaded',
      subject: '$.Invoice.INV-2024-002',
      data: invoiceData,
    })
  )

  // Subscribe to workflow events
  on(
    $.Event.pattern({
      type: 'workflow.state.changed',
      subject: '$.Workflow.INV-2024-002',
    }),
    async (event) => {
      console.log(`Workflow state changed: ${event.data.from} -> ${event.data.to}`)

      if (event.data.to === 'pending_approval') {
        console.log('Invoice requires approval')
        console.log(`Approval levels: ${event.data.context.approvals.levels.length}`)
        console.log(`Current approvers:`, event.data.context.approvals.pending.approvers)
      } else if (event.data.to === 'approved') {
        console.log('All approvals received, processing payment')
      } else if (event.data.to === 'paid') {
        console.log('Payment completed successfully')
        console.log(`Transaction ID: ${event.data.context.payment.transactionId}`)
      }
    }
  )

  // Simulate approval process
  setTimeout(async () => {
    // Department manager approves
    await send(
      $.Event.create({
        type: 'invoice.approval.response',
        subject: '$.Approval.INV-2024-002-L1',
        data: {
          workflowId: 'INV-2024-002',
          approverId: 'manager-engineering',
          decision: 'approve',
          comments: 'Valid license renewal for engineering team',
        },
      })
    )
  }, 5000)

  setTimeout(async () => {
    // Finance director approves
    await send(
      $.Event.create({
        type: 'invoice.approval.response',
        subject: '$.Approval.INV-2024-002-L2',
        data: {
          workflowId: 'INV-2024-002',
          approverId: 'director-finance',
          decision: 'approve',
          comments: 'Within approved budget',
        },
      })
    )
  }, 10000)

  setTimeout(async () => {
    // CFO approves
    await send(
      $.Event.create({
        type: 'invoice.approval.response',
        subject: '$.Approval.INV-2024-002-L3',
        data: {
          workflowId: 'INV-2024-002',
          approverId: 'cfo',
          decision: 'approve',
          comments: 'Approved - critical business software',
        },
      })
    )
  }, 15000)
}

Batch Invoice Processing

// Example 3: Process multiple invoices in batch
async function processBatchInvoices() {
  // Batch of invoices from email or FTP
  const invoiceBatch = [
    {
      file: 'vendor-a-inv-001.pdf',
      vendor: 'Vendor A',
      expectedAmount: 1250.0,
    },
    {
      file: 'vendor-b-inv-045.pdf',
      vendor: 'Vendor B',
      expectedAmount: 3400.0,
    },
    {
      file: 'vendor-c-inv-789.pdf',
      vendor: 'Vendor C',
      expectedAmount: 450.0,
    },
    {
      file: 'vendor-d-inv-321.pdf',
      vendor: 'Vendor D',
      expectedAmount: 8900.0,
    },
  ]

  // Process all invoices concurrently
  const workflowIds = await Promise.all(
    invoiceBatch.map(async (invoice) => {
      const workflowId = $.generateId('workflow')

      await send(
        $.Event.create({
          type: 'invoice.uploaded',
          subject: `$.Invoice.${workflowId}`,
          data: {
            fileUrl: `https://storage.company.com/batch/${invoice.file}`,
            mimeType: 'application/pdf',
            userId: 'batch-processor',
            userEmail: '[email protected]',
            metadata: {
              vendor: invoice.vendor,
              expectedAmount: invoice.expectedAmount,
              batchId: 'batch-2024-01-27',
            },
          },
        })
      )

      return workflowId
    })
  )

  console.log(`Started processing ${workflowIds.length} invoices`)

  // Monitor batch completion
  const results = await Promise.all(
    workflowIds.map(async (workflowId) => {
      // Wait for workflow completion
      let workflow = await db.workflows.findOne({ id: workflowId })

      while (workflow.status === 'processing') {
        await new Promise((resolve) => setTimeout(resolve, 5000))
        workflow = await db.workflows.findOne({ id: workflowId })
      }

      return {
        workflowId,
        success: workflow.success,
        finalState: workflow.finalState,
        amount: workflow.context.structuredData?.amounts?.total,
        error: workflow.error,
      }
    })
  )

  // Generate batch report
  const successful = results.filter((r) => r.success).length
  const failed = results.filter((r) => !r.success).length
  const totalValue = results.filter((r) => r.success).reduce((sum, r) => sum + (r.amount || 0), 0)

  console.log('\n=== Batch Processing Report ===')
  console.log(`Total invoices: ${results.length}`)
  console.log(`Successful: ${successful}`)
  console.log(`Failed: ${failed}`)
  console.log(`Total value processed: $${totalValue.toFixed(2)}`)
  console.log('\nFailed invoices:')

  results
    .filter((r) => !r.success)
    .forEach((r) => {
      console.log(`  - ${r.workflowId}: ${r.error}`)
    })
}

Troubleshooting

Common issues and their resolutions help operations teams quickly diagnose and fix workflow problems.

Common Workflow Failures

OCR Extraction Fails: The most common cause is poor image quality or unsupported invoice formats. Resolution steps: verify the uploaded file is not corrupted, check file size is under 10MB, ensure invoice is in PDF or image format, verify text is not handwritten, review OCR confidence scores in logs, and consider manual data entry for problematic invoices.

Duplicate Invoice Detection: False positives can occur when vendors resubmit corrected invoices. Resolution: check if invoice number was previously processed, verify vendor name matches exactly, compare amounts and dates to determine if legitimate duplicate, add override flag to workflow context if valid resubmission, update duplicate detection logic to handle invoice corrections.

Approval Timeouts: Invoices stuck in pending approval state exceed SLA targets. Resolution: check approver availability and notification delivery, verify approval URLs are accessible, escalate to next approval level or manager, extend timeout for specific cases requiring additional review, send reminder notifications, check if approvers are on vacation and route to delegates.

Payment Processing Failures: Integration issues with Stripe or QuickBooks cause payment failures. Resolution: verify API credentials are valid and not expired, check integration service health status, review error messages from payment processor, retry payment with exponential backoff, verify sufficient funds in payment account, check for network connectivity issues, route to manual payment processing if integration down.

Validation Errors: Business rule violations prevent invoice approval. Resolution: review specific validation errors in workflow context, check purchase order data is correct, verify vendor information is up to date, confirm invoice amounts match expected values, update business rules if too restrictive, provide override mechanism for exceptional cases.

State Machine Stuck: Workflow fails to progress through states. Resolution: check for pending actions that haven't completed, review state transition logs for errors, verify all required data is present in context, restart workflow from last valid state, check for database connection issues, review timeout configurations.

Integration Timeouts: External systems fail to respond within timeout limits. Resolution: increase timeout values for slow integrations, implement caching to reduce external calls, add circuit breaker patterns to handle outages, queue requests for retry during outages, implement graceful degradation, monitor integration health proactively.

Error Recovery Procedures

When workflows enter error states, follow these recovery procedures: For extraction failures, review the original invoice file, attempt manual OCR using alternative tools, create support ticket for manual data entry, update OCR model if consistent failures detected. For validation failures, review validation error messages, update invoice data if errors are correctable, override validation for exceptional cases with approval, update business rules if causing false failures. For payment failures, verify payment account status, check integration credentials, retry payment manually through ERP, update vendor payment information if incorrect, escalate to finance team for resolution.

The workflow service includes comprehensive audit logging tracking every state transition, data modification, and decision point. Access audit logs through the admin dashboard or query directly: await db.auditLogs.find({ workflowId: 'INV-2024-001' }). Logs retain for seven years to meet compliance requirements.

For critical issues blocking multiple workflows, escalate immediately to the platform engineering team. Include workflow IDs, error messages, time ranges, and impact assessment. The team monitors a dedicated Slack channel #invoice-workflow-support for urgent issues requiring immediate attention.


State Machine Diagram

stateDiagram-v2 [*] --> received received --> extracting: Upload validated received --> invalid: Invalid format extracting --> validating: OCR complete extracting --> extraction_failed: OCR failed validating --> auto_approved: Low value + PO validating --> pending_approval: Requires approval validating --> validation_failed: Validation error auto_approved --> processing_payment pending_approval --> approved: All approvals received pending_approval --> rejected: Approval denied pending_approval --> approval_timeout: SLA exceeded approved --> processing_payment processing_payment --> paid: Payment successful processing_payment --> payment_failed: Payment error payment_failed --> manual_review: Retry exhausted payment_failed --> processing_payment: Retry extraction_failed --> manual_review validation_failed --> manual_review approval_timeout --> pending_approval: Extended approval_timeout --> rejected: Max timeout manual_review --> validating: Issue resolved manual_review --> rejected: Cannot resolve paid --> [*] rejected --> [*] invalid --> [*]

This comprehensive workflow service demonstrates the full capabilities of the .do platform for building production-ready automation services. The invoice processing workflow handles complex multi-step operations with reliability, performance, and comprehensive monitoring.