Service Architecture

Service architecture on the .do platform focuses on creating composable, autonomous services that can be orchestrated through semantic patterns.

Services-as-Software

Unlike traditional microservices, Services-as-Software are:

Semantic: Every interface follows $.Subject.predicate.Object patterns
Autonomous: Services operate independently with their own data and logic
Composable: Services can be combined in multiple ways
Monetizable: Built-in billing and marketplace distribution
Versionable: Services are versioned and can coexist

Architecture Principles

1. Clear Boundaries

Each service has a well-defined scope and responsibility:

// ✅ Good: Focused service
const EmailService = $.service({
  name: 'Email Delivery Service',
  operations: {
    send: $.Email.send,
    schedule: $.Email.schedule,
    track: $.Email.track,
  },
})

// ❌ Bad: Too broad
const CommunicationService = $.service({
  name: 'Communication Service',
  operations: {
    sendEmail: $.Email.send,
    sendSMS: $.SMS.send,
    makeCall: $.Phone.call,
    sendPush: $.Push.send,
  },
})

2. Explicit Contracts

Define clear input and output schemas:

import { Email, Person } from 'schema.org.ai'

const EmailService = $.service({
  operations: {
    send: {
      input: {
        to: { type: 'string', format: 'email', required: true },
        from: { type: 'string', format: 'email', required: true },
        subject: { type: 'string', required: true },
        body: { type: 'string', required: true },
        template: { type: 'string', required: false },
      },
      output: {
        messageId: { type: 'string' },
        status: { type: 'enum', values: ['sent', 'queued', 'failed'] },
        timestamp: { type: 'datetime' },
      },
    },
  },
})

3. Loose Coupling

Services communicate through events and semantic patterns, not direct calls:

// Event-driven communication
on($.Order.created, async (order) => {
  // Each service independently reacts
  await send($.Email.send, {
    to: order.customer.email,
    template: 'order-confirmation',
    data: order,
  })

  await send($.Inventory.reserve, {
    items: order.items,
    orderId: order.id,
  })

  await send($.Analytics.track, {
    event: 'order_created',
    properties: { orderId: order.id, total: order.total },
  })
})

4. Data Ownership

Each service owns its data and exposes it through well-defined operations:

// UserService owns user data
const UserService = $.service({
  data: {
    users: $.db.collection<Person>('users'),
    sessions: $.db.collection('sessions'),
  },

  operations: {
    // Public operations
    getProfile: async (userId: string) => {
      return await UserService.data.users.findOne({ id: userId })
    },

    updateProfile: async (userId: string, updates: Partial<Person>) => {
      return await UserService.data.users.update({ id: userId }, updates)
    },

    // Internal operations (not exposed)
    _hashPassword: (password: string) => {
      // Internal logic
    },
  },
})

Service Patterns

1. API Services

Expose HTTP endpoints for external consumption:

const APIService = $.service({
  name: 'Product API',
  type: 'api',

  endpoints: {
    'GET /products': async (c) => {
      const products = await db.list($.Product)
      return c.json(products)
    },

    'GET /products/:id': async (c) => {
      const id = c.req.param('id')
      const product = await db.get($.Product, id)
      return c.json(product)
    },

    'POST /products': async (c) => {
      const data = await c.req.json()
      const product = await db.create($.Product, data)
      return c.json(product, 201)
    },
  },
})

2. Event Services

Process events and emit new events:

const OrderProcessingService = $.service({
  name: 'Order Processing',
  type: 'event',

  handlers: {
    [$.Order.created]: async (order) => {
      // Validate order
      const validation = await validateOrder(order)
      if (!validation.valid) {
        await send($.Order.rejected, { order, reason: validation.reason })
        return
      }

      // Process payment
      const payment = await processPayment(order)
      if (payment.status === 'failed') {
        await send($.Order.payment_failed, { order, payment })
        return
      }

      // Fulfill order
      await send($.Order.confirmed, { order, payment })
    },
  },
})

3. Worker Services

Run background jobs and scheduled tasks:

const ReportingService = $.service({
  name: 'Reporting Service',
  type: 'worker',

  jobs: {
    dailyReport: {
      schedule: '0 0 * * *', // Daily at midnight
      handler: async () => {
        const stats = await calculateDailyStats()
        await send($.Report.generated, { type: 'daily', data: stats })
      },
    },

    weeklyDigest: {
      schedule: '0 9 * * 1', // Monday at 9am
      handler: async () => {
        const users = await db.list($.User, { where: { emailPrefs: 'weekly' } })
        for (const user of users) {
          await send($.Email.send, {
            to: user.email,
            template: 'weekly-digest',
            data: await getUserDigest(user),
          })
        }
      },
    },
  },
})

4. Agent Services

AI-powered autonomous services:

const CustomerSupportAgent = $.service({
  name: 'Customer Support Agent',
  type: 'agent',

  capabilities: {
    answerQuestion: async (question: string, context: any) => {
      const answer = await ai.generate({
        model: 'gpt-5',
        prompt: question,
        context: context,
        schema: $.Answer,
      })
      return answer
    },

    resolveIssue: async (issue: string, user: Person) => {
      // Agent autonomously decides how to resolve
      const resolution = await ai.agentic({
        goal: 'Resolve customer issue',
        tools: ['refund', 'replace', 'escalate'],
        context: { issue, user },
      })
      return resolution
    },
  },
})

Service Composition

1. Sequential Composition

Chain services together:

// Order -> Payment -> Fulfillment
const OrderFlow = $.compose([OrderValidationService, PaymentService, FulfillmentService])

on($.Order.created, async (order) => {
  const result = await OrderFlow.execute(order)
  await send($.Order.completed, result)
})

2. Parallel Composition

Run services concurrently:

// Parallel processing
const EnrichmentPipeline = $.compose.parallel([EmailVerificationService, CompanyLookupService, SocialProfileService])

on($.Lead.captured, async (lead) => {
  const enriched = await EnrichmentPipeline.execute(lead)
  await send($.Lead.enriched, enriched)
})

3. Conditional Composition

Route based on conditions:

const PaymentRouter = $.compose.conditional({
  conditions: [
    { when: (order) => order.total < 100, use: BasicPaymentService },
    { when: (order) => order.total < 1000, use: StandardPaymentService },
    { when: (order) => order.total >= 1000, use: EnterprisePaymentService },
  ],
  default: StandardPaymentService,
})

Service Communication

1. Synchronous (RPC)

Direct service-to-service calls:

// Call another service
const user = await $.rpc.UserService.getProfile(userId)
const balance = await $.rpc.BillingService.getBalance(userId)

2. Asynchronous (Events)

Loosely coupled event-driven communication:

// Emit event
await send($.User.registered, user)

// Multiple services can react
on($.User.registered, async (user) => {
  // Email service
  await send($.Email.welcome, user)
})

on($.User.registered, async (user) => {
  // Analytics service
  await send($.Analytics.track, { event: 'signup', user })
})

3. Streaming

Real-time data streams:

const stream = $.stream.subscribe($.Order.status_changed)

for await (const event of stream) {
  console.log('Order status:', event.data.status)
  // React to status changes in real-time
}

Service Deployment

1. Edge Deployment

Deploy to Cloudflare Workers:

export default $.service({
  name: 'API Service',
  deployment: {
    target: 'edge',
    regions: 'global',
  },
})

2. Regional Deployment

Deploy to specific regions:

export default $.service({
  name: 'Data Processing',
  deployment: {
    target: 'regional',
    regions: ['us-east', 'eu-west'],
  },
})

3. Hybrid Deployment

Combine edge and regional:

export default $.service({
  name: 'E-commerce Platform',
  deployment: {
    api: { target: 'edge', regions: 'global' },
    database: { target: 'regional', regions: ['us-east'] },
    analytics: { target: 'edge', regions: 'global' },
  },
})

Service Versioning

1. Semantic Versioning

Follow semver for service versions:

const EmailService_v1 = $.service({
  name: 'Email Service',
  version: '1.0.0',
  operations: {
    send: $.Email.send,
  },
})

const EmailService_v2 = $.service({
  name: 'Email Service',
  version: '2.0.0',
  operations: {
    send: $.Email.send,
    schedule: $.Email.schedule, // New operation
  },
})

2. Gradual Migration

Support multiple versions simultaneously:

// Route based on caller version
const EmailService = $.service({
  versions: {
    '1.x': EmailService_v1,
    '2.x': EmailService_v2,
  },
})

Service Monitoring

1. Health Checks

Define service health checks:

const service = $.service({
  name: 'Payment Service',
  health: {
    check: async () => {
      const dbOk = await checkDatabase()
      const apiOk = await checkStripeAPI()
      return {
        status: dbOk && apiOk ? 'healthy' : 'unhealthy',
        checks: { database: dbOk, stripe: apiOk },
      }
    },
    interval: 30000, // Check every 30 seconds
  },
})

2. Metrics

Expose service metrics:

const service = $.service({
  name: 'Order Service',
  metrics: {
    requests: $.metric.counter('requests_total'),
    latency: $.metric.histogram('request_latency_ms'),
    errors: $.metric.counter('errors_total'),
  },
})

Best Practices

Do's

Keep services focused and single-purpose
Use semantic patterns for all operations
Define explicit contracts with schemas
Version services properly
Implement health checks and metrics
Use events for loose coupling
Document service interfaces

Don'ts

Don't create monolithic services
Don't share databases between services
Don't couple services tightly
Don't ignore versioning
Don't skip error handling
Don't forget to monitor services

Advanced Architecture Patterns

Service Mesh Pattern

Use a service mesh for sophisticated inter-service communication:

const ServiceMesh = $.mesh({
  services: [
    UserService,
    OrderService,
    PaymentService,
    InventoryService,
  ],

  policies: {
    // Automatic retries
    retry: {
      maxAttempts: 3,
      backoff: 'exponential',
      timeout: 5000,
    },

    // Circuit breaking
    circuitBreaker: {
      threshold: 5,
      timeout: 60000,
      fallback: 'cached',
    },

    // Load balancing
    loadBalancing: {
      strategy: 'round-robin',
      healthCheck: true,
    },

    // Service discovery
    discovery: {
      enabled: true,
      ttl: 30000,
    },
  },

  observability: {
    tracing: true,
    metrics: true,
    logging: true,
  },
})

// Use mesh for service communication
const order = await ServiceMesh.call(OrderService.getOrder, orderId)

Event Sourcing Pattern

Store all changes as events for complete audit trail:

// Event store
const EventStore = $.service({
  name: 'Event Store',
  type: 'storage',

  operations: {
    append: async (stream: string, event: Event) => {
      return await db.Events.create({
        stream,
        type: event.type,
        data: event.data,
        metadata: {
          timestamp: new Date(),
          version: await getStreamVersion(stream),
        },
      })
    },

    getStream: async (stream: string, fromVersion = 0) => {
      return await db.Events.list({
        where: {
          stream,
          'metadata.version': { $gte: fromVersion },
        },
        orderBy: { 'metadata.version': 'asc' },
      })
    },

    project: async (stream: string, projection: Function) => {
      const events = await EventStore.getStream(stream)
      return events.reduce(projection, {})
    },
  },
})

// Define aggregate
const OrderAggregate = {
  // Initial state
  initial: () => ({
    id: null,
    status: 'draft',
    items: [],
    total: 0,
  }),

  // Apply events to state
  apply: (state, event) => {
    switch (event.type) {
      case 'OrderCreated':
        return { ...state, id: event.data.id, status: 'created' }

      case 'ItemAdded':
        return {
          ...state,
          items: [...state.items, event.data.item],
          total: state.total + event.data.item.price,
        }

      case 'OrderConfirmed':
        return { ...state, status: 'confirmed' }

      case 'OrderShipped':
        return { ...state, status: 'shipped' }

      default:
        return state
    }
  },
}

// Use event sourcing
const orderId = 'order-123'
const orderState = await EventStore.project(orderId, OrderAggregate.apply)

CQRS Pattern

Separate read and write models for optimal performance:

// Write model (Commands)
const OrderCommands = $.service({
  name: 'Order Commands',
  type: 'command',

  commands: {
    createOrder: async (data) => {
      // Validate
      const validation = await validateOrder(data)
      if (!validation.valid) throw new Error(validation.error)

      // Create order
      const order = await db.Orders.create(data)

      // Emit event
      await send($.Order.created, order)

      return { orderId: order.$id }
    },

    confirmOrder: async (orderId) => {
      const order = await db.Orders.get(orderId)
      if (order.status !== 'pending') {
        throw new Error('Order cannot be confirmed')
      }

      await db.Orders.update(orderId, { status: 'confirmed' })
      await send($.Order.confirmed, { orderId })

      return { success: true }
    },
  },
})

// Read model (Queries)
const OrderQueries = $.service({
  name: 'Order Queries',
  type: 'query',

  // Optimized read database
  database: 'read-replica',

  queries: {
    getOrder: async (orderId) => {
      return await readDb.Orders.get(orderId)
    },

    listOrders: async (filters) => {
      return await readDb.Orders.list({
        where: filters,
        include: ['customer', 'items', 'payment'],
      })
    },

    getOrderStats: async (customerId) => {
      return await readDb.OrderStats.get(customerId)
    },
  },
})

// Projection: Update read model from events
on($.Order.created, async (order) => {
  await readDb.Orders.create(order)
  await readDb.OrderStats.increment(order.customerId, 'totalOrders')
})

on($.Order.confirmed, async ({ orderId }) => {
  const order = await db.Orders.get(orderId)
  await readDb.Orders.update(orderId, { status: 'confirmed' })
  await readDb.OrderStats.add(order.customerId, 'revenue', order.total)
})

Saga Pattern

Manage distributed transactions across services:

const OrderSaga = $.saga({
  name: 'Order Processing Saga',

  steps: [
    {
      name: 'reserve-inventory',
      action: async (order) => {
        return await send($.Inventory.reserve, {
          items: order.items,
          orderId: order.id,
        })
      },
      compensation: async (order, reservation) => {
        await send($.Inventory.release, { reservationId: reservation.id })
      },
    },

    {
      name: 'process-payment',
      action: async (order) => {
        return await send($.Payment.process, {
          orderId: order.id,
          amount: order.total,
          customer: order.customerId,
        })
      },
      compensation: async (order, payment) => {
        await send($.Payment.refund, { paymentId: payment.id })
      },
    },

    {
      name: 'create-shipment',
      action: async (order) => {
        return await send($.Shipment.create, {
          orderId: order.id,
          items: order.items,
          address: order.shippingAddress,
        })
      },
      compensation: async (order, shipment) => {
        await send($.Shipment.cancel, { shipmentId: shipment.id })
      },
    },

    {
      name: 'send-confirmation',
      action: async (order) => {
        await send($.Email.send, {
          to: order.customer.email,
          template: 'order-confirmation',
          data: order,
        })
      },
      compensation: async (order) => {
        await send($.Email.send, {
          to: order.customer.email,
          template: 'order-cancelled',
          data: order,
        })
      },
    },
  ],
})

// Execute saga
on($.Order.created, async (order) => {
  try {
    await OrderSaga.execute(order)
    await send($.Order.completed, { orderId: order.id })
  } catch (error) {
    // Saga automatically executes compensation steps
    await send($.Order.failed, { orderId: order.id, error })
  }
})

Performance Optimization

Caching Strategies

Implement multi-level caching:

const CachedService = $.service({
  name: 'Product Service',

  cache: {
    // L1: In-memory cache
    l1: {
      enabled: true,
      ttl: 60000, // 1 minute
      maxSize: 1000,
    },

    // L2: Distributed cache (Redis)
    l2: {
      enabled: true,
      ttl: 3600000, // 1 hour
      provider: 'redis',
    },

    // L3: CDN edge cache
    l3: {
      enabled: true,
      ttl: 86400000, // 24 hours
      provider: 'cloudflare',
    },
  },

  operations: {
    getProduct: async (productId: string) => {
      // Automatic cache lookup and population
      return await db.Products.get(productId)
    },

    listProducts: async (filters) => {
      // Cache with dynamic keys
      const cacheKey = `products:${JSON.stringify(filters)}`
      return await db.Products.list(filters)
    },
  },
})

// Manual cache control
const product = await CachedService.getProduct('prod-123', {
  cache: 'skip', // Bypass cache
})

// Invalidate cache
await CachedService.cache.invalidate('prod-123')
await CachedService.cache.invalidatePattern('products:*')

Database Optimization

Optimize database access patterns:

const OptimizedService = $.service({
  name: 'Order Service',

  database: {
    // Connection pooling
    pool: {
      min: 2,
      max: 10,
      idleTimeout: 30000,
    },

    // Read replicas
    replicas: {
      enabled: true,
      strategy: 'round-robin',
      endpoints: [
        'replica-1.db.do',
        'replica-2.db.do',
        'replica-3.db.do',
      ],
    },

    // Query optimization
    optimization: {
      // Automatic index suggestions
      indexHints: true,

      // Query plan caching
      planCache: true,

      // Batch operations
      batching: {
        enabled: true,
        maxBatchSize: 100,
        maxWaitTime: 100,
      },
    },
  },

  operations: {
    getOrders: async (filters) => {
      // Use read replica for queries
      return await db.Orders.list(filters, {
        replica: true,
        hint: 'idx_customer_status_date',
      })
    },

    createOrders: async (orders) => {
      // Batch insert for efficiency
      return await db.Orders.createMany(orders, {
        batch: true,
      })
    },
  },
})

Rate Limiting

Implement rate limiting for API protection:

const RateLimitedService = $.service({
  name: 'API Service',

  rateLimit: {
    // Global rate limit
    global: {
      max: 1000,
      window: 60000, // per minute
    },

    // Per-user rate limit
    user: {
      max: 100,
      window: 60000,
    },

    // Per-endpoint rate limits
    endpoints: {
      'POST /orders': {
        max: 10,
        window: 60000,
      },
      'GET /products': {
        max: 100,
        window: 60000,
      },
    },

    // Rate limit storage
    storage: 'redis',

    // Custom key generation
    keyGenerator: (c) => {
      return `ratelimit:${c.req.header('x-user-id')}:${c.req.path}`
    },
  },

  endpoints: {
    'POST /orders': async (c) => {
      // Automatic rate limiting applied
      const data = await c.req.json()
      const order = await db.Orders.create(data)
      return c.json(order)
    },
  },
})

Security Patterns

Authentication Service

Implement secure authentication:

const AuthService = $.service({
  name: 'Authentication Service',

  security: {
    jwt: {
      secret: env.JWT_SECRET,
      algorithm: 'RS256',
      expiresIn: '1h',
      refreshExpiresIn: '7d',
    },

    password: {
      minLength: 12,
      requireUppercase: true,
      requireLowercase: true,
      requireNumbers: true,
      requireSymbols: true,
      algorithm: 'argon2',
    },

    mfa: {
      enabled: true,
      methods: ['totp', 'sms', 'email'],
    },
  },

  operations: {
    register: async (email: string, password: string) => {
      // Validate password strength
      const validation = await validatePassword(password)
      if (!validation.valid) throw new Error(validation.error)

      // Hash password
      const hashedPassword = await hash(password)

      // Create user
      const user = await db.Users.create({
        email,
        password: hashedPassword,
        verified: false,
      })

      // Send verification email
      await send($.Email.sendVerification, { userId: user.$id })

      return { userId: user.$id }
    },

    login: async (email: string, password: string) => {
      // Find user
      const user = await db.Users.findOne({ where: { email } })
      if (!user) throw new Error('Invalid credentials')

      // Verify password
      const valid = await verify(password, user.password)
      if (!valid) throw new Error('Invalid credentials')

      // Check MFA
      if (user.mfaEnabled) {
        return { requiresMFA: true, userId: user.$id }
      }

      // Generate tokens
      const accessToken = await generateJWT(user, '1h')
      const refreshToken = await generateJWT(user, '7d')

      return { accessToken, refreshToken, user }
    },

    verifyMFA: async (userId: string, code: string) => {
      const user = await db.Users.get(userId)
      const valid = await verifyTOTP(user.mfaSecret, code)

      if (!valid) throw new Error('Invalid MFA code')

      const accessToken = await generateJWT(user, '1h')
      const refreshToken = await generateJWT(user, '7d')

      return { accessToken, refreshToken, user }
    },
  },
})

Authorization Service

Implement role-based access control (RBAC):

const AuthorizationService = $.service({
  name: 'Authorization Service',

  rbac: {
    roles: {
      admin: {
        permissions: ['*'], // All permissions
      },
      manager: {
        permissions: [
          'orders:read',
          'orders:create',
          'orders:update',
          'products:read',
          'products:update',
          'users:read',
        ],
      },
      user: {
        permissions: [
          'orders:read:own',
          'orders:create:own',
          'products:read',
        ],
      },
    },

    // Resource-based permissions
    resources: {
      orders: {
        read: ['admin', 'manager', 'user:own'],
        create: ['admin', 'manager', 'user:own'],
        update: ['admin', 'manager'],
        delete: ['admin'],
      },
      products: {
        read: ['*'],
        create: ['admin', 'manager'],
        update: ['admin', 'manager'],
        delete: ['admin'],
      },
    },
  },

  operations: {
    can: async (userId: string, permission: string, resource?: any) => {
      const user = await db.Users.get(userId)

      // Check role permissions
      const role = user.role
      const permissions = AuthorizationService.rbac.roles[role].permissions

      // Wildcard permission
      if (permissions.includes('*')) return true

      // Exact match
      if (permissions.includes(permission)) return true

      // Resource ownership check
      if (permission.endsWith(':own') && resource) {
        const basePermission = permission.replace(':own', '')
        if (permissions.includes(basePermission)) {
          return resource.userId === userId || resource.ownerId === userId
        }
      }

      return false
    },

    require: async (userId: string, permission: string, resource?: any) => {
      const allowed = await AuthorizationService.can(userId, permission, resource)
      if (!allowed) {
        throw new Error(`Permission denied: ${permission}`)
      }
    },
  },
})

// Use in services
const OrderService = $.service({
  operations: {
    getOrder: async (userId: string, orderId: string) => {
      const order = await db.Orders.get(orderId)

      // Check permission
      await AuthorizationService.require(userId, 'orders:read', order)

      return order
    },

    updateOrder: async (userId: string, orderId: string, updates: any) => {
      const order = await db.Orders.get(orderId)

      // Check permission
      await AuthorizationService.require(userId, 'orders:update', order)

      return await db.Orders.update(orderId, updates)
    },
  },
})

Testing Services

Unit Testing

Test individual service operations:

import { describe, it, expect, beforeEach } from 'vitest'
import { UserService } from './user-service'

describe('UserService', () => {
  beforeEach(async () => {
    // Setup test database
    await db.test.reset()
  })

  describe('createUser', () => {
    it('should create a new user', async () => {
      const user = await UserService.createUser({
        email: '[email protected]',
        name: 'Test User',
      })

      expect(user).toHaveProperty('$id')
      expect(user.email).toBe('[email protected]')
      expect(user.name).toBe('Test User')
    })

    it('should throw error for duplicate email', async () => {
      await UserService.createUser({
        email: '[email protected]',
        name: 'User 1',
      })

      await expect(
        UserService.createUser({
          email: '[email protected]',
          name: 'User 2',
        })
      ).rejects.toThrow('Email already exists')
    })

    it('should validate email format', async () => {
      await expect(
        UserService.createUser({
          email: 'invalid-email',
          name: 'Test User',
        })
      ).rejects.toThrow('Invalid email format')
    })
  })

  describe('getUser', () => {
    it('should retrieve existing user', async () => {
      const created = await UserService.createUser({
        email: '[email protected]',
        name: 'Test User',
      })

      const user = await UserService.getUser(created.$id)

      expect(user).toEqual(created)
    })

    it('should throw error for non-existent user', async () => {
      await expect(
        UserService.getUser('non-existent-id')
      ).rejects.toThrow('User not found')
    })
  })
})

Integration Testing

Test service interactions:

import { describe, it, expect, beforeEach } from 'vitest'
import { OrderService, PaymentService, InventoryService } from './services'

describe('Order Processing Integration', () => {
  beforeEach(async () => {
    await db.test.reset()
    await db.test.seed('products', 'inventory')
  })

  it('should process order end-to-end', async () => {
    // Create order
    const order = await OrderService.createOrder({
      customerId: 'customer-1',
      items: [
        { productId: 'product-1', quantity: 2 },
      ],
    })

    expect(order.status).toBe('pending')

    // Process payment
    const payment = await PaymentService.processPayment({
      orderId: order.$id,
      amount: order.total,
    })

    expect(payment.status).toBe('success')

    // Check inventory was reserved
    const inventory = await InventoryService.getReservation(order.$id)
    expect(inventory.reserved).toBe(true)

    // Verify order confirmed
    const confirmed = await OrderService.getOrder(order.$id)
    expect(confirmed.status).toBe('confirmed')
  })

  it('should rollback on payment failure', async () => {
    const order = await OrderService.createOrder({
      customerId: 'customer-1',
      items: [{ productId: 'product-1', quantity: 2 }],
    })

    // Simulate payment failure
    await expect(
      PaymentService.processPayment({
        orderId: order.$id,
        amount: -1, // Invalid amount
      })
    ).rejects.toThrow()

    // Verify inventory was released
    const inventory = await InventoryService.getReservation(order.$id)
    expect(inventory).toBeNull()

    // Verify order was cancelled
    const cancelled = await OrderService.getOrder(order.$id)
    expect(cancelled.status).toBe('cancelled')
  })
})

E2E Testing

Test complete user workflows:

import { describe, it, expect } from 'vitest'
import { test, expect as playwrightExpect } from '@playwright/test'

describe('E2E: Order Workflow', () => {
  test('user can complete purchase', async ({ page }) => {
    // Navigate to product page
    await page.goto('/products/wireless-headphones')

    // Add to cart
    await page.click('[data-testid="add-to-cart"]')
    await playwrightExpect(page.locator('.cart-count')).toHaveText('1')

    // Go to checkout
    await page.click('[data-testid="checkout"]')

    // Fill shipping information
    await page.fill('[name="name"]', 'John Doe')
    await page.fill('[name="email"]', '[email protected]')
    await page.fill('[name="address"]', '123 Main St')
    await page.fill('[name="city"]', 'San Francisco')
    await page.fill('[name="zip"]', '94102')

    // Fill payment information
    await page.fill('[name="cardNumber"]', '4242424242424242')
    await page.fill('[name="expiry"]', '12/25')
    await page.fill('[name="cvv"]', '123')

    // Submit order
    await page.click('[data-testid="submit-order"]')

    // Verify confirmation
    await playwrightExpect(page.locator('.order-confirmation')).toBeVisible()
    await playwrightExpect(page.locator('.order-number')).toContainText('ORDER-')

    // Verify email was sent
    const email = await getLastEmail('[email protected]')
    expect(email.subject).toBe('Order Confirmation')
    expect(email.body).toContain('ORDER-')
  })
})

Monitoring and Observability

Distributed Tracing

Implement distributed tracing across services:

const TracedService = $.service({
  name: 'Order Service',

  tracing: {
    enabled: true,
    provider: 'opentelemetry',

    // Sampling strategy
    sampling: {
      rate: 0.1, // Sample 10% of requests
      alwaysSample: ['POST /orders'], // Always sample critical endpoints
    },

    // Custom attributes
    attributes: {
      'service.name': 'order-service',
      'service.version': '1.0.0',
      'deployment.environment': env.ENVIRONMENT,
    },
  },

  operations: {
    createOrder: async (data) => {
      // Automatic span creation
      const span = trace.currentSpan()
      span.setAttribute('order.items.count', data.items.length)
      span.setAttribute('order.total', data.total)

      // Create order
      const order = await db.Orders.create(data)

      // Add span event
      span.addEvent('order.created', {
        'order.id': order.$id,
      })

      return order
    },
  },
})

Metrics Collection

Collect and expose service metrics:

const MetricsService = $.service({
  name: 'Metrics Service',

  metrics: {
    // Counter metrics
    requestsTotal: $.metric.counter({
      name: 'requests_total',
      description: 'Total number of requests',
      labels: ['method', 'path', 'status'],
    }),

    // Histogram metrics
    requestDuration: $.metric.histogram({
      name: 'request_duration_ms',
      description: 'Request duration in milliseconds',
      labels: ['method', 'path'],
      buckets: [10, 50, 100, 500, 1000, 5000],
    }),

    // Gauge metrics
    activeConnections: $.metric.gauge({
      name: 'active_connections',
      description: 'Number of active connections',
    }),

    // Summary metrics
    orderValue: $.metric.summary({
      name: 'order_value',
      description: 'Order value distribution',
      percentiles: [0.5, 0.9, 0.95, 0.99],
    }),
  },

  middleware: [
    // Automatic metrics collection
    async (c, next) => {
      const start = Date.now()

      MetricsService.metrics.activeConnections.inc()

      try {
        await next()

        const duration = Date.now() - start
        MetricsService.metrics.requestsTotal.inc({
          method: c.req.method,
          path: c.req.path,
          status: c.res.status,
        })

        MetricsService.metrics.requestDuration.observe(
          { method: c.req.method, path: c.req.path },
          duration
        )
      } finally {
        MetricsService.metrics.activeConnections.dec()
      }
    },
  ],
})

// Expose metrics endpoint
app.get('/metrics', async (c) => {
  const metrics = await MetricsService.collect()
  return c.text(metrics, 200, {
    'Content-Type': 'text/plain; version=0.0.4',
  })
})

Logging

Implement structured logging:

const LoggingService = $.service({
  name: 'Logging Service',

  logging: {
    level: env.LOG_LEVEL || 'info',
    format: 'json',

    // Log destinations
    destinations: [
      { type: 'console' },
      { type: 'file', path: '/var/log/app.log' },
      { type: 'cloudwatch', logGroup: '/app/production' },
    ],

    // Structured fields
    fields: {
      service: 'order-service',
      environment: env.ENVIRONMENT,
      version: env.VERSION,
    },

    // PII redaction
    redact: ['email', 'password', 'ssn', 'creditCard'],
  },

  operations: {
    log: (level: string, message: string, context?: any) => {
      const log = {
        timestamp: new Date().toISOString(),
        level,
        message,
        ...LoggingService.logging.fields,
        ...context,
      }

      // Redact sensitive fields
      const redacted = redactPII(log, LoggingService.logging.redact)

      // Send to destinations
      LoggingService.logging.destinations.forEach((dest) => {
        sendLog(dest, redacted)
      })
    },
  },
})

// Use in services
const OrderService = $.service({
  operations: {
    createOrder: async (data) => {
      LoggingService.log('info', 'Creating order', {
        customerId: data.customerId,
        itemCount: data.items.length,
        total: data.total,
      })

      try {
        const order = await db.Orders.create(data)

        LoggingService.log('info', 'Order created successfully', {
          orderId: order.$id,
        })

        return order
      } catch (error) {
        LoggingService.log('error', 'Failed to create order', {
          error: error.message,
          stack: error.stack,
          customerId: data.customerId,
        })

        throw error
      }
    },
  },
})

Next Steps

Data Modeling → - Design your data structures
Workflows → - Orchestrate services
Examples → - See complete architectures

Architecture Tip: Great service architectures are modular, semantic, and loosely coupled. Each service should be independently deployable and scalable.

Service Architecture

On this page