Data Modeling

Data modeling on the .do platform leverages semantic types from established vocabularies to create rich, meaningful data structures that carry business context.

Semantic Data Modeling

Traditional data models focus on structure (tables, columns, indexes). Semantic data models focus on meaning:

// Traditional approach
interface User {
  id: string
  name: string
  email: string
}

// Semantic approach
import { Person } from 'schema.org.ai'

const user: Person = {
  $type: 'Person',
  $id: 'person-123',
  name: 'John Doe',
  email: '[email protected]',
  jobTitle: 'Software Engineer',
  worksFor: {
    $type: 'Organization',
    name: 'Acme Inc',
  },
}

The semantic approach provides:

Type safety: TypeScript knows all valid properties
Context: Data carries its own meaning
Interoperability: Aligns with external systems
Extensibility: Easy to add new properties

Vocabulary Sources

Schema.org

The most comprehensive vocabulary with 817 types and 1,518 properties:

import { Person, Organization, Product, Order, Invoice, Event, Place, CreativeWork } from 'schema.org.ai'

// E-commerce product
const product: Product = {
  $type: 'Product',
  $id: 'product-123',
  name: 'Wireless Headphones',
  description: 'Premium noise-cancelling headphones',
  brand: { $type: 'Brand', name: 'AudioTech' },
  offers: {
    $type: 'Offer',
    price: 299.99,
    priceCurrency: 'USD',
    availability: 'InStock',
  },
  aggregateRating: {
    $type: 'AggregateRating',
    ratingValue: 4.5,
    reviewCount: 1247,
  },
}

NAICS (Industry Classifications)

1,170 industry codes for business classification:

import { NAICS } from 'naics.org.ai'
import { Organization } from 'schema.org.ai'

const business: Organization = {
  $type: 'Organization',
  name: 'Acme Software Inc',
  industry: NAICS.Software_Publishers_511210,
  naicsCode: '511210',
  description: 'Custom software development',
}

O*NET (Occupations and Skills)

923 occupation types and associated skills:

import { Occupation } from 'onet.org.ai'
import { Person } from 'schema.org.ai'

const employee: Person = {
  $type: 'Person',
  name: 'Jane Smith',
  hasOccupation: {
    $type: 'Occupation',
    name: 'Software Developer',
    onetCode: '15-1252.00',
    skills: ['JavaScript', 'TypeScript', 'React'],
    experienceLevel: 'Senior',
  },
}

Custom Vocabularies

Create domain-specific vocabularies:

// Define custom types
export interface Subscription {
  $type: 'Subscription'
  $id: string
  plan: 'free' | 'pro' | 'enterprise'
  status: 'active' | 'cancelled' | 'expired'
  customer: Person
  startDate: Date
  endDate: Date
  mrr: number
}

// Use in your application
const subscription: Subscription = {
  $type: 'Subscription',
  $id: 'sub-123',
  plan: 'pro',
  status: 'active',
  customer: { $type: 'Person', name: 'John Doe' },
  startDate: new Date('2024-01-01'),
  endDate: new Date('2025-01-01'),
  mrr: 99,
}

Data Relationships

One-to-One

// User has one profile
const user: Person = {
  $type: 'Person',
  $id: 'user-123',
  name: 'John Doe',
  profile: {
    $type: 'ProfilePage',
    $id: 'profile-123',
    bio: 'Software engineer and entrepreneur',
    avatar: 'https://example.com/avatar.jpg',
  },
}

One-to-Many

// Organization has many employees
const organization: Organization = {
  $type: 'Organization',
  $id: 'org-123',
  name: 'Acme Inc',
  employees: [
    { $type: 'Person', $id: 'person-1', name: 'Alice' },
    { $type: 'Person', $id: 'person-2', name: 'Bob' },
    { $type: 'Person', $id: 'person-3', name: 'Carol' },
  ],
}

// Or reference by ID
const organization: Organization = {
  $type: 'Organization',
  $id: 'org-123',
  name: 'Acme Inc',
  employee: ['person-1', 'person-2', 'person-3'], // References
}

Many-to-Many

// Products have many tags, tags have many products
const product: Product = {
  $type: 'Product',
  $id: 'product-123',
  name: 'Laptop',
  keywords: ['electronics', 'computers', 'productivity'], // Tag references
}

// Query products by tag
const electronicsProducts = await db.list($.Product, {
  where: { keywords: { $contains: 'electronics' } },
})

Hierarchical

// Organization hierarchy
const company: Organization = {
  $type: 'Organization',
  $id: 'company',
  name: 'Acme Corp',
  subOrganization: [
    {
      $type: 'Organization',
      $id: 'engineering',
      name: 'Engineering',
      parentOrganization: 'company',
      subOrganization: [
        {
          $type: 'Organization',
          $id: 'frontend',
          name: 'Frontend Team',
          parentOrganization: 'engineering',
        },
        {
          $type: 'Organization',
          $id: 'backend',
          name: 'Backend Team',
          parentOrganization: 'engineering',
        },
      ],
    },
  ],
}

Data Operations

Create

// Create new entity
const user = await db.create($.Person, {
  name: 'John Doe',
  email: '[email protected]',
  jobTitle: 'Engineer',
})

Read

// Get by ID
const user = await db.get($.Person, 'user-123')

// Query with filters
const users = await db.list($.Person, {
  where: {
    jobTitle: 'Engineer',
    worksFor: { name: 'Acme Inc' },
  },
  limit: 10,
  orderBy: { name: 'asc' },
})

// Find one
const user = await db.findOne($.Person, {
  where: { email: '[email protected]' },
})

Update

// Update entity
const updated = await db.update($.Person, 'user-123', {
  jobTitle: 'Senior Engineer',
  salary: 120000,
})

// Partial update
const updated = await db.patch($.Person, 'user-123', {
  jobTitle: 'Senior Engineer',
})

Delete

// Delete by ID
await db.delete($.Person, 'user-123')

// Delete with conditions
await db.deleteMany($.Person, {
  where: { status: 'inactive', lastActive: { $lt: oneYearAgo } },
})

Data Validation

Schema Validation

import { z } from 'zod'

const PersonSchema = z.object({
  $type: z.literal('Person'),
  name: z.string().min(1).max(100),
  email: z.string().email(),
  age: z.number().min(0).max(120).optional(),
  jobTitle: z.string().optional(),
})

// Validate before creating
const validated = PersonSchema.parse(data)
const user = await db.create($.Person, validated)

Business Rules

// Custom validation rules
const validateOrder = (order: Order) => {
  if (order.items.length === 0) {
    throw new Error('Order must have at least one item')
  }

  const total = order.items.reduce((sum, item) => sum + item.price * item.quantity, 0)
  if (total !== order.total) {
    throw new Error('Order total does not match items')
  }

  if (order.customer.age < 18 && order.items.some((item) => item.ageRestriction === 18)) {
    throw new Error('Customer not old enough for restricted items')
  }

  return true
}

Data Indexing

Primary Indexes

// Define indexes for performance
const Users = db.collection<Person>('users', {
  indexes: {
    email: { unique: true },
    jobTitle: {},
    'worksFor.name': {},
  },
})

// Queries use indexes automatically
const users = await Users.find({
  where: { jobTitle: 'Engineer' }, // Uses index
})

Composite Indexes

const Orders = db.collection<Order>('orders', {
  indexes: {
    // Composite index for common query pattern
    'customer-status-date': {
      fields: ['customer.id', 'status', 'orderDate'],
      unique: false,
    },
  },
})

// Efficient query with composite index
const orders = await Orders.find({
  where: {
    'customer.id': 'user-123',
    status: 'completed',
    orderDate: { $gte: lastMonth },
  },
})

Data Migrations

Version Control

// migrations/001-add-user-profile.ts
export const up = async (db) => {
  const users = await db.list($.Person)

  for (const user of users) {
    await db.update($.Person, user.$id, {
      profile: {
        $type: 'ProfilePage',
        bio: '',
        avatar: null,
      },
    })
  }
}

export const down = async (db) => {
  const users = await db.list($.Person)

  for (const user of users) {
    await db.update($.Person, user.$id, {
      profile: undefined,
    })
  }
}

Schema Evolution

// Handle schema changes gracefully
const user = await db.get($.Person, 'user-123')

// Old schema: { name: string }
// New schema: { givenName: string, familyName: string }

if (user.name && !user.givenName) {
  // Migrate old format
  const [givenName, familyName] = user.name.split(' ')
  await db.update($.Person, 'user-123', {
    givenName,
    familyName,
    name: undefined, // Remove old field
  })
}

Data Privacy

PII Protection

// Mark sensitive fields
const User = db.collection<Person>('users', {
  schema: {
    name: { type: 'string', pii: true },
    email: { type: 'string', pii: true },
    ssn: { type: 'string', pii: true, encrypted: true },
    phone: { type: 'string', pii: true },
  },
})

// Automatic PII handling
const user = await db.get($.Person, 'user-123', {
  includePII: false, // Masks PII fields
})

Data Retention

// Set retention policies
const Logs = db.collection('logs', {
  retention: {
    duration: '90 days',
    action: 'delete',
  },
})

// Automatic cleanup after 90 days

Data Caching

Automatic Caching

// Enable caching for collection
const Products = db.collection<Product>('products', {
  cache: {
    enabled: true,
    ttl: 3600, // 1 hour
    invalidateOn: ['create', 'update', 'delete'],
  },
})

// Cached automatically
const product = await Products.get('product-123')

Manual Cache Control

// Explicit cache management
import { cache } from 'sdk.do'

// Set cache
await cache.set('products:featured', featuredProducts, { ttl: 3600 })

// Get from cache
const cached = await cache.get('products:featured')

// Invalidate cache
await cache.delete('products:featured')

Best Practices

Do's

Use established vocabularies (Schema.org, NAICS, O*NET)
Define clear relationships between entities
Validate data at boundaries
Index frequently queried fields
Version your schema changes
Protect sensitive data
Cache appropriately

Don'ts

Don't create redundant types
Don't ignore data validation
Don't skip indexing
Don't hard-code relationships
Don't expose PII unnecessarily
Don't over-normalize data
Don't forget about performance

Advanced Data Patterns

Polymorphic Types

Handle multiple types with union types:

import { Person, Organization } from 'schema.org.ai'

// Customer can be Person or Organization
type Customer = Person | Organization

interface Order {
  $type: 'Order'
  $id: string
  customer: Customer // Polymorphic field
  items: OrderItem[]
  total: number
}

// Type guards for polymorphism
function isPersonCustomer(customer: Customer): customer is Person {
  return customer.$type === 'Person'
}

function isOrganizationCustomer(customer: Customer): customer is Organization {
  return customer.$type === 'Organization'
}

// Use with type guards
const order = await db.Orders.get('order-123')

if (isPersonCustomer(order.customer)) {
  console.log(`Customer: ${order.customer.name}`)
  console.log(`Email: ${order.customer.email}`)
} else if (isOrganizationCustomer(order.customer)) {
  console.log(`Organization: ${order.customer.name}`)
  console.log(`Tax ID: ${order.customer.taxID}`)
}

Embedded Documents

Embed related data for performance:

interface BlogPost {
  $type: 'BlogPost'
  $id: string
  title: string
  content: string
  author: Person // Embedded author data
  comments: Comment[] // Embedded comments
  tags: string[]
  publishedAt: Date
}

interface Comment {
  $id: string
  author: Person // Embedded author in comment
  text: string
  createdAt: Date
  likes: number
}

// Create with embedded data
const post = await db.BlogPosts.create({
  title: 'Getting Started with Business-as-Code',
  content: '...',
  author: {
    $type: 'Person',
    $id: 'person-123',
    name: 'John Doe',
    email: '[email protected]',
    avatar: 'https://example.com/avatar.jpg',
  },
  comments: [
    {
      $id: 'comment-1',
      author: {
        $type: 'Person',
        $id: 'person-456',
        name: 'Jane Smith',
      },
      text: 'Great article!',
      createdAt: new Date(),
      likes: 5,
    },
  ],
  tags: ['business-as-code', 'tutorial'],
  publishedAt: new Date(),
})

// Query with embedded data (single read)
const post = await db.BlogPosts.get('post-123')
console.log(post.author.name) // No additional query needed
console.log(post.comments.length) // No additional query needed

References vs Embedding

Choose between references and embedding:

// Embedded (denormalized) - Better for read performance
interface OrderEmbedded {
  $type: 'Order'
  $id: string
  customer: {
    // Full customer data embedded
    $type: 'Person'
    $id: string
    name: string
    email: string
    address: Address
  }
  product: {
    // Full product data embedded
    $type: 'Product'
    $id: string
    name: string
    price: number
    description: string
  }
}

// Referenced (normalized) - Better for write performance and consistency
interface OrderReferenced {
  $type: 'Order'
  $id: string
  customerId: string // Reference to customer
  productId: string // Reference to product
}

// Hybrid approach - Embed critical fields, reference for details
interface OrderHybrid {
  $type: 'Order'
  $id: string
  customerId: string // Reference for full data
  customerName: string // Embedded for display
  customerEmail: string // Embedded for notifications
  productId: string // Reference for full data
  productName: string // Embedded for display
  productPrice: number // Embedded for immutability
}

// Query with population
const order = await db.Orders.get('order-123', {
  populate: ['customer', 'product'], // Automatically fetch references
})

Time-Series Data

Model time-series data efficiently:

interface Metric {
  $type: 'Metric'
  $id: string
  name: string
  value: number
  timestamp: Date
  labels: Record<string, string>
}

// Collection with time-series optimization
const Metrics = db.collection<Metric>('metrics', {
  indexes: {
    // Compound index for time-series queries
    'name-timestamp': {
      fields: ['name', 'timestamp'],
      unique: false,
    },
    // TTL index for automatic cleanup
    timestamp: {
      expireAfterSeconds: 2592000, // 30 days
    },
  },

  // Time-series specific options
  timeseries: {
    timeField: 'timestamp',
    metaField: 'labels',
    granularity: 'minutes',
  },
})

// Efficient time-series queries
const metrics = await Metrics.find({
  where: {
    name: 'cpu_usage',
    timestamp: {
      $gte: new Date('2024-01-01'),
      $lte: new Date('2024-01-31'),
    },
    'labels.host': 'server-1',
  },
  orderBy: { timestamp: 'asc' },
})

// Aggregation pipelines for analytics
const hourlyAverages = await Metrics.aggregate([
  {
    $match: {
      name: 'cpu_usage',
      timestamp: { $gte: new Date('2024-01-01') },
    },
  },
  {
    $group: {
      _id: {
        $dateToString: { format: '%Y-%m-%d %H:00', date: '$timestamp' },
      },
      avg: { $avg: '$value' },
      max: { $max: '$value' },
      min: { $min: '$value' },
    },
  },
  {
    $sort: { _id: 1 },
  },
])

Audit Trails

Implement comprehensive audit trails:

interface Auditable {
  createdAt: Date
  createdBy: string
  updatedAt: Date
  updatedBy: string
  version: number
  history: AuditEntry[]
}

interface AuditEntry {
  timestamp: Date
  user: string
  action: 'create' | 'update' | 'delete'
  changes: FieldChange[]
  ip: string
  userAgent: string
}

interface FieldChange {
  field: string
  oldValue: any
  newValue: any
}

// Auditable entity
interface AuditableOrder extends Order, Auditable {
  $type: 'Order'
}

// Automatic audit trail middleware
const auditMiddleware = async (collection: string, operation: string, data: any, context: any) => {
  const auditEntry: AuditEntry = {
    timestamp: new Date(),
    user: context.userId,
    action: operation as any,
    changes: [],
    ip: context.ip,
    userAgent: context.userAgent,
  }

  if (operation === 'update') {
    // Calculate changes
    const current = await db[collection].get(data.$id)
    auditEntry.changes = calculateChanges(current, data)
  }

  // Add audit entry
  data.history = data.history || []
  data.history.push(auditEntry)
  data.updatedAt = new Date()
  data.updatedBy = context.userId
  data.version = (data.version || 0) + 1

  return data
}

// Usage
const updated = await db.Orders.update('order-123', updates, {
  context: { userId: 'user-456', ip: '192.168.1.1', userAgent: 'Mozilla...' },
})

// Query audit history
const order = await db.Orders.get('order-123')
console.log('Order history:', order.history)

// Find who made specific change
const statusChange = order.history.find((entry) => entry.changes.some((change) => change.field === 'status'))

Soft Deletes

Implement soft delete pattern:

interface SoftDeletable {
  deleted: boolean
  deletedAt?: Date
  deletedBy?: string
}

interface SoftDeletableUser extends Person, SoftDeletable {
  $type: 'Person'
}

// Collection with soft delete support
const Users = db.collection<SoftDeletableUser>('users', {
  defaultFilters: {
    deleted: false, // Automatically filter deleted records
  },
})

// Soft delete
const softDelete = async (userId: string, context: any) => {
  return await Users.update(userId, {
    deleted: true,
    deletedAt: new Date(),
    deletedBy: context.userId,
  })
}

// Hard delete
const hardDelete = async (userId: string) => {
  return await Users.delete(userId, { force: true })
}

// Query including deleted
const allUsers = await Users.find({
  includeDeleted: true,
})

// Query only deleted
const deletedUsers = await Users.find({
  where: { deleted: true },
})

// Restore deleted
const restore = async (userId: string) => {
  return await Users.update(userId, {
    deleted: false,
    deletedAt: null,
    deletedBy: null,
  })
}

Multi-Tenancy

Model multi-tenant data:

interface Tenant {
  $type: 'Tenant'
  $id: string
  name: string
  domain: string
  settings: TenantSettings
  status: 'active' | 'suspended' | 'trial'
}

interface TenantSettings {
  theme: string
  features: string[]
  limits: {
    users: number
    storage: number
    apiCalls: number
  }
}

// Tenant-scoped entity
interface TenantScoped {
  tenantId: string
}

interface TenantScopedOrder extends Order, TenantScoped {
  $type: 'Order'
}

// Tenant-aware collection
const Orders = db.collection<TenantScopedOrder>('orders', {
  tenantField: 'tenantId',

  indexes: {
    // Compound index with tenant
    'tenant-customer-date': {
      fields: ['tenantId', 'customerId', 'orderDate'],
    },
  },
})

// Automatic tenant scoping
const context = { tenantId: 'tenant-123' }

// All queries automatically scoped to tenant
const orders = await Orders.find({}, { context })

// Create with automatic tenant assignment
const order = await Orders.create(
  {
    customerId: 'customer-456',
    items: [],
    total: 0,
  },
  { context }
)

// Tenant isolation enforcement
const crossTenantQuery = await Orders.find(
  {
    where: { tenantId: 'other-tenant' },
  },
  { context }
)
// Returns empty - tenant isolation enforced

Data Quality

Validation Rules

Implement comprehensive validation:

import { z } from 'zod'

// Complex validation schemas
const EmailSchema = z.string().email().refine(
  async (email) => {
    // Check if email exists
    const existing = await db.Users.findOne({ where: { email } })
    return !existing
  },
  { message: 'Email already exists' }
)

const PhoneSchema = z
  .string()
  .regex(/^\+[1-9]\d{1,14}$/, 'Must be valid E.164 format')
  .refine(async (phone) => {
    // Verify phone number via API
    return await verifyPhoneNumber(phone)
  }, 'Invalid phone number')

const PersonSchema = z.object({
  $type: z.literal('Person'),
  name: z.string().min(1).max(100),
  email: EmailSchema,
  phone: PhoneSchema.optional(),
  age: z.number().int().min(0).max(120),
  address: z.object({
    street: z.string(),
    city: z.string(),
    state: z.string().length(2),
    zip: z.string().regex(/^\d{5}(-\d{4})?$/),
    country: z.string().length(2),
  }),
  preferences: z.object({
    newsletter: z.boolean(),
    notifications: z.boolean(),
  }),
})

// Cross-field validation
const OrderSchema = z
  .object({
    $type: z.literal('Order'),
    items: z.array(OrderItemSchema).min(1),
    subtotal: z.number().positive(),
    tax: z.number().nonnegative(),
    shipping: z.number().nonnegative(),
    total: z.number().positive(),
  })
  .refine(
    (order) => {
      const calculatedTotal = order.subtotal + order.tax + order.shipping
      return Math.abs(calculatedTotal - order.total) < 0.01
    },
    { message: 'Total does not match subtotal + tax + shipping' }
  )

// Conditional validation
const SubscriptionSchema = z
  .object({
    plan: z.enum(['free', 'pro', 'enterprise']),
    price: z.number().nonnegative(),
    features: z.array(z.string()),
  })
  .refine(
    (sub) => {
      if (sub.plan === 'free' && sub.price > 0) return false
      if (sub.plan !== 'free' && sub.price === 0) return false
      return true
    },
    { message: 'Price must match plan type' }
  )

Data Sanitization

Clean and normalize data:

// Sanitization utilities
const sanitize = {
  email: (email: string) => email.toLowerCase().trim(),

  phone: (phone: string) => {
    // Convert to E.164 format
    return phone.replace(/\D/g, '').replace(/^(\d)/, '+$1')
  },

  text: (text: string) => {
    // Remove HTML tags, trim whitespace
    return text.replace(/<[^>]*>/g, '').trim()
  },

  name: (name: string) => {
    // Capitalize words, remove extra spaces
    return name
      .trim()
      .replace(/\s+/g, ' ')
      .split(' ')
      .map((word) => word.charAt(0).toUpperCase() + word.slice(1).toLowerCase())
      .join(' ')
  },

  url: (url: string) => {
    // Ensure protocol, remove trailing slash
    let sanitized = url.trim()
    if (!sanitized.match(/^https?:\/\//)) {
      sanitized = `https://${sanitized}`
    }
    return sanitized.replace(/\/$/, '')
  },
}

// Automatic sanitization middleware
const sanitizeMiddleware = (schema: any) => {
  return async (data: any) => {
    const sanitized = { ...data }

    if (sanitized.email) sanitized.email = sanitize.email(sanitized.email)
    if (sanitized.phone) sanitized.phone = sanitize.phone(sanitized.phone)
    if (sanitized.name) sanitized.name = sanitize.name(sanitized.name)
    if (sanitized.url) sanitized.url = sanitize.url(sanitized.url)

    return sanitized
  }
}

// Use in collection
const Users = db.collection<Person>('users', {
  beforeCreate: [sanitizeMiddleware(PersonSchema)],
  beforeUpdate: [sanitizeMiddleware(PersonSchema)],
})

Data Enrichment

Automatically enrich data:

// Enrichment pipeline
const enrichPerson = async (person: Person): Promise<Person> => {
  const enriched = { ...person }

  // Enrich from email
  if (person.email) {
    const emailData = await enrichFromEmail(person.email)
    enriched.avatar = emailData.avatar || enriched.avatar
    enriched.linkedin = emailData.linkedin || enriched.linkedin
  }

  // Enrich from domain
  if (person.email) {
    const domain = person.email.split('@')[1]
    const companyData = await enrichFromDomain(domain)
    enriched.worksFor = companyData.organization
  }

  // Geocode address
  if (person.address) {
    const geocode = await geocodeAddress(person.address)
    enriched.location = {
      $type: 'GeoCoordinates',
      latitude: geocode.lat,
      longitude: geocode.lng,
    }
  }

  // Calculate derived fields
  if (person.birthDate) {
    enriched.age = calculateAge(person.birthDate)
  }

  return enriched
}

// Automatic enrichment
const Users = db.collection<Person>('users', {
  afterCreate: [enrichPerson],
  afterUpdate: [enrichPerson],
})

Data Consistency

Transactions

Ensure atomic operations:

// Multi-document transaction
const transferFunds = async (fromAccount: string, toAccount: string, amount: number) => {
  const session = await db.startSession()

  try {
    await session.withTransaction(async () => {
      // Debit from account
      await db.Accounts.update(
        fromAccount,
        {
          balance: { $decrement: amount },
        },
        { session }
      )

      // Credit to account
      await db.Accounts.update(
        toAccount,
        {
          balance: { $increment: amount },
        },
        { session }
      )

      // Create transaction record
      await db.Transactions.create(
        {
          from: fromAccount,
          to: toAccount,
          amount,
          timestamp: new Date(),
        },
        { session }
      )
    })

    return { success: true }
  } catch (error) {
    // Transaction automatically rolled back
    return { success: false, error }
  } finally {
    await session.endSession()
  }
}

Optimistic Locking

Prevent concurrent modification conflicts:

interface Versioned {
  version: number
}

interface VersionedOrder extends Order, Versioned {}

// Update with version check
const updateOrder = async (orderId: string, updates: any, expectedVersion: number) => {
  const result = await db.Orders.updateOne(
    {
      $id: orderId,
      version: expectedVersion, // Only update if version matches
    },
    {
      ...updates,
      version: expectedVersion + 1, // Increment version
      updatedAt: new Date(),
    }
  )

  if (result.matchedCount === 0) {
    throw new Error('Order was modified by another process. Please refresh and try again.')
  }

  return result
}

// Retry logic for conflicts
const updateWithRetry = async (orderId: string, updateFn: Function, maxRetries = 3) => {
  for (let attempt = 0; attempt < maxRetries; attempt++) {
    const order = await db.Orders.get(orderId)

    try {
      const updates = await updateFn(order)
      return await updateOrder(orderId, updates, order.version)
    } catch (error) {
      if (error.message.includes('modified by another process') && attempt < maxRetries - 1) {
        // Retry with fresh data
        continue
      }
      throw error
    }
  }
}

Pessimistic Locking

Lock records during updates:

// Acquire exclusive lock
const withLock = async (key: string, fn: Function, timeout = 30000) => {
  const lockId = await acquireLock(key, timeout)

  try {
    return await fn()
  } finally {
    await releaseLock(key, lockId)
  }
}

// Use lock for critical operations
const updateInventory = async (productId: string, quantity: number) => {
  return await withLock(`product:${productId}`, async () => {
    const product = await db.Products.get(productId)

    if (product.inventory < quantity) {
      throw new Error('Insufficient inventory')
    }

    return await db.Products.update(productId, {
      inventory: product.inventory - quantity,
    })
  })
}

Data Access Patterns

Repository Pattern

Encapsulate data access logic:

class UserRepository {
  private collection = db.collection<Person>('users')

  async create(data: Partial<Person>): Promise<Person> {
    const validated = PersonSchema.parse(data)
    return await this.collection.create(validated)
  }

  async findById(id: string): Promise<Person | null> {
    return await this.collection.get(id)
  }

  async findByEmail(email: string): Promise<Person | null> {
    return await this.collection.findOne({ where: { email } })
  }

  async findActive(): Promise<Person[]> {
    return await this.collection.find({
      where: { status: 'active', deleted: false },
    })
  }

  async search(query: string): Promise<Person[]> {
    return await this.collection.find({
      where: {
        $or: [{ name: { $regex: query, $options: 'i' } }, { email: { $regex: query, $options: 'i' } }],
      },
    })
  }

  async update(id: string, updates: Partial<Person>): Promise<Person> {
    return await this.collection.update(id, updates)
  }

  async delete(id: string): Promise<void> {
    await this.collection.delete(id)
  }

  // Domain-specific methods
  async activateUser(id: string): Promise<Person> {
    return await this.update(id, {
      status: 'active',
      activatedAt: new Date(),
    })
  }

  async getUserStats(id: string) {
    const user = await this.findById(id)
    const orders = await db.Orders.count({ where: { customerId: id } })
    const totalSpent = await db.Orders.aggregate([
      { $match: { customerId: id } },
      { $group: { _id: null, total: { $sum: '$total' } } },
    ])

    return {
      user,
      orderCount: orders,
      totalSpent: totalSpent[0]?.total || 0,
    }
  }
}

// Use repository
const userRepo = new UserRepository()
const user = await userRepo.findByEmail('[email protected]')
const stats = await userRepo.getUserStats(user.$id)

Query Builder

Build complex queries programmatically:

class QueryBuilder<T> {
  private filters: any[] = []
  private sortFields: any[] = []
  private limitValue?: number
  private skipValue?: number
  private populateFields: string[] = []

  where(field: string, operator: string, value: any): this {
    this.filters.push({ field, operator, value })
    return this
  }

  orWhere(field: string, operator: string, value: any): this {
    this.filters.push({ field, operator, value, or: true })
    return this
  }

  sort(field: string, direction: 'asc' | 'desc' = 'asc'): this {
    this.sortFields.push({ field, direction })
    return this
  }

  limit(n: number): this {
    this.limitValue = n
    return this
  }

  skip(n: number): this {
    this.skipValue = n
    return this
  }

  populate(...fields: string[]): this {
    this.populateFields.push(...fields)
    return this
  }

  async execute(collection: any): Promise<T[]> {
    // Build query object
    const query: any = { where: {} }

    // Apply filters
    const andFilters: any[] = []
    const orFilters: any[] = []

    for (const filter of this.filters) {
      const condition = this.buildCondition(filter)
      if (filter.or) {
        orFilters.push(condition)
      } else {
        andFilters.push(condition)
      }
    }

    if (andFilters.length > 0) {
      query.where.$and = andFilters
    }

    if (orFilters.length > 0) {
      query.where.$or = orFilters
    }

    // Apply sorting
    if (this.sortFields.length > 0) {
      query.orderBy = this.sortFields.reduce(
        (acc, { field, direction }) => ({
          ...acc,
          [field]: direction,
        }),
        {}
      )
    }

    // Apply pagination
    if (this.limitValue) query.limit = this.limitValue
    if (this.skipValue) query.skip = this.skipValue

    // Apply population
    if (this.populateFields.length > 0) {
      query.populate = this.populateFields
    }

    return await collection.find(query)
  }

  private buildCondition(filter: any) {
    const { field, operator, value } = filter

    switch (operator) {
      case '=':
        return { [field]: value }
      case '!=':
        return { [field]: { $ne: value } }
      case '>':
        return { [field]: { $gt: value } }
      case '>=':
        return { [field]: { $gte: value } }
      case '<':
        return { [field]: { $lt: value } }
      case '<=':
        return { [field]: { $lte: value } }
      case 'in':
        return { [field]: { $in: value } }
      case 'contains':
        return { [field]: { $regex: value, $options: 'i' } }
      default:
        return { [field]: value }
    }
  }
}

// Use query builder
const query = new QueryBuilder<Order>()
  .where('status', '=', 'completed')
  .where('total', '>', 100)
  .orWhere('priority', '=', 'high')
  .sort('orderDate', 'desc')
  .limit(10)
  .populate('customer', 'items')

const orders = await query.execute(db.Orders)

Next Steps

Workflows → - Design business processes
Semantic Patterns → - Master the patterns
Examples → - See complete data models

Data Modeling Tip: Great data models are semantic, validated, and performant. Use established vocabularies whenever possible.

Data Modeling

On this page