Evaluate

Evaluate code quality, performance, and business metrics with automated analysis and reporting

Evaluate code quality, performance, security, and business impact with automated analysis, metrics, and continuous monitoring.

Overview

The Evaluate phase ensures your Business-as-Code meets quality standards across technical and business dimensions. Through automated analysis, metrics tracking, and continuous evaluation, you maintain high standards while moving fast.

This phase emphasizes:

Code Quality: Maintainability, complexity, test coverage
Performance: Speed, efficiency, resource usage
Security: Vulnerabilities, compliance, best practices
Business Impact: Revenue, user satisfaction, KPIs
Team Velocity: Deployment frequency, lead time, MTTR

Evaluation Primitives

Code Quality Metrics

Automated code quality analysis:

// Track code quality metrics
on($.Code.committed, async (commit, $) => {
  const analysis = await $.CodeQuality.analyze(commit, {
    metrics: ['complexity', 'duplication', 'coverage', 'maintainability'],
  })

  // Store metrics
  await db.create($.CodeMetrics, {
    commitId: commit.id,
    complexity: analysis.complexity,
    duplication: analysis.duplication,
    coverage: analysis.coverage,
    maintainability: analysis.maintainability,
    score: analysis.overall,
  })

  // Alert on quality issues
  if (analysis.overall < 70) {
    await send($.Alert.send, {
      type: 'code-quality',
      severity: 'medium',
      message: `Code quality score: ${analysis.overall}/100`,
      details: analysis,
    })
  }
})

// Track test coverage
on($.Tests.completed, async (results, $) => {
  await $.Metrics.record('test_coverage', {
    lines: results.coverage.lines,
    branches: results.coverage.branches,
    functions: results.coverage.functions,
  })

  if (results.coverage.lines < 80) {
    await send($.Alert.send, {
      type: 'low-coverage',
      message: `Test coverage: ${results.coverage.lines}% (target: 80%)`,
    })
  }
})

Performance Evaluation

Monitor performance metrics:

// Performance benchmarking
on($.Deploy.completed, async (deployment, $) => {
  // Run performance benchmarks
  const benchmarks = await $.Performance.benchmark({
    endpoints: ['/api/products', '/api/orders', '/api/users'],
    duration: '5 minutes',
    concurrency: 100,
  })

  // Compare with baseline
  for (const endpoint of benchmarks) {
    const baseline = await $.Performance.getBaseline(endpoint.path)

    const regression = ((endpoint.p95 - baseline.p95) / baseline.p95) * 100

    if (regression > 20) {
      await send($.Alert.send, {
        type: 'performance-regression',
        endpoint: endpoint.path,
        current: endpoint.p95,
        baseline: baseline.p95,
        regression: `${regression.toFixed(1)}%`,
      })
    }
  }
})

// Real-time performance tracking
on($.Request.completed, async (request, $) => {
  await $.Metrics.record('api_latency', {
    endpoint: request.path,
    method: request.method,
    duration: request.duration,
    status: request.status,
  })
})

Security Evaluation

Continuous security scanning:

// Dependency vulnerability scanning
on($.Dependencies.updated, async (changes, $) => {
  const vulnerabilities = await $.Security.scanDependencies({
    packages: changes.packages,
  })

  for (const vuln of vulnerabilities) {
    await db.create($.Vulnerability, {
      package: vuln.package,
      version: vuln.version,
      severity: vuln.severity,
      cve: vuln.cve,
      fixedIn: vuln.fixedIn,
    })

    if (vuln.severity === 'critical' || vuln.severity === 'high') {
      await send($.Alert.send, {
        type: 'security-vulnerability',
        severity: vuln.severity,
        package: vuln.package,
        cve: vuln.cve,
      })
    }
  }
})

// Code security analysis
on($.Code.committed, async (commit, $) => {
  const issues = await $.Security.scan(commit, {
    checks: ['sql-injection', 'xss', 'secrets', 'authentication'],
  })

  if (issues.length > 0) {
    await send($.Team.security, {
      type: 'security-issues-detected',
      commit: commit.id,
      issues,
    })
  }
})

Business Impact Evaluation

Measure business outcomes:

// Feature impact analysis
on($.Feature.deployed, async (feature, $) => {
  // Track for 30 days
  await $.Schedule.create({
    trigger: '+30 days',
    action: async () => {
      const impact = await $.Analytics.featureImpact({
        feature: feature.name,
        period: '30 days',
        metrics: ['adoption', 'engagement', 'revenue', 'retention'],
      })

      await db.create($.FeatureImpact, {
        featureId: feature.id,
        adoption: impact.adoption,
        engagement: impact.engagement,
        revenue: impact.revenue,
        retention: impact.retention,
        success: impact.adoption > 0.4 && impact.engagement > 0.6,
      })

      // Report to team
      await send($.Team.product, {
        type: 'feature-impact-report',
        feature: feature.name,
        impact,
      })
    },
  })
})

// Revenue impact tracking
on($.Experiment.graduated, async (experiment, $) => {
  const revenueImpact = await calculateRevenueImpact(experiment.id)

  await $.Metrics.record('experiment_revenue_impact', {
    experiment: experiment.name,
    incrementalRevenue: revenueImpact.incremental,
    roi: revenueImpact.roi,
  })
})

Team Velocity Metrics

Track development efficiency:

// DORA metrics tracking
const DORAMetrics = {
  // Deployment Frequency
  trackDeployment: async (deployment) => {
    await $.Metrics.increment('deployments', {
      environment: deployment.environment,
      timestamp: new Date(),
    })
  },

  // Lead Time for Changes
  trackLeadTime: async (pr) => {
    const leadTime = pr.mergedAt - pr.firstCommitAt

    await $.Metrics.record('lead_time', {
      duration: leadTime,
      pr: pr.number,
    })
  },

  // Mean Time to Recovery
  trackIncident: async (incident) => {
    if (incident.status === 'resolved') {
      const mttr = incident.resolvedAt - incident.createdAt

      await $.Metrics.record('mttr', {
        duration: mttr,
        incident: incident.id,
      })
    }
  },

  // Change Failure Rate
  trackDeploymentFailure: async (deployment) => {
    if (deployment.status === 'failed' || deployment.rolledBack) {
      await $.Metrics.increment('deployment_failures')
    }
  },
}

// Calculate DORA metrics
on($.Schedule.weekly, async () => {
  const dora = await calculateDORAMetrics()

  await send($.Dashboard.update, {
    dashboard: 'engineering',
    metrics: {
      deployment_frequency: dora.deploymentFrequency,
      lead_time: dora.leadTime,
      mttr: dora.mttr,
      change_failure_rate: dora.changeFailureRate,
      elite: dora.isElite, // Elite team: deploy daily, <1hr lead time, <1hr MTTR, <15% failure
    },
  })
})

Evaluation Dashboards

Engineering Health Dashboard

Comprehensive engineering metrics:

const engineeringDashboard = {
  quality: {
    testCoverage: await $.Metrics.current('test_coverage'),
    codeComplexity: await $.Metrics.average('code_complexity', '30d'),
    technicalDebt: await calculateTechnicalDebt(),
    bugs: await $.Metrics.count('bugs', { status: 'open' }),
  },

  performance: {
    p50Latency: await $.Metrics.percentile('api_latency', 50),
    p95Latency: await $.Metrics.percentile('api_latency', 95),
    p99Latency: await $.Metrics.percentile('api_latency', 99),
    errorRate: await $.Metrics.rate('errors', '24h'),
    uptime: await $.Metrics.uptime('30d'),
  },

  velocity: {
    deploymentsPerDay: await $.Metrics.rate('deployments', '7d'),
    leadTime: await $.Metrics.median('lead_time', '30d'),
    cycleTime: await $.Metrics.median('cycle_time', '30d'),
    throughput: await $.Metrics.count('prs_merged', '7d'),
  },

  security: {
    vulnerabilities: await $.Metrics.count('vulnerabilities'),
    criticalVulns: await $.Metrics.count('vulnerabilities', { severity: 'critical' }),
    daysToRemediate: await $.Metrics.average('remediation_time'),
  },
}

Business Health Dashboard

Business outcome metrics:

const businessDashboard = {
  growth: {
    mrr: await $.Metrics.current('mrr'),
    arr: await $.Metrics.current('arr'),
    growth: await $.Metrics.growth('mrr', '30d'),
    newCustomers: await $.Metrics.count('customers', { period: '30d', status: 'new' }),
  },

  retention: {
    churn: await $.Metrics.rate('churn', '30d'),
    retention: await $.Metrics.rate('retention', '30d'),
    nps: await $.Metrics.average('nps'),
    csat: await $.Metrics.average('csat'),
  },

  product: {
    dau: await $.Metrics.count('active_users', '1d'),
    mau: await $.Metrics.count('active_users', '30d'),
    engagement: await $.Metrics.average('engagement_score'),
    activation: await $.Metrics.rate('activation'),
  },

  economics: {
    ltv: await $.Metrics.average('customer_ltv'),
    cac: await $.Metrics.average('customer_cac'),
    ltvCacRatio: await $.Metrics.ratio('ltv', 'cac'),
    payback: await $.Metrics.median('cac_payback_months'),
  },
}

Automated Quality Gates

Block deployments that don't meet standards:

// Quality gate checks
const qualityGates = {
  preDeployment: async (deployment) => {
    const checks = await Promise.all([
      // Test coverage
      checkTestCoverage({ minimum: 80 }),

      // No critical bugs
      checkOpenBugs({ severity: ['critical', 'high'], max: 0 }),

      // Performance benchmarks
      checkPerformance({ p95Threshold: 200 }),

      // Security scan
      checkSecurity({ allowedSeverities: ['low', 'medium'] }),

      // Code quality
      checkCodeQuality({ minimumScore: 70 }),
    ])

    const passed = checks.every((c) => c.passed)

    if (!passed) {
      const failures = checks.filter((c) => !c.passed)

      await send($.Alert.send, {
        type: 'quality-gate-failed',
        deployment: deployment.id,
        failures,
      })

      // Block deployment
      return { allowed: false, failures }
    }

    return { allowed: true }
  },
}

// Apply quality gates
on($.Deployment.requested, async (deployment, $) => {
  const result = await qualityGates.preDeployment(deployment)

  if (!result.allowed) {
    await db.update($.Deployment, deployment.id, {
      status: 'blocked',
      blockReason: 'quality-gates-failed',
      failures: result.failures,
    })
  } else {
    await send($.Deployment.approved, deployment)
  }
})

Best Practices

Do's

Automate evaluation - Continuous, not manual
Track trends - Not just snapshots
Set clear thresholds - Know what's acceptable
Act on insights - Metrics without action are waste
Balance dimensions - Quality, speed, and cost
Celebrate wins - Recognize improvements

Don'ts

Don't track vanity metrics - Focus on actionable data
Don't ignore context - Metrics need interpretation
Don't optimize locally - Consider system-wide impact
Don't punish with metrics - Use for learning
Don't set too many metrics - Focus on key indicators

CLI Tools

# Evaluate code quality
do evaluate quality

# Run performance benchmarks
do evaluate performance

# Security scan
do evaluate security

# Business metrics
do evaluate business --period 30d

# Team velocity
do evaluate velocity --dora

# Generate full report
do evaluate report --comprehensive

Next Steps

Deploy → - Deploy evaluated code
Manage → - Monitor in production
Iterate → - Improve based on evaluation

Evaluation Tip: You improve what you measure. Choose metrics that drive the behavior you want.

Evaluate

On this page