Fallback Chains for Unreliable External Services

The Three-Failure-Point Pipeline

function send_audio_with_fallback(string $recipient, string $text, string $correlationId): void
{
    $log = new StructuredLogger('/var/log/app/audio.log', $correlationId, IS_PROD);

    // Step 1: TTS generation
    try {
        $wavBytes = generate_tts($text);
        $log->info('tts.success', ['bytes' => strlen($wavBytes)]);
    } catch (\Exception $e) {
        $log->error('tts.failed', ['error' => $e->getMessage()]);
        // Fallback: send text directly
        send_whatsapp_text($recipient, $text);
        $log->info('text.fallback.sent');
        return;
    }

    // Step 2: Format conversion
    $tmpWav = $tmpOgg = null;
    try {
        $tmpWav = save_temp($wavBytes, '.wav');
        $tmpOgg = convert_wav_to_ogg($tmpWav);
        $log->info('conversion.success', ['ogg_bytes' => filesize($tmpOgg)]);
    } catch (\Exception $e) {
        $log->error('conversion.failed', ['error' => $e->getMessage()]);
        send_whatsapp_text($recipient, $text);
        $log->info('text.fallback.sent');
        return;
    } finally {
        if ($tmpWav && file_exists($tmpWav)) unlink($tmpWav);
    }

    // Step 3: Upload + send
    try {
        $mediaId = upload_to_meta($tmpOgg, 'audio/ogg; codecs=opus');
        $log->info('upload.success', ['media_id' => substr($mediaId, 0, 8) . '…']);
        send_whatsapp_audio($recipient, $mediaId);
        $log->info('audio.sent');
    } catch (\Exception $e) {
        $log->error('send.failed', ['error' => $e->getMessage()]);
        send_whatsapp_text($recipient, $text);
        $log->info('text.fallback.sent');
    } finally {
        if ($tmpOgg && file_exists($tmpOgg)) unlink($tmpOgg);
    }
}

Circuit Breaker Pattern

class CircuitBreaker
{
    private const FAILURE_THRESHOLD = 5;
    private const RESET_TIMEOUT_SEC = 300;

    public function isOpen(string $service, PDO $pdo): bool
    {
        $stmt = $pdo->prepare(
            'SELECT failure_count, last_failure_at FROM circuit_state WHERE service = ?'
        );
        $stmt->execute([$service]);
        $state = $stmt->fetch();

        if (!$state) return false;

        // If past reset timeout, close the circuit automatically
        $sinceLastFailure = time() - strtotime($state['last_failure_at']);
        if ($sinceLastFailure > self::RESET_TIMEOUT_SEC) {
            $this->reset($service, $pdo);
            return false;
        }

        return $state['failure_count'] >= self::FAILURE_THRESHOLD;
    }

    public function recordFailure(string $service, PDO $pdo): void
    {
        $pdo->prepare(
            'INSERT INTO circuit_state (service, failure_count, last_failure_at)
             VALUES (?, 1, NOW())
             ON DUPLICATE KEY UPDATE
               failure_count = failure_count + 1,
               last_failure_at = NOW()'
        )->execute([$service]);
    }
}

What to Watch For

• Per-step logging is the only way to know which step failed — Without step-level logs, "audio send failed" could mean TTS crashed, FFmpeg crashed, Meta upload failed, or the send call failed. Log every step.
• Text fallback is not optional — A pipeline that fails silently leaves the user waiting indefinitely. Always send a text fallback if audio fails.
• Circuit breaker state in the database — Redis is faster, but if Redis is down, you need the circuit breaker most. Storing circuit state in MySQL means the fallback mechanism works even during a cache outage.