🎯 Retry / Fallback Agent
1️⃣ Core Framework
When discussing Retry / Fallback Agent, I frame it as an AI-wrapped LeetCode pattern: recover from model, tool, retrieval, or infrastructure failures without looping forever or causing cascading failure.
The core is failure classification, retry policy, fallback path, idempotency, circuit breaker, and graceful degradation.
I usually cover it in this order:
- problem definition
- LeetCode pattern mapping
- core algorithm
- production architecture
- scaling and latency
- failure handling
- evaluation and observability
- Staff-level trade-offs
👉 Interview Answer
For Retry / Fallback Agent, I would first translate the AI behavior into a concrete algorithmic problem. The baseline is failure classification, retry policy, fallback path, idempotency, circuit breaker, and graceful degradation. Then I would explain how that algorithm changes in production when we add latency budgets, permissions, versioning, evaluation, and failure handling. That gives both a clean coding solution and a Staff-level system design answer.
2️⃣ What Problem Are We Solving?
The system must recover from model, tool, retrieval, or infrastructure failures without looping forever or causing cascading failure.
In coding-interview language, this means:
- define the input and output clearly
- choose the right data structure
- keep complexity under control
- handle edge cases explicitly
- explain why the algorithm is correct
- then extend the solution to a production AI system
AI system interpretation:
- error classifier
- retry manager
- fallback router
- circuit breaker
- timeout manager
- idempotency store
- dead-letter queue
- alerting system
👉 Interview Answer
I do not start by saying this is just an LLM feature. I first identify the deterministic system problem underneath it. For this topic, the deterministic part is failure classification, retry policy, fallback path, idempotency, circuit breaker, and graceful degradation. Once that is clear, I can discuss models, prompts, tools, and memory as system components rather than magic behavior.
3️⃣ LeetCode Pattern Mapping
This topic can be practiced through these LeetCode-style patterns:
- Jump Game
- Gas Station
- Climbing Stairs
- Design Hit Counter
- Queue Reconstruction
- State machine problems
The key is not to memorize the list. The key is to explain the bridge:
AI system behavior
↓
Algorithmic abstraction
↓
Data structure choice
↓
Complexity analysis
↓
Production constraints
👉 Interview Answer
I would map Retry / Fallback Agent to a LeetCode pattern by identifying the state, ordering rule, and constraint. If the problem asks for best K items, I think heap or selection. If the problem has dependencies, I think graph and topological sort. If the problem has bounded history, I think cache, queue, sliding window, or time-indexed storage.
4️⃣ Core Algorithms and Data Structures
state machine
- Used when Retry / Fallback Agent needs state machine.
- Explain the invariant.
- Explain the complexity.
- Explain the failure mode when scale increases.
exponential backoff
- Used when Retry / Fallback Agent needs exponential backoff.
- Explain the invariant.
- Explain the complexity.
- Explain the failure mode when scale increases.
jitter
- Used when Retry / Fallback Agent needs jitter.
- Explain the invariant.
- Explain the complexity.
- Explain the failure mode when scale increases.
retry budget
- Used when Retry / Fallback Agent needs retry budget.
- Explain the invariant.
- Explain the complexity.
- Explain the failure mode when scale increases.
circuit breaker
- Used when Retry / Fallback Agent needs circuit breaker.
- Explain the invariant.
- Explain the complexity.
- Explain the failure mode when scale increases.
fallback graph
- Used when Retry / Fallback Agent needs fallback graph.
- Explain the invariant.
- Explain the complexity.
- Explain the failure mode when scale increases.
idempotency key
- Used when Retry / Fallback Agent needs idempotency key.
- Explain the invariant.
- Explain the complexity.
- Explain the failure mode when scale increases.
Baseline Complexity Discussion
- Start with the simplest exact solution.
- Analyze time and space complexity.
- Identify the bottleneck.
- Add indexing, caching, batching, or approximation only when justified.
👉 Interview Answer
My baseline answer is intentionally simple first. I would rather show a correct O(N log K) or O(N) design and then optimize it than jump directly to a complex distributed system. After the baseline is clear, I discuss where the bottleneck appears and which production mechanism addresses it.
5️⃣ Problem Definition
Define the exact input, output, constraints, and correctness expectation before discussing implementation.
For Retry / Fallback Agent, the important details are:
- clear invariant
- bounded resource usage
- explicit state
- versioned behavior
- safe fallback
- measurable quality
- debuggable traces
- well-defined ownership
Staff-level detail:
The staff-level point is that retries are a reliability tool, not a correctness guarantee. The system must classify errors, retry only safe operations, use budgets, and degrade gracefully when dependencies are unhealthy.
Memorize this answer:
For Retry / Fallback Agent, I would design the algorithm first and then wrap it with production controls. The algorithm gives the local correctness property. The system design gives permission safety, latency control, observability, and failure recovery. At Staff level, I would explicitly separate model quality from system guarantees.
6️⃣ LeetCode Mapping
Map the AI behavior to a recognizable algorithmic pattern so the interviewer sees both coding skill and system design intuition.
For Retry / Fallback Agent, the important details are:
- clear invariant
- bounded resource usage
- explicit state
- versioned behavior
- safe fallback
- measurable quality
- debuggable traces
- well-defined ownership
Staff-level detail:
The staff-level point is that retries are a reliability tool, not a correctness guarantee. The system must classify errors, retry only safe operations, use budgets, and degrade gracefully when dependencies are unhealthy.
Memorize this answer:
For Retry / Fallback Agent, I would design the algorithm first and then wrap it with production controls. The algorithm gives the local correctness property. The system design gives permission safety, latency control, observability, and failure recovery. At Staff level, I would explicitly separate model quality from system guarantees.
7️⃣ Data Model
Describe what data needs to be represented explicitly, because hidden state is where many agent systems become unreliable.
For Retry / Fallback Agent, the important details are:
- primary id
- user or tenant scope
- session or task scope
- timestamp
- version
- score
- status
- permission metadata
- trace id
- expiration or retention rule
Staff-level detail:
The staff-level point is that retries are a reliability tool, not a correctness guarantee. The system must classify errors, retry only safe operations, use budgets, and degrade gracefully when dependencies are unhealthy.
Memorize this answer:
For Retry / Fallback Agent, I would design the algorithm first and then wrap it with production controls. The algorithm gives the local correctness property. The system design gives permission safety, latency control, observability, and failure recovery. At Staff level, I would explicitly separate model quality from system guarantees.
8️⃣ Core Algorithm
Explain the baseline algorithm, its complexity, and the condition where it stops being enough.
For Retry / Fallback Agent, the important details are:
- clear invariant
- bounded resource usage
- explicit state
- versioned behavior
- safe fallback
- measurable quality
- debuggable traces
- well-defined ownership
Staff-level detail:
The staff-level point is that retries are a reliability tool, not a correctness guarantee. The system must classify errors, retry only safe operations, use budgets, and degrade gracefully when dependencies are unhealthy.
Memorize this answer:
For Retry / Fallback Agent, I would design the algorithm first and then wrap it with production controls. The algorithm gives the local correctness property. The system design gives permission safety, latency control, observability, and failure recovery. At Staff level, I would explicitly separate model quality from system guarantees.
9️⃣ Production Architecture
Move from the algorithm to a deployable path with clear components, ownership, and failure boundaries.
For Retry / Fallback Agent, the important details are:
- error classifier
- retry manager
- fallback router
- circuit breaker
- timeout manager
- idempotency store
- dead-letter queue
- alerting system
Staff-level detail:
The staff-level point is that retries are a reliability tool, not a correctness guarantee. The system must classify errors, retry only safe operations, use budgets, and degrade gracefully when dependencies are unhealthy.
Memorize this answer:
For Retry / Fallback Agent, I would design the algorithm first and then wrap it with production controls. The algorithm gives the local correctness property. The system design gives permission safety, latency control, observability, and failure recovery. At Staff level, I would explicitly separate model quality from system guarantees.
10️⃣ Scaling Strategy
Separate stateless scaling from stateful scaling and describe where bottlenecks appear first.
For Retry / Fallback Agent, the important details are:
- clear invariant
- bounded resource usage
- explicit state
- versioned behavior
- safe fallback
- measurable quality
- debuggable traces
- well-defined ownership
Staff-level detail:
The staff-level point is that retries are a reliability tool, not a correctness guarantee. The system must classify errors, retry only safe operations, use budgets, and degrade gracefully when dependencies are unhealthy.
Memorize this answer:
For Retry / Fallback Agent, I would design the algorithm first and then wrap it with production controls. The algorithm gives the local correctness property. The system design gives permission safety, latency control, observability, and failure recovery. At Staff level, I would explicitly separate model quality from system guarantees.
11️⃣ Latency Budget
Break down latency by component and explain which calls are synchronous, asynchronous, cached, or batchable.
For Retry / Fallback Agent, the important details are:
- clear invariant
- bounded resource usage
- explicit state
- versioned behavior
- safe fallback
- measurable quality
- debuggable traces
- well-defined ownership
Staff-level detail:
The staff-level point is that retries are a reliability tool, not a correctness guarantee. The system must classify errors, retry only safe operations, use budgets, and degrade gracefully when dependencies are unhealthy.
Memorize this answer:
For Retry / Fallback Agent, I would design the algorithm first and then wrap it with production controls. The algorithm gives the local correctness property. The system design gives permission safety, latency control, observability, and failure recovery. At Staff level, I would explicitly separate model quality from system guarantees.
12️⃣ Correctness Model
State what correctness means in this system, because AI systems often have probabilistic quality and deterministic safety rules at the same time.
For Retry / Fallback Agent, the important details are:
- clear invariant
- bounded resource usage
- explicit state
- versioned behavior
- safe fallback
- measurable quality
- debuggable traces
- well-defined ownership
Staff-level detail:
The staff-level point is that retries are a reliability tool, not a correctness guarantee. The system must classify errors, retry only safe operations, use budgets, and degrade gracefully when dependencies are unhealthy.
Memorize this answer:
For Retry / Fallback Agent, I would design the algorithm first and then wrap it with production controls. The algorithm gives the local correctness property. The system design gives permission safety, latency control, observability, and failure recovery. At Staff level, I would explicitly separate model quality from system guarantees.
13️⃣ Failure Handling
List expected failures and explain how the system degrades without corrupting state or violating permissions.
For Retry / Fallback Agent, the important details are:
- infinite retry
- retry storm
- duplicated writes
- hiding real errors
- bad fallback quality
- cascading failure
- provider outage
- non-idempotent retries
Staff-level detail:
The staff-level point is that retries are a reliability tool, not a correctness guarantee. The system must classify errors, retry only safe operations, use budgets, and degrade gracefully when dependencies are unhealthy.
Memorize this answer:
For Retry / Fallback Agent, I would design the algorithm first and then wrap it with production controls. The algorithm gives the local correctness property. The system design gives permission safety, latency control, observability, and failure recovery. At Staff level, I would explicitly separate model quality from system guarantees.
14️⃣ Security and Privacy
Explain scope, authorization, auditability, data minimization, and safe prompt construction.
For Retry / Fallback Agent, the important details are:
- tenant isolation
- user-level authorization
- least privilege tool access
- redaction before logging
- prompt injection defense
- audit logs
- retention policy
- delete and correction workflow
Staff-level detail:
The staff-level point is that retries are a reliability tool, not a correctness guarantee. The system must classify errors, retry only safe operations, use budgets, and degrade gracefully when dependencies are unhealthy.
Memorize this answer:
For Retry / Fallback Agent, I would design the algorithm first and then wrap it with production controls. The algorithm gives the local correctness property. The system design gives permission safety, latency control, observability, and failure recovery. At Staff level, I would explicitly separate model quality from system guarantees.
15️⃣ Evaluation
Define offline and online metrics, then explain how regressions are detected after model or index changes.
For Retry / Fallback Agent, the important details are:
- retry success rate
- fallback rate
- timeout rate
- circuit open count
- duplicate write count
- mean recovery time
- error budget burn
- user-visible failure rate
Staff-level detail:
The staff-level point is that retries are a reliability tool, not a correctness guarantee. The system must classify errors, retry only safe operations, use budgets, and degrade gracefully when dependencies are unhealthy.
Memorize this answer:
For Retry / Fallback Agent, I would design the algorithm first and then wrap it with production controls. The algorithm gives the local correctness property. The system design gives permission safety, latency control, observability, and failure recovery. At Staff level, I would explicitly separate model quality from system guarantees.
16️⃣ Observability
Trace the full path with request id, model version, prompt version, tool calls, tokens, costs, and quality signals.
For Retry / Fallback Agent, the important details are:
- clear invariant
- bounded resource usage
- explicit state
- versioned behavior
- safe fallback
- measurable quality
- debuggable traces
- well-defined ownership
Staff-level detail:
The staff-level point is that retries are a reliability tool, not a correctness guarantee. The system must classify errors, retry only safe operations, use budgets, and degrade gracefully when dependencies are unhealthy.
Memorize this answer:
For Retry / Fallback Agent, I would design the algorithm first and then wrap it with production controls. The algorithm gives the local correctness property. The system design gives permission safety, latency control, observability, and failure recovery. At Staff level, I would explicitly separate model quality from system guarantees.
17️⃣ Trade-offs
Compare simple and production-grade designs, including accuracy, latency, cost, complexity, and operational risk.
For Retry / Fallback Agent, the important details are:
- exactness vs latency
- quality vs cost
- freshness vs stability
- model flexibility vs deterministic guardrails
- simplicity vs operational control
- recall vs precision
- cache reuse vs staleness
- automation vs human approval
Staff-level detail:
The staff-level point is that retries are a reliability tool, not a correctness guarantee. The system must classify errors, retry only safe operations, use budgets, and degrade gracefully when dependencies are unhealthy.
Memorize this answer:
For Retry / Fallback Agent, I would design the algorithm first and then wrap it with production controls. The algorithm gives the local correctness property. The system design gives permission safety, latency control, observability, and failure recovery. At Staff level, I would explicitly separate model quality from system guarantees.
18️⃣ Staff-Level Framing
Show that the model is only one component and the system must own boundaries, budgets, safety, and debuggability.
For Retry / Fallback Agent, the important details are:
- clear invariant
- bounded resource usage
- explicit state
- versioned behavior
- safe fallback
- measurable quality
- debuggable traces
- well-defined ownership
Staff-level detail:
The staff-level point is that retries are a reliability tool, not a correctness guarantee. The system must classify errors, retry only safe operations, use budgets, and degrade gracefully when dependencies are unhealthy.
Memorize this answer:
For Retry / Fallback Agent, I would design the algorithm first and then wrap it with production controls. The algorithm gives the local correctness property. The system design gives permission safety, latency control, observability, and failure recovery. At Staff level, I would explicitly separate model quality from system guarantees.
1️⃣9️⃣ High-Level Architecture
Agent call
↓
Failure
↓
Classify error
↓
Retryable check
↓
Backoff retry
↓
Fallback path
↓
Circuit breaker decision
↓
Final or degraded response
This flow should be explained as two paths:
Online Path
- handles user-facing latency
- applies permission and budget checks
- returns final response or fallback
Offline Path
- builds indexes or summaries
- refreshes models and metadata
- runs evaluation and regression checks
- prepares caches or warm state
👉 Interview Answer
I separate online and offline paths because they have different reliability and latency requirements. The online path must be fast, bounded, and permission-safe. The offline path can be heavier and is responsible for indexing, evaluation, refresh, and quality improvement.
2️⃣0️⃣ Production Failure Modes
infinite retry
Why it matters for Retry / Fallback Agent:
- it can reduce answer quality
- it can increase latency or cost
- it can violate user trust if not bounded
Mitigation:
- detect it through metrics and traces
- add deterministic guardrails
- provide fallback or degraded mode
- run regression tests before rollout
retry storm
Why it matters for Retry / Fallback Agent:
- it can reduce answer quality
- it can increase latency or cost
- it can violate user trust if not bounded
Mitigation:
- detect it through metrics and traces
- add deterministic guardrails
- provide fallback or degraded mode
- run regression tests before rollout
duplicated writes
Why it matters for Retry / Fallback Agent:
- it can reduce answer quality
- it can increase latency or cost
- it can violate user trust if not bounded
Mitigation:
- detect it through metrics and traces
- add deterministic guardrails
- provide fallback or degraded mode
- run regression tests before rollout
hiding real errors
Why it matters for Retry / Fallback Agent:
- it can reduce answer quality
- it can increase latency or cost
- it can violate user trust if not bounded
Mitigation:
- detect it through metrics and traces
- add deterministic guardrails
- provide fallback or degraded mode
- run regression tests before rollout
bad fallback quality
Why it matters for Retry / Fallback Agent:
- it can reduce answer quality
- it can increase latency or cost
- it can violate user trust if not bounded
Mitigation:
- detect it through metrics and traces
- add deterministic guardrails
- provide fallback or degraded mode
- run regression tests before rollout
cascading failure
Why it matters for Retry / Fallback Agent:
- it can reduce answer quality
- it can increase latency or cost
- it can violate user trust if not bounded
Mitigation:
- detect it through metrics and traces
- add deterministic guardrails
- provide fallback or degraded mode
- run regression tests before rollout
provider outage
Why it matters for Retry / Fallback Agent:
- it can reduce answer quality
- it can increase latency or cost
- it can violate user trust if not bounded
Mitigation:
- detect it through metrics and traces
- add deterministic guardrails
- provide fallback or degraded mode
- run regression tests before rollout
non-idempotent retries
Why it matters for Retry / Fallback Agent:
- it can reduce answer quality
- it can increase latency or cost
- it can violate user trust if not bounded
Mitigation:
- detect it through metrics and traces
- add deterministic guardrails
- provide fallback or degraded mode
- run regression tests before rollout
👉 Interview Answer
I would not rely on the model to fix production failures by itself. The system should classify failures, apply deterministic mitigation, and expose traces so engineers can debug the path after the fact.
2️⃣1️⃣ Metrics and Evaluation
A strong answer needs metrics. I would track:
- retry success rate
- fallback rate
- timeout rate
- circuit open count
- duplicate write count
- mean recovery time
- error budget burn
- user-visible failure rate
Offline Evaluation
- fixed benchmark set
- golden examples
- adversarial cases
- regression checks by version
- per-category breakdown
Online Evaluation
- user success signals
- latency p95 / p99
- cost per successful task
- fallback rate
- quality feedback
- alert thresholds
👉 Interview Answer
For AI systems, I would measure both system metrics and quality metrics. Latency, cost, and error rate tell me whether the service is healthy. Recall, precision, groundedness, and user correction rate tell me whether the answer is useful.
2️⃣2️⃣ Common Interview Follow-ups
Q: How would you start with a simple solution?
A: Start with an exact single-node algorithm, define complexity, and only add distributed components when the bottleneck is clear.
Q: How do you scale it?
A: Scale stateless services horizontally, shard or index stateful stores, and protect expensive model/tool calls with cache, queue, or rate limits.
Q: How do you keep it safe?
A: Use permission checks, scoped state, schema validation, audit logs, and deterministic guardrails before model output is trusted.
Q: How do you evaluate quality?
A: Use offline golden sets plus online success signals, and compare versions before rollout.
Q: How do you reduce latency?
A: Cache safe results, precompute offline artifacts, batch expensive work, use approximate search when acceptable, and set strict timeouts.
Q: How do you handle failures?
A: Classify errors, retry only safe transient failures, use fallback paths, and surface clear degraded responses.
Q: What is the Staff-level insight?
A: The staff-level point is that retries are a reliability tool, not a correctness guarantee. The system must classify errors, retry only safe operations, use budgets, and degrade gracefully when dependencies are unhealthy.
2️⃣3️⃣ Answer Bank for Memorization
Memorization Paragraph 1
For Retry / Fallback Agent, I would first identify the deterministic algorithm underneath the AI feature. The problem is to recover from model, tool, retrieval, or infrastructure failures without looping forever or causing cascading failure. That maps to Jump Game, Gas Station, Climbing Stairs. Once the algorithm is clear, I would add production concerns such as latency, permissions, versioning, observability, and fallback.
Memorization Paragraph 2
My baseline design for Retry / Fallback Agent is simple and exact. I define the data model, choose the right data structure, and analyze time and space complexity. Then I explain where it breaks at scale and what index, cache, queue, or distributed component I would introduce.
Memorization Paragraph 3
At Staff level, I would not present Retry / Fallback Agent as just a prompt or model behavior. I would describe the system boundary: what is deterministic, what is probabilistic, what is cached, what is versioned, what is permission-checked, and what is observable.
Memorization Paragraph 4
The main trade-off in Retry / Fallback Agent is quality versus latency and cost. A more accurate path may use more ranking, validation, or model calls. A faster path may use cache, approximation, or simpler heuristics. I would choose based on the product’s correctness requirement and error budget.
Memorization Paragraph 5
For production readiness, I would add request tracing, model and prompt versioning, offline evaluation, online metrics, failure classification, and rollback strategy. Without these, Retry / Fallback Agent can work in a demo but fail silently in production.
2️⃣4️⃣ Senior / Staff-Level Summary Answer
I would explain Retry / Fallback Agent as an AI system built on top of a concrete LeetCode-style algorithm. The algorithmic core is failure classification, retry policy, fallback path, idempotency, circuit breaker, and graceful degradation. The production system must add explicit state, permission checks, versioning, latency budgets, evaluation, and observability. The Staff-level answer is to separate model behavior from system guarantees: the model can help rank, summarize, or decide, but the platform must enforce correctness, safety, and recovery.
中文部分
🎯 Retry / Fallback Agent
Retry / Fallback Agent 可以包装成 reliability state machine。Staff 级重点是 error classification、backoff、jitter、retry budget、idempotency、circuit breaker 和 graceful degradation。
1️⃣ 中文核心框架
讨论 Retry / Fallback Agent 时，我会按这个顺序回答：
- 先把 AI 功能翻译成算法问题
- 说明对应的 LeetCode 题型
- 给出 baseline algorithm 和复杂度
- 再扩展到 production architecture
- 最后讲 Staff 级 trade-off、failure、eval 和 observability
可背诵回答：
Retry / Fallback Agent 不是单纯的 prompt 技巧，而是一个可以映射到 LeetCode pattern 的系统问题。 它的核心是 failure classification, retry policy, fallback path, idempotency, circuit breaker, and graceful degradation。 我会先讲清楚算法和复杂度，再说明生产系统里如何处理权限、延迟、成本、失败和评估。 Staff 级回答的重点是：model 负责智能能力，system 负责边界、预算、安全和可观测性。
2️⃣ 对应 LeetCode 题型
- Jump Game
- Gas Station
- Climbing Stairs
- Design Hit Counter
- Queue Reconstruction
- State machine problems
这些题型的共同点是：
- 都有明确的数据结构选择
- 都需要复杂度分析
- 都可以扩展成生产系统里的组件设计
- 都能自然引出 Staff 级 trade-off
面试表达：
我会把 Retry / Fallback Agent 先抽象成算法题。 如果题目关注排序和最优结果，我会考虑 heap、sorting、quickselect 或 ranking。 如果题目关注状态变化，我会考虑 state machine、graph traversal 或 cache design。 这样回答可以同时覆盖 coding 和 system design。
3️⃣ 问题定义
先定义输入、输出、约束和正确性标准，避免一上来只讲模型。
在 Retry / Fallback Agent 里，我会强调：
- 显式 state
- 清晰 invariant
- 版本控制
- 权限边界
- token / cost budget
- fallback
- traceability
- offline + online eval
中文背诵段落：
对于 Retry / Fallback Agent，我不会只说调用 LLM。 我会先定义系统中确定性的部分，比如数据结构、状态、排序规则、缓存规则或图依赖。 然后再把模型能力放到受控的系统边界里，用权限、预算、trace、eval 和 fallback 保证生产可靠性。
4️⃣ LeetCode 映射
把 AI 行为映射到熟悉的算法题型，让面试官看到算法和系统设计之间的连接。
在 Retry / Fallback Agent 里，我会强调：
- 显式 state
- 清晰 invariant
- 版本控制
- 权限边界
- token / cost budget
- fallback
- traceability
- offline + online eval
中文背诵段落：
对于 Retry / Fallback Agent，我不会只说调用 LLM。 我会先定义系统中确定性的部分，比如数据结构、状态、排序规则、缓存规则或图依赖。 然后再把模型能力放到受控的系统边界里，用权限、预算、trace、eval 和 fallback 保证生产可靠性。
5️⃣ 数据模型
明确 state、metadata、score、version、permission scope 等字段，因为 agent 系统的问题往往来自隐藏状态。
在 Retry / Fallback Agent 里，我会强调：
- 显式 state
- 清晰 invariant
- 版本控制
- 权限边界
- token / cost budget
- fallback
- traceability
- offline + online eval
中文背诵段落：
对于 Retry / Fallback Agent，我不会只说调用 LLM。 我会先定义系统中确定性的部分，比如数据结构、状态、排序规则、缓存规则或图依赖。 然后再把模型能力放到受控的系统边界里，用权限、预算、trace、eval 和 fallback 保证生产可靠性。
6️⃣ 核心算法
讲 baseline algorithm、复杂度、适用场景，以及什么时候需要升级到 production design。
在 Retry / Fallback Agent 里，我会强调：
- 显式 state
- 清晰 invariant
- 版本控制
- 权限边界
- token / cost budget
- fallback
- traceability
- offline + online eval
中文背诵段落：
对于 Retry / Fallback Agent，我不会只说调用 LLM。 我会先定义系统中确定性的部分，比如数据结构、状态、排序规则、缓存规则或图依赖。 然后再把模型能力放到受控的系统边界里，用权限、预算、trace、eval 和 fallback 保证生产可靠性。
7️⃣ 生产架构
把算法放进真实系统，说明组件边界、调用路径和 failure boundary。
在 Retry / Fallback Agent 里，我会强调：
- error classifier
- retry manager
- fallback router
- circuit breaker
- timeout manager
- idempotency store
- dead-letter queue
- alerting system
中文背诵段落：
对于 Retry / Fallback Agent，我不会只说调用 LLM。 我会先定义系统中确定性的部分，比如数据结构、状态、排序规则、缓存规则或图依赖。 然后再把模型能力放到受控的系统边界里，用权限、预算、trace、eval 和 fallback 保证生产可靠性。
8️⃣ 扩展策略
区分 stateless layer 和 stateful layer，说明瓶颈在哪里出现。
在 Retry / Fallback Agent 里，我会强调：
- 显式 state
- 清晰 invariant
- 版本控制
- 权限边界
- token / cost budget
- fallback
- traceability
- offline + online eval
中文背诵段落：
对于 Retry / Fallback Agent，我不会只说调用 LLM。 我会先定义系统中确定性的部分，比如数据结构、状态、排序规则、缓存规则或图依赖。 然后再把模型能力放到受控的系统边界里，用权限、预算、trace、eval 和 fallback 保证生产可靠性。
9️⃣ 延迟预算
拆分每个组件的 latency，说明哪些可以 cache、batch、async 或降级。
在 Retry / Fallback Agent 里，我会强调：
- 显式 state
- 清晰 invariant
- 版本控制
- 权限边界
- token / cost budget
- fallback
- traceability
- offline + online eval
中文背诵段落：
对于 Retry / Fallback Agent，我不会只说调用 LLM。 我会先定义系统中确定性的部分，比如数据结构、状态、排序规则、缓存规则或图依赖。 然后再把模型能力放到受控的系统边界里，用权限、预算、trace、eval 和 fallback 保证生产可靠性。
10️⃣ 正确性模型
AI 结果质量是概率性的，但权限、安全、幂等、预算这些规则必须是确定性的。
在 Retry / Fallback Agent 里，我会强调：
- 显式 state
- 清晰 invariant
- 版本控制
- 权限边界
- token / cost budget
- fallback
- traceability
- offline + online eval
中文背诵段落：
对于 Retry / Fallback Agent，我不会只说调用 LLM。 我会先定义系统中确定性的部分，比如数据结构、状态、排序规则、缓存规则或图依赖。 然后再把模型能力放到受控的系统边界里，用权限、预算、trace、eval 和 fallback 保证生产可靠性。
11️⃣ 失败处理
说明 timeout、empty result、schema mismatch、permission denied 等失败如何处理。
在 Retry / Fallback Agent 里，我会强调：
- infinite retry
- retry storm
- duplicated writes
- hiding real errors
- bad fallback quality
- cascading failure
- provider outage
- non-idempotent retries
中文背诵段落：
对于 Retry / Fallback Agent，我不会只说调用 LLM。 我会先定义系统中确定性的部分，比如数据结构、状态、排序规则、缓存规则或图依赖。 然后再把模型能力放到受控的系统边界里，用权限、预算、trace、eval 和 fallback 保证生产可靠性。
12️⃣ 安全与隐私
强调 scope、authorization、audit、data minimization 和 prompt safety。
在 Retry / Fallback Agent 里，我会强调：
- tenant isolation
- user permission
- least privilege
- 日志脱敏
- prompt injection 防护
- audit trail
- retention / deletion policy
中文背诵段落：
对于 Retry / Fallback Agent，我不会只说调用 LLM。 我会先定义系统中确定性的部分，比如数据结构、状态、排序规则、缓存规则或图依赖。 然后再把模型能力放到受控的系统边界里，用权限、预算、trace、eval 和 fallback 保证生产可靠性。
13️⃣ 评估体系
定义 offline eval 和 online metrics，说明 model/index/prompt 变化后如何防止 regression。
在 Retry / Fallback Agent 里，我会强调：
- retry success rate
- fallback rate
- timeout rate
- circuit open count
- duplicate write count
- mean recovery time
- error budget burn
- user-visible failure rate
中文背诵段落：
对于 Retry / Fallback Agent，我不会只说调用 LLM。 我会先定义系统中确定性的部分，比如数据结构、状态、排序规则、缓存规则或图依赖。 然后再把模型能力放到受控的系统边界里，用权限、预算、trace、eval 和 fallback 保证生产可靠性。
14️⃣ 可观测性
trace request id、model version、prompt version、tool calls、tokens、cost 和 quality signals。
在 Retry / Fallback Agent 里，我会强调：
- 显式 state
- 清晰 invariant
- 版本控制
- 权限边界
- token / cost budget
- fallback
- traceability
- offline + online eval
中文背诵段落：
对于 Retry / Fallback Agent，我不会只说调用 LLM。 我会先定义系统中确定性的部分，比如数据结构、状态、排序规则、缓存规则或图依赖。 然后再把模型能力放到受控的系统边界里，用权限、预算、trace、eval 和 fallback 保证生产可靠性。
15️⃣ Trade-off
比较简单方案和生产方案，在准确率、延迟、成本、复杂度之间做权衡。
在 Retry / Fallback Agent 里，我会强调：
- 准确率 vs 延迟
- 质量 vs 成本
- 新鲜度 vs 稳定性
- 模型灵活性 vs deterministic guardrails
- 简单实现 vs 运维复杂度
- 自动化 vs 人工审批
中文背诵段落：
对于 Retry / Fallback Agent，我不会只说调用 LLM。 我会先定义系统中确定性的部分，比如数据结构、状态、排序规则、缓存规则或图依赖。 然后再把模型能力放到受控的系统边界里，用权限、预算、trace、eval 和 fallback 保证生产可靠性。
16️⃣ Staff 级表达
强调 model 只是一个组件，系统要负责边界、预算、安全、trace 和可恢复性。
在 Retry / Fallback Agent 里，我会强调：
- 显式 state
- 清晰 invariant
- 版本控制
- 权限边界
- token / cost budget
- fallback
- traceability
- offline + online eval
中文背诵段落：
对于 Retry / Fallback Agent，我不会只说调用 LLM。 我会先定义系统中确定性的部分，比如数据结构、状态、排序规则、缓存规则或图依赖。 然后再把模型能力放到受控的系统边界里，用权限、预算、trace、eval 和 fallback 保证生产可靠性。
1️⃣7️⃣ 中文高阶追问
Q: 如何从简单方案开始？
A: 先给单机 baseline，说明复杂度，然后指出瓶颈。
Q: 如何扩展到生产环境？
A: stateless 层水平扩展，stateful 层用 index/sharding/cache/queue，并控制 model/tool 成本。
Q: 如何保证安全？
A: 权限过滤、scope、schema validation、audit logs、敏感信息过滤。
Q: 如何做评估？
A: offline golden set + online metrics，版本升级前做 regression。
Q: 如何降低延迟？
A: cache、batch、precompute、approximation、timeout、fallback。
Q: Staff 级 insight 是什么？
A: The staff-level point is that retries are a reliability tool, not a correctness guarantee. The system must classify errors, retry only safe operations, use budgets, and degrade gracefully when dependencies are unhealthy.
1️⃣8️⃣ 中文背诵答案库
背诵段落 1
Retry / Fallback Agent 的核心不是模型本身，而是把 AI 行为抽象成一个可解释、可评估、可扩展的系统问题。它对应的算法核心是 failure classification, retry policy, fallback path, idempotency, circuit breaker, and graceful degradation。面试里我会先讲 baseline algorithm，再讲 production constraints。
背诵段落 2
如果只从 demo 角度看，Retry / Fallback Agent 可能只是一次模型调用。但在 production 里，它需要明确的 state、version、permission、budget、fallback 和 observability。Staff 级回答要把这些边界讲清楚。
背诵段落 3
我会把 Retry / Fallback Agent 的设计分成 online path 和 offline path。online path 关注用户请求延迟和权限安全，offline path 关注 index、cache、eval、refresh 和质量改进。
背诵段落 4
这个题的核心 trade-off 是质量、延迟和成本之间的平衡。更高质量通常意味着更多检索、rerank、验证或模型调用；更低延迟通常需要 cache、预计算或近似算法。
背诵段落 5
生产系统里，Retry / Fallback Agent 必须有 trace 和 eval。否则一旦结果变差，很难判断问题来自模型、prompt、index、tool、cache、permission 还是 context packing。
1️⃣9️⃣ 中文 Staff 总结
Retry / Fallback Agent 的 Staff 级回答，要从 LeetCode pattern 讲到 production AI system。 算法部分解决局部正确性：failure classification, retry policy, fallback path, idempotency, circuit breaker, and graceful degradation。 系统部分解决生产可靠性：权限、版本、延迟、成本、失败、评估和可观测性。 我会明确区分 model capability 和 system guarantee。模型可以帮助理解和生成，但系统必须负责边界和恢复。
note:
Retry / Fallback Agent = LeetCode pattern + AI system wrapper + Staff-level production constraints.
Agent call
↓
Failure
↓
Classify error
↓
Retryable check
↓
Backoff retry
↓
Fallback path
↓
Circuit breaker decision
↓
Final or degraded response
Remember:
- Start with the algorithm.
- Explain complexity.
- Add production boundaries.
- Add safety and evaluation.
- End with Staff-level trade-offs.
Implement