---
title: "LLM integration for existing products and workflows"
canonical: "https://amartripathi.com/services/llm-integration"
---

# LLM integration for existing products and workflows

I add language-model capabilities to existing products without forcing the complete product into one provider or one opaque workflow.

Updated: September 2026

Canonical page: https://amartripathi.com/services/llm-integration

## Quick answer

A production LLM integration should isolate provider details, validate structured output, expose failures, control tools, and preserve normal product behavior. The model becomes one bounded dependency inside the system.

## Evidence and decision points

### 12+ providers

Built production routing across providers including OpenAI, Anthropic, Google, xAI, Meta, DeepSeek, and Mistral.

### Streaming interfaces

Built responsive conversational interfaces with Server-Sent Events and long-running workflow support.

### Tool workflows

Connected models to approved tools, search, code execution, media generation, and product actions.

## Choose the integration boundary

A useful integration has a narrow purpose. It might summarize a record, classify an input, answer from approved documents, draft content, or choose an allowed tool. The product must define the boundary before choosing a model.

I keep model requests behind typed application interfaces where practical. This makes validation, testing, provider changes, fallback behavior, and cost controls easier to manage.

- Define inputs, outputs, uncertainty, and prohibited behavior.

- Validate structured output before application code uses it.

- Keep retries idempotent when a workflow can cause side effects.

- Ask for user approval before sensitive or irreversible tool actions.

## Provider routing and resilience

Different models offer different latency, capability, context, modality, and cost characteristics. A product may need one model, a preferred model with fallback, or user-controlled selection.

Routing should remain understandable. I avoid adding providers without a product reason. Each new path increases testing, monitoring, pricing, and failure complexity.

- Normalize message and tool contracts without hiding important provider differences.

- Set explicit timeouts, retry rules, limits, and fallback behavior.

- Track latency, errors, token use, and model selection at useful boundaries.

- Preserve provider-specific capabilities only where they produce a real user benefit.

## Evaluation and handover

Evaluation cases should represent real product inputs. They need expected behavior, acceptable variations, and known failure conditions. A demo prompt is not a sufficient regression suite.

The handover explains provider dependencies, prompt and schema contracts, data handling, operating limits, and the process for changing models safely.

## Single-provider and multi-provider integration

| Decision | Single provider | Multiple providers |

| --- | --- | --- |

| Complexity | Lower | Higher routing and evaluation cost |

| Capability | Focused on one model family | Can combine specialist capabilities |

| Resilience | Provider outage affects the feature | Fallback is possible when contracts align |

| Best fit | Clear stable requirement | User choice, resilience, or capability differences |

## Common questions

### Can you integrate OpenAI, Anthropic, Google, and OpenRouter?

Yes. I have worked with multi-provider systems and can select a direct or routed architecture based on the product requirement.

### Can the integration use structured output?

Yes. I define and validate structured contracts before product code accepts model output. Validation and failure states remain explicit.

### Can you add tool calling to an existing application?

Yes. Tool contracts, permissions, retries, confirmation points, and audit behavior should match the risk of each action.

### Can you replace an existing model provider?

Yes, when the current product behavior and evaluation cases are clear. Provider migration still needs capability and output comparison.

## Sources and further reading

- [OWASP Top 10 for LLM applications](https://genai.owasp.org/llm-top-10/): OWASP describes prompt injection, excessive agency, output handling, and related application risks.

- [Vercel AI SDK documentation](https://sdk.vercel.ai/docs): The official documentation covers provider integrations, streaming, structured output, and tool patterns.

## Related resources

- [AI SaaS development](https://amartripathi.com/services/ai-saas-development)

- [How to hire an AI developer](https://amartripathi.com/guides/how-to-hire-ai-developer)

- [Magica AI chat platform](https://amartripathi.com/case-studies/magica-ai-chat-platform)

## Contact

Email Amar Tripathi at [theamartripathi@gmail.com](mailto:theamartripathi@gmail.com) with the product context, current stack, expected outcome, constraints, and target timeline.