LangChain vs Raw API Calls: When the Abstraction Pays for Itself

Every time I start a new project involving an LLM, I face the same decision: use LangChain (or one of its equivalents) or call the API directly. I have done both enough times to have a clear view of which choice leads to regret and when.

The short version: LangChain is a good library that gets used in the wrong situations constantly. The raw API is lower friction than people expect.

What LangChain actually gives you

LangChain is a framework for chaining LLM calls, tool use, retrieval, and memory into pipelines. At its core it solves real problems:

• Standardized interfaces. Swap between OpenAI, Anthropic, and local models without rewriting your chain logic.
• Built-in retrieval. Document loaders, text splitters, vector store integrations, and retrieval chains are all provided.
• Agent abstractions. ReAct loops, tool schemas, and execution managed for you.
• Memory. Conversation history, window buffers, and summarization memory out of the box.

For a project that genuinely uses all of these — a RAG chatbot over multiple document types, with memory, running on a model you might need to swap — that is real leverage.

Where it stops paying for itself

The problem is that most projects do not need all of that. They need one or two capabilities, and LangChain requires you to learn its entire mental model to use any of them.

The abstraction leaks constantly in production. When an API response comes back in an unexpected shape, or a chain fails in a non-obvious way, you are debugging through three layers of framework before you reach the actual error. I have spent more time reading LangChain source code than I care to admit.

The upgrade treadmill is also real. LangChain v0.1 and v0.2 had significant breaking changes. The community moves fast, which is good for features and bad for stability.

The raw API is more approachable than it looks

The OpenAI and Anthropic APIs are well-designed. A chat completion with tool use is maybe thirty lines of Python. A retrieval loop with a vector store is another fifty. The code you write is debuggable, greppable, and exactly as complex as the problem requires.

from openai import OpenAI
 
client = OpenAI()
 
def ask(messages: list[dict], tools: list[dict] | None = None) -> str:
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=messages,
        tools=tools or [],
    )
    return response.choices[0].message

That is the entire abstraction you need for most tasks. When you want streaming, you add stream=True. When you want structured output, you add a response_format. There is no indirection.

A decision framework

Before reaching for LangChain, I ask:

1. Do I need to swap models? If the model is fixed, the provider abstraction does nothing for you.
2. Do I have RAG over heterogeneous documents? This is LangChain's genuine stronghold. The loader ecosystem is excellent.
3. Am I prototyping or shipping? LangChain is faster to prototype with. Raw API calls are more maintainable in production.
4. How many people will debug this? The more people, the more the framework's implicit behavior becomes a liability.

What I do now

I start with the raw API. If I find myself writing the same retrieval or memory boilerplate for the third time, I extract it into a module. If that module starts looking like LangChain, I consider whether the real solution is just using LangChain.

That rarely happens. Most LLM features I ship are small enough that the raw API stays the right tool. The projects where LangChain genuinely helped were all RAG-heavy with multiple document types and a real need for the loader ecosystem.

Use LangChain when the problem matches what it is good at. Do not use it because it feels like the grown-up thing to do.