Framework behaviour is decided below the public API

A framework's public API is the set of names you call and the arguments you pass them. Its behaviour is a much larger thing: every default that fills in an argument you did not supply, every ordering decision about when a callback runs relative to a state write, and every adapter that has to translate the object you built into the bytes that leave the process. None of that is in the signature you called, most of it is not in the documentation, and a version number that promises the signatures have not moved promises nothing at all about the rest.

That gap is not a flaw in any particular library. A public API is a contract about what you may call, and the useful work a runtime does is precisely the work you did not have to describe, which means the useful work is exactly the part with no contract on it. The practical consequence is that some fraction of the questions you will have about your own agent's behaviour can only be answered by reading somebody else's source, and the skill worth having is getting to the right file quickly enough that reading it feels like debugging rather than like an expedition.

Finding the file in under a minute#

The first obstacle is usually locating the code, and there are three moves that between them cover nearly every case. The direct one is asking Python where a module lives, which beats searching GitHub because it gives you the version you are actually running rather than whatever is on the default branch.

python -c "import langchain_openai.chat_models.base as m; print(m.__file__)"
# /srv/app/.venv/lib/python3.12/site-packages/langchain_openai/chat_models/base.py

pip show langchain-openai | head -3
# Name: langchain-openai
# Version: 0.3.15

rg -n "parallel_tool_calls" /srv/app/.venv/lib/python3.12/site-packages/langchain_openai/
# chat_models/base.py:1184:            payload["parallel_tool_calls"] = False

The distinction that first command turns on is worth being explicit about, since it is the reason to prefer it over a browser. The installed tree is the artefact that was built, published and resolved by your lockfile; the repository is the tree the maintainers work in. They are usually the same code, and "usually" is doing real work in that sentence — packaging steps, release branches and generated files all make it possible for the published artefact to differ from the branch you would land on. Reading the file the interpreter imported removes that possibility entirely, at no cost.

The second move is a breakpoint, for the cases where you do not know which function to grep for. Setting PYTHONBREAKPOINT and dropping into the debugger at the moment the request is built, then walking up the stack with where, names every layer between your call and the wire, which is exactly the list you could not see. On an agent framework that stack is often nine to fifteen frames deep, and reading the frame names once is worth more than any architecture diagram, because it is the real call path rather than the intended one.

The third is logging the payload at the transport boundary, since the argument is ultimately about what went over the wire. Most HTTP clients can be made to log the request body with a single environment variable or a one-line event hook, and a captured body settles questions that source reading only makes plausible.

import httpx, json, logging

def log_request(request: httpx.Request) -> None:
    logging.info("→ %s %s", request.method, request.url)
    logging.info("%s", json.dumps(json.loads(request.content)["parallel_tool_calls"]))

client = httpx.Client(event_hooks={"request": [log_request]})

The three moves answer different questions and it is worth keeping them separate. Source reading tells you what the code can do. A stack trace tells you which code ran. A captured body tells you what actually left the machine, and that last one is the only evidence that cannot be argued with, because every other artefact is an account of how the bytes were produced rather than the bytes.

Reading the adapter that changed the request#

In one such incident the captured body carried "parallel_tool_calls": false, which nothing in the application had set. Fourteen lines into the file rg had pointed at, the reason was visible.

# site-packages/langchain_openai/chat_models/base.py  (0.3.15)
def _get_request_payload(self, input_, *, stop=None, **kwargs) -> dict:
    payload = super()._get_request_payload(input_, stop=stop, **kwargs)
    tools = payload.get("tools") or []
    if any(t.get("function", {}).get("strict") for t in tools):
        # strict schemas and parallel tool calls are not both honoured upstream
        payload["parallel_tool_calls"] = False
    return payload

The team had enabled strict schemas four months earlier, for argument validation — strict mode being the provider-side setting under which the model's tool arguments are constrained to the JSON schema you declared, so that a required field cannot be omitted and an invented one cannot be added — and had been getting parallel tool calls the whole time because the adapter had not yet made the two interact. A patch release connected them.

What that connection costs takes four steps, and each of them is checkable. The flag being false means the provider will not return more than one tool call in a single assistant turn. An agent that needs three tools for a request therefore cannot get them in one turn, so it needs three turns, each returning one tool call. Each turn is a complete round trip: send the whole conversation, wait for generation, execute the tool, send again. And round trips do not overlap, because turn two cannot begin until turn one's result is in the message list. So runs that used to make one model call and then execute three tools concurrently were now making three model calls, and three sequential round-trips is where the extra time appeared.

Put the observed numbers against that chain and they agree. The p95 went from 12.4 seconds to 31.8 seconds, a difference a little under twenty seconds, and two extra round-trips at roughly six and a half seconds each account for essentially all of it — with the ratio between the two figures landing close to the two-and-a-half-fold slowdown the team later reported upstream. The agreement is worth having because of what it rules out. A regression that showed up as each call getting slower would be a different diagnosis entirely — a model change, a network path, a serialisation cost — and the arithmetic separates "more calls" from "slower calls" before anybody opens a profiler.

It is also worth stating what that agreement does not license. Matching the gap to a count of round trips establishes that round trips were added; it says nothing about which mechanism added them. Any change that prevents more than one tool call per turn produces the same arithmetic, so the numbers narrow the search to the layer that decides that and stop there. Naming the flag needed the captured body, and naming the line that sets it needed the source.

The documentation said nothing about this. The changelog entry, "fix: honour strict schemas in tool payloads", was accurate and told you nothing either, and the reason is structural rather than negligent: a changelog describes the intent a maintainer had, and the mechanism chosen to achieve that intent is an implementation detail from where they are standing. It is not an implementation detail from where you are standing, and no discipline about writing changelogs closes that gap, because the author cannot know which of their internal choices your system depends on.

Diffing the two versions you actually ran#

Finding the current behaviour is half the work; the other half is establishing what changed, because that is what tells you whether to pin, adapt or revert. The reliable version of this does not involve the project's git history at all, since the packaged artefact and the repository are not always the same tree. Downloading both versions and diffing the installed trees answers the question in about ninety seconds.

pip download langchain-openai==0.3.14 --no-deps --no-binary :all: -d /tmp/v14
pip download langchain-openai==0.3.15 --no-deps --no-binary :all: -d /tmp/v15
for v in 14 15; do (cd /tmp/v$v && tar xf *.tar.gz); done
diff -ru /tmp/v14/langchain_openai-0.3.14 /tmp/v15/langchain_openai-0.3.15 \
  | grep -A5 -B5 parallel_tool_calls
# +        if any(t.get("function", {}).get("strict") for t in tools):
# +            payload["parallel_tool_calls"] = False

A diff scoped to the symptom is usually four to forty lines, which is small enough to read and small enough to paste into an issue. It also converts the conversation with the maintainers from "our agent got slow" into "0.3.15 sets parallel_tool_calls to false whenever any tool declares strict, which was not documented and changes latency by 2.5x for multi-tool turns" — a report that gets answered, because it contains the diagnosis.

The same technique settles arguments about your own dependency graph. Bumping a framework often bumps six transitive packages, and the adapter is frequently not the one named in the pull request title. Comparing the two lockfiles first, then diffing only the packages whose versions moved, keeps the search bounded.

Defaults, ordering and adapters#

Three categories account for most of what surprises people below the public API, and knowing which one you are in tells you where to look.

Defaults you never set are the largest. A retry policy that treats every exception as retryable unless it is on a short deny-list will retry a client-side timeout, which is the mechanism behind the duplicate charge from earlier in this topic, and no configuration file in your repository records that decision, because you never made it. That absence is the trap rather than the default itself: searching your own repository for a setting you are wondering about returns nothing, and nothing reads as "we are not doing that" when it means "somebody else decided". Finding these means grepping for the option name in the installed package and reading the .get(name, default) call rather than the documentation table, since the two drift.

Callback ordering is the second, and it produces bugs that look like data corruption. A metrics callback that reads the checkpoint store when on_chain_end fires will see the previous step's state on some runtimes, because the state write happens after the callback dispatch. Nothing about that is wrong; it is simply an ordering nobody wrote down, and the code around the dispatch site is the only place it is stated. What makes the resulting bug so convincing is that the data is correct and merely one step old, so it survives every check that looks at whether values are plausible.

Adapters are the third, and they are the most volatile because they track a moving target. Every provider changes its API surface, and the adapter absorbs those changes on your behalf, which is valuable right up to the moment it absorbs one you were depending on. Adapter code is also where the framework's abstraction is thinnest and easiest to read: it takes a normalised object and produces a JSON body, and you can hold both ends in your head.

Two things make this reading harder than reading your own code, and both have workarounds. Async frameworks bury the interesting frame behind executors and task groups, so a stack trace shows the scheduler rather than the caller; asyncio run with the debug flag set, or a breakpoint inside the coroutine rather than at the call site, restores the causal chain. Decorator-heavy libraries hide the real function behind wrappers, and inspect.unwrap on the object you actually hold, or reading func.__wrapped__, gets you to the body without guessing which decorator did what. Neither takes more than a minute once you have done it once.

The package's own test suite is the other underused source, and it is usually a better specification than the documentation because it is executable and it is kept passing. Searching the tests for the option you care about tells you what the maintainers consider correct behaviour, which distinguishes a bug from a deliberate change you disagree with — and that distinction determines whether your issue is a bug report or a feature request, and therefore how it will be handled.

Pinning is a decision with a date on it#

Once you know what moved, the immediate fix is usually to pin, and pinning is worth doing precisely rather than reflexively. An exact pin with a hash in a lockfile stops the bleeding and is honest about what it is: a deliberate decision to stay on 0.3.14 while a follow-up happens.

What makes it dangerous is leaving it unlabelled, and the mechanism is worth naming because it is not laziness. A pin with no recorded reason cannot be evaluated by anybody later: the only way to find out whether it is still needed is to remove it and see what breaks, which is an unbounded risk taken for a small tidiness gain, so every reviewer who considers it correctly declines. A pin without a reason therefore becomes permanent by the rational action of everyone who meets it, and eighteen months later somebody is trying to work out why the project cannot move past a version that predates half its dependencies.

# pyproject.toml
[project]
dependencies = [
  # Pinned 2025-05-13. 0.3.15 forces parallel_tool_calls=false when any tool
  # declares strict, which took multi-tool turns from 12.4s to 31.8s at p95.
  # Upstream issue #7412. Revisit when it lands, or drop strict schemas.
  "langchain-openai==0.3.14",
]

The durable fix is not the pin but the test that would have caught it, and it belongs at the wire, because that is the layer where the surprise happened. A test that asserts the request body — that parallel_tool_calls is present and true, that the tool count is what you declared, that the system prompt is the string you wrote — runs in milliseconds against a recorded transport and fails on the upgrade pull request rather than in production. Teams that adopt a framework and test only their own functions have no coverage at all over the thing most likely to change under them.

def test_payload_allows_parallel_tool_calls(recorded_transport):
    agent.invoke({"messages": [("user", "check order 4471")]})
    body = recorded_transport.last_request_json()
    assert body["parallel_tool_calls"] is True
    assert [t["function"]["name"] for t in body["tools"]] == ["search_orders", "issue_refund"]

The choice of assertion layer is the whole design of that test. Asserting on your own function's return value tests code that did not change. Asserting on the model's answer tests something that varies for unrelated reasons and will be flaky. Asserting on the serialised request tests the exact artefact the upgrade moved, and it is deterministic, which is why it can run on every pull request without anybody learning to ignore it.

Reading source is not the same as owning it#

The distinction matters when someone proposes the next step. Monkey-patching _get_request_payload in the application would have fixed the latency in ten minutes and created a private fork with no version bound, which breaks silently the moment the method signature changes — a class of failure considerably harder to diagnose than the one it replaced. The failure mode is specific: a patched method that no longer matches the caller's expectations usually still runs, so the symptom is wrong behaviour rather than an import error, and nothing in the traceback mentions that a patch exists. Vendoring the package has the same shape at a larger scale.

The sequence that keeps working is to read, pin with a reason, report upstream with the diff, and add the assertion at the boundary, in that order, and to accept the occasional patch only when the alternative is worse and the patch is small enough to re-derive. Each step in that order is cheap to reverse, which is the property that makes an order worth following under pressure.

Reading source also reveals something about the relationship that does not show up in a feature comparison, which is how much of your system now depends on internals that are free to move. The adapter decides your request bytes, the checkpointer decides your state format, the callback contract decides what your dashboards can see, and none of those are covered by a version-compatibility promise in the way a public function signature is.

That is a manageable amount of exposure while you are debugging one file, and it is a different quantity from the one a version policy measures. Semantic versioning makes a promise about names; every dependency above is on behaviour. The distance between those two is what the pin, the diff and the wire test between them are all attempting to cover.

It becomes a different kind of question again when someone asks how hard it would be to stop using this framework. That answer does not depend on how many adapters you have read. It depends almost entirely on what shape your persisted state is in and on how many modules have learned to speak the framework's dialect, and both of those are measurable today.