The LLM observability SDKs that PyPI projects actually import

Published 14 November 2026 · Data as of 13 November 2026

Scripts and results for this post on GitHub

Lists of LangSmith and Langfuse alternatives rank tools by features, prices and GitHub stars. Of the seven top-ranking lists we read, six are published by a vendor that puts its own product first. This post ranks 22 LLM observability SDKs by a different measure: how many published Python projects import each one in their code.

py-code.org reads the code in the latest release of 354,763 PyPI projects and records every import. We counted the projects that import each SDK. We left out the vendor's own add-on packages, such as langfuse-* packages, so the count shows projects built by other people.

The short answer

LLM observability tools ranked by the projects that import them

The table counts the projects whose code imports each SDK. "Of the top 60,000" counts the ones among the 60,000 most downloaded projects on PyPI. The downloads column is the SDK package's own downloads in the last month, from pypistats.org.

Langfuse vs LangSmith: more projects import Langfuse, more installs pull in LangSmith

571 projects import langfuse, and 278 import langsmith. Downloads point the other way: langsmith had 93 million downloads in the last month and langfuse had 31 million. The reason is the dependency tree. langchain-core lists langsmith as a required dependency, so every install of LangChain or LangGraph also installs LangSmith, whether or not anyone traces anything.

The import count also misses some LangSmith users. LangSmith's docs say that "no extra code is needed to log a trace" from a LangChain app: setting the environment variable LANGSMITH_TRACING=true and an API key is enough (LangSmith docs). A project that traces this way never imports langsmith. So for LangChain code, the 278 is a lower bound.

Langfuse and LangSmith also differ in licence. Langfuse's repository is MIT licensed, except for its ee folders, and can be self-hosted (Langfuse on GitHub). ClickHouse acquired Langfuse in January 2026, and Langfuse's announcement says it "stays open source and self-hostable" (Langfuse blog). LangSmith is made by LangChain, the company behind the LangChain framework.

Langfuse users connect it in several ways. Among the projects that import it, 117 import its LangChain callback (langfuse.langchain), 50 import its drop-in OpenAI client (langfuse.openai), and 54 import the observe decorator directly.

Which observability SDK goes with which agent framework

Each framework's users pick a different tracing SDK. The table lists, for the most imported agent frameworks, how many of the projects that import the framework also import any observability SDK, and the three SDKs they import most.

• Pydantic AI and Logfire. 103 of the 239 projects that import Logfire also import Pydantic AI. Both come from the Pydantic team, and the full pydantic-ai package installs Logfire as an extra (pydantic-ai on PyPI).
• LangChain and LangGraph projects import Langfuse first and LangSmith second.
• The newer agent SDKs (Claude Agent SDK, OpenAI Agents SDK and Google ADK) import Arize's OpenInference more often than Langfuse, though the numbers are small: 29 against 26 projects for the Claude Agent SDK. OpenInference publishes one instrumentation package per framework, such as openinference-instrumentation-openai-agents. CrewAI projects are split evenly between Langfuse and OpenInference.

OpenTelemetry-based SDKs: OpenInference, OpenLLMetry, Logfire and OpenLIT

Several SDKs build on OpenTelemetry, so their traces can go to any backend that accepts OpenTelemetry data, not only the vendor's.

• OpenInference is Arize's "set of conventions and plugins that is complementary to OpenTelemetry", supported natively by Arize Phoenix and Arize AX, and usable with any OpenTelemetry-compatible backend (OpenInference on GitHub). So OpenInference records the traces, and Phoenix is one place to view them. The table counts both in one row: 266 projects import either, 220 of them import openinference, and 46 import only Phoenix's or Arize's own modules. By downloads, openinference-semantic-conventions, which every OpenInference instrumentation package and Phoenix require, had 12 million in the last month, and arize-phoenix itself had 0.6 million.
• OpenLLMetry is Traceloop's "set of extensions built on top of OpenTelemetry" (OpenLLMetry on GitHub). Its instrumentation packages carry the plain names opentelemetry-instrumentation-openai, -anthropic and -langchain, so many projects use OpenLLMetry without importing anything called Traceloop. Counting the traceloop module and these instrumentation packages together, 152 projects import OpenLLMetry.
• Pydantic Logfire (239 projects) is also built on OpenTelemetry. Of the projects that import it, 73 call logfire.instrument_pydantic_ai.
• OpenLIT is imported by 13 projects.

Langfuse and LangSmith alternatives, by usage

After the top four, the counts drop quickly:

• MLflow Tracing (121 projects). The mlflow package also covers experiment tracking and model serving, so its 22 million downloads in the last month do not measure tracing use. Of the projects that use its tracing, 15 also import DSPy.
• W&B Weave (54) and Comet Opik (50).
• AgentOps (26), Braintrust (25) and Laminar (20). Braintrust had 8.4 million downloads and Laminar 4.3 million, far more than their import counts suggest.
• LangWatch (13), TruLens (11), Lunary (10), Literal AI (10) and PromptLayer (6).
• Datadog LLM Observability (6). It ships inside ddtrace, Datadog's general tracing package, so we counted only imports of ddtrace.llmobs.
• Helicone (0 projects). Helicone works as a gateway: its quick start points the OpenAI client's base URL at Helicone (Helicone docs), so code never imports it. Mintlify acquired Helicone in March 2026 and said it "will continue operating in maintenance mode" (Mintlify).

Fewer than 1 in 20 LLM projects import a tracing SDK

25,906 projects import an LLM SDK (OpenAI, Anthropic or LiteLLM) or one of the agent frameworks we counted. Only 1,235 of them (4.8%) also import an LLM observability SDK. Across all 354,763 projects, 1,597 import one.

This understates tracing in deployed apps. Many PyPI projects are libraries, and we expect a library to leave tracing to the application that uses it. LangSmith with LangChain and Helicone can also trace an app without an import in its code. What the count does show is which SDKs developers write code against when they add tracing to something they publish.

Evaluation libraries and gateways: Ragas, DeepEval and Portkey

Evaluation libraries score an LLM's answers rather than record traces, so they are not in the ranking above. The two most imported are Ragas (143 projects) and DeepEval (143 projects). Portkey, an LLM gateway, is not in the ranking either: 35 projects import it.

How to check who uses an SDK

• On py-code.org, search for a module to see the most downloaded projects that import it, with the file and line: langfuse, langsmith, logfire.
• From a terminal: uvx py-code-org who-uses langfuse lists the same projects.

You can count the importers yourself on the Hugging Face copy of the index, in your browser or with DuckDB. The query below counts every project that imports each plain module name. Like the table, it leaves out each vendor's own packages and the iflow-mcp* copies, so most of its counts match the table's. The rows for OpenInference and Traceloop are lower, because the table also counts other modules for them, such as Phoenix's and Traceloop's instrumentation modules.

In your browser, on Hugging Face

The dataset's SQL console runs DuckDB inside your browser, and you need no account. Open this query in the console. The first run can stop with "Table with name refs does not exist", because it starts before the table has loaded. Click Run Query again: the console then loads refs and runs the query. To type it yourself, open the dataset viewer, click SQL Console, click refs under Subsets and Splits to load it, and use refs as the table name:

SELECT module, count(DISTINCT project) AS projects
FROM refs
WHERE module IN ('langfuse', 'langsmith', 'logfire', 'openinference', 'traceloop',
                 'weave', 'opik', 'agentops', 'braintrust', 'lmnr')
  -- leave out each SDK's own packages (langfuse-*, openinference-*) and the iflow-mcp copies
  AND NOT starts_with(lower(project), 'iflow-mcp')
  AND NOT starts_with(replace(replace(lower(project), '_', '-'), '.', '-') || '-', module || '-')
GROUP BY module
ORDER BY projects DESC;

In our test the first run took about 4 minutes, most of it loading the 40 million rows of refs into the browser. Later queries took a few seconds. The console reads the latest build of the dataset, which is rebuilt weekly. It matched this post on 14 November 2026, and its counts will drift after the next rebuild.

With DuckDB, on the exact snapshot

DuckDB reads the same Parquet files from Hugging Face in place. The @9d81d70d… in each path pins the 13 November 2026 build, so this query always gives the same numbers:

SELECT module, count(DISTINCT project) AS projects
FROM 'hf://datasets/pypi-data/py-code@9d81d70d6519d77f4dee4811b3e3a5d88379c17d/refs.parquet'
WHERE module IN ('langfuse', 'langsmith', 'logfire', 'openinference', 'traceloop',
                 'weave', 'opik', 'agentops', 'braintrust', 'lmnr')
  -- leave out each SDK's own packages (langfuse-*, openinference-*) and the iflow-mcp copies
  AND NOT starts_with(lower(project), 'iflow-mcp')
  AND NOT starts_with(replace(replace(lower(project), '_', '-'), '.', '-') || '-', module || '-')
GROUP BY module
ORDER BY projects DESC;

It returned:

┌───────────────┬──────────┐
│    module     │ projects │
│    varchar    │  int64   │
├───────────────┼──────────┤
│ langfuse      │      571 │
│ langsmith     │      278 │
│ logfire       │      239 │
│ openinference │      227 │
│ traceloop     │       74 │
│ weave         │       54 │
│ opik          │       50 │
│ agentops      │       26 │
│ braintrust    │       26 │
│ lmnr          │       20 │
└───────────────┴──────────┘
  10 rows        2 columns

The script behind every number in this post, with the exact rules for each SDK, is in the pypi-data/blog repository with its results.

How the numbers were counted

• Which projects. The index holds the latest release of 354,763 projects: every project released between 13 November 2025 and 13 November 2026, plus the 60,000 most downloaded projects even when their latest release is older.
• What counts as using an SDK. A project counts when the code in its latest release imports the SDK's module anywhere, including tests and examples. Environment-variable setups and proxies, which need no import, are not counted.
• Vendor packages left out. Projects whose names start with the SDK's own name, such as langfuse-* or openinference-*, are not counted for that SDK. We also dropped 2,144 iflow-mcp_* projects, which are copies of other people's MCP servers republished under one prefix.
• Downloads. Package downloads are for the last month, from pypistats.org, fetched on 14 November 2026. "Of the top 60,000" uses the index's own ranking, which comes from ClickHouse's public PyPI dataset.

Frequently asked questions

Sources

• py-code.org dataset on Hugging Face, revision 9d81d70d (13 November 2026)
• pypi-data/blog, the script and results for this post
• pypistats.org, monthly downloads (fetched 14 November 2026)
• LangSmith docs: trace with LangChain
• langchain-core on PyPI (requires langsmith)
• Langfuse on GitHub and Langfuse is joining ClickHouse (January 2026)
• OpenInference on GitHub
• OpenLLMetry on GitHub
• Helicone quick start and Mintlify acquires Helicone (March 2026)