{"slug": "profile-your-polars-queries", "title": "Profile your Polars queries", "summary": "Polars released its query profiler for open source users for free, enabling profiling via a single line, `pl.Config.enable_monitoring()`, after installing `polars_cloud`. The profiler hooks into the streaming engine, sending the optimized logical and physical plans plus per-node progress telemetry every 5 seconds to https://cloud.pola.rs/portal/, while query data stays in the user's environment; Polars says an MCP for agents is coming soon.", "body_md": "TL;DR; Add `pl.Config.enable_monitoring()` to your python environment to enable profiling of your query allowing you to view your query’s progress live and find & optimize bottlenecks.\n\nOur query profiler is now available for open source Polars users for free<sup>[1](#user-content-fn-1)</sup>. This enables users to debug & profile queries running on their own (local) infrastructure using an intuitive interface and soon an MCP for your agents. The profiler gets you the most detailed information of what Polars during execution. You get a glimpse under the hood of the query engine. You can see rows flowing through the query, see how many rows are filtered which join produces most data and much more.\n\nWith this information you can point out bottle necks in your queries, recommend optimizations (using an agent) and act as a live progress indicator for long running queries. The query is executed locally and sends live telemetry data to the platform. This functionality is available for the streaming and distributed engine.\n\n## Getting started\n\nUsing the query profiler is a one line change in your data pipelines. First `pip install polars_cloud` in your python environment. This enables the functionality to send telemetry data to the platform. Second, enable the flag `pl.Config.enable_monitoring()` in the root of your script.\n\nTry it out today using the script below:\n\n``` python\nfrom datetime import date\n\nimport polars as pl\npl.Config.enable_monitoring()\n\nBASE = \"s3://polars-public-datasets/tpch/sf1\"\nOPTS = {\"aws_skip_signature\": \"true\", \"aws_region\": \"eu-west-1\"}\nlineitem = pl.scan_parquet(f\"{BASE}/lineitem/*.parquet\", storage_options=OPTS)\norders = pl.scan_parquet(f\"{BASE}/orders/*.parquet\", storage_options=OPTS)\nlate = lineitem.filter(pl.col(\"l_commitdate\") < pl.col(\"l_receiptdate\"))\n\nresult = (\n    orders.join(late, left_on=\"o_orderkey\", right_on=\"l_orderkey\", how=\"semi\")\n    .filter(pl.col(\"o_orderdate\").is_between(date(1993, 7, 1), date(1993, 10, 1), closed=\"left\"))\n    .group_by(\"o_orderpriority\")\n    .agg(pl.len().alias(\"order_count\"))\n    .sort(\"o_orderpriority\")\n    .collect()\n)\n```\n\n## How it works\n\nThe profiler hooks into the streaming engine and sends back telemetry data at key points:\n\n- After query planning & optimization the optimized plan (logical & physical) is sent\n- During query execution the progress of each node is sent at fixed intervals (currently every 5s)\n- Once the query is finished a final flush is done to ensure accurate metrics and timings\n\n**Security & Privacy**\n\nOnly the query plan is shared with the platform, the data never leaves your environment.\n\n## Analyzing your Query\n\nIf you are in charge of running data pipelines, the query profiler is a great tool for analyzing and optimizing (historical) queries. It allows you to observe where time is spent, identify bottlenecks and optimize your queries.\n\nDuring a query (or for historical queries) you can view the progress at [https://cloud.pola.rs/portal/](https://cloud.pola.rs/portal/) under queries. Clicking on the query shows the high level details and both plans.\n\nThe logical plan contains the query plan after optimizations and the physical plan contains the detailed execution nodes with performance metrics included.\n\nIn the query above we can see that 24.9 MiB + 15 MiB ~ 40 MiB were loaded from S3 and the majority of the CPU time was spent on the join. This query was bound by I/O speed as the CPU time of the join was low (~103ms) compared to total query time (~5s) indicating the join was waiting on data loaded from S3. For  more details on how to use the query profiler go to [our user guide](https://docs.cloud.pola.rs/polars-cloud/run/query-profile).\n\n## Try it out!\n\nRun your queries and let us know your experience by commenting on our [Discord](https://discord.gg/4UfP5cfBE7) or adding feature requests to our [issue tracker](https://github.com/pola-rs/polars/issues).\n\n## Up Next\n\nWe are already working on the next big release for the query profiler. In the upcoming weeks you can expect the release of our MCP server which allows your agents to profile queries. Additionally we are adding node specific metrics to our plans to increase the capabilities to analyze your queries.\n\n## Footnotes\n\n1. \nUsage is limited to fair use to prevent excessive platform use. [↩](#user-content-fnref-1)", "url": "https://wpnews.pro/news/profile-your-polars-queries", "canonical_source": "https://pola.rs/posts/profile-local-queries/", "published_at": "2026-09-17 00:00:00+00:00", "updated_at": "2026-09-17 14:28:07.270915+00:00", "lang": "en", "topics": ["developer-tools", "mlops", "agent-protocols"], "entities": ["Polars", "polars_cloud", "pl.Config.enable_monitoring()", "https://cloud.pola.rs/portal/", "Discord", "GitHub"], "alternates": {"html": "https://wpnews.pro/news/profile-your-polars-queries", "markdown": "https://wpnews.pro/news/profile-your-polars-queries.md", "text": "https://wpnews.pro/news/profile-your-polars-queries.txt", "jsonld": "https://wpnews.pro/news/profile-your-polars-queries.jsonld"}}