Skip to content

pandas.md sometimes takes 86–148 s to execute instead of ~12 s #112

Open

Description

@mmcky

In the full cache.yml builds, pandas.md normally executes in 10–16 s. Two recent runs took 86 s and 148 s. That is still well under the 600 s per-cell timeout, but it is the lecture most likely to time out next if it gets worse.

Data

pandas.md execution time in the last 12 cache.yml runs, from the "Executed notebook in X seconds" lines in the job logs. Other lectures are shown for comparison.

Run (UTC) Commit anaconda pandas polars numba jax_intro
2026-10-13 06:00 87fcd31 2026.07 11.52 s 10.01 s 26.07 s 24.78 s
2026-10-13 05:11 a561ca1 2026.07 147.58 s 9.68 s 25.89 s 23.97 s
2026-10-13 04:49 2ca72b8 2025.12 16.14 s 10.99 s 20.34 s 20.43 s
2026-10-13 03:35 a2b384e 2025.12 11.89 s timed out¹ 20.24 s 21.21 s
2026-10-12 03:35 90336d2 2025.12 86.10 s timed out¹ 16.90 s 18.64 s
2026-10-05 03:29 90336d2 2025.12 11.63 s 9.14 s 26.20 s 23.94 s
2026-09-28 03:27 90336d2 2025.12 12.52 s 9.58 s 20.31 s 20.64 s
2026-09-21 03:24 90336d2 2025.12 10.50 s 8.76 s 26.32 s 25.72 s
2026-09-14 03:24 90336d2 2025.12 11.62 s 10.11 s 26.84 s 25.22 s
2026-09-07 03:23 90336d2 2025.12 11.23 s 9.52 s 26.53 s 26.33 s
2026-08-31 03:23 b727528 2025.12 11.94 s 9.72 s 26.30 s 25.12 s
2026-08-24 03:25 b727528 2025.12 10.78 s 9.23 s 26.23 s 25.12 s

¹ The polars FRED hang, fixed upstream in QuantEcon/lecture-python-programming#647 and synced here in #108.

What the data says

  • Not tied to the anaconda 2026.07 upgrade (⬆️ Bump anaconda from 2025.12 to 2026.07 #102). There is one slow run under each environment, and the most recent 2026.07 run took 11.52 s.
  • Specific to pandas.md. In both slow runs every other lecture ran at its usual speed. That includes polars, which also fetches FRED and Yahoo Finance data, and in the 05:11 run it took 9.68 s.
  • Not visible in the logs. Neither the jupyter-book log nor the cached executed notebooks record per-cell timings, so the slow cell is unknown.

Suspects

These are the network calls in pandas.md, ranked by what is unique to this lecture:

  1. World Bank API through wbgapi: wb.series.info(...) and wb.data.DataFrame(...). No other lecture uses this API.
  2. !pip install --upgrade wbgapi and !pip install --upgrade yfinance: runtime installs from PyPI.
  3. FRED: two requests.get(...) calls plus pd.read_csv(url), which fetches through urllib. The polars lecture's FRED fetch, also via requests, was fast in the same slow run, which makes this less likely.
  4. Yahoo Finance: the yf.Ticker(...) loop. polars runs the same loop and was not slow.

Both slow runs came from GitHub-hosted runners. Run locally, each of these cells finished in under 3 s.

Possible next steps

  • Find the slow cell. For example, execute pandas.md a few times in CI with per-cell timing recorded, or time the network cells separately on a runner.
  • If it is the World Bank API: snapshot the small, fixed series the lecture reads into QuantEcon/data-lectures, as was done for test_pwt.csv. Alternatively, give the calls a timeout.
  • The English lecture has the same cells, so whatever is found likely applies upstream, in QuantEcon/lecture-python-programming, and in the other editions.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Wyb6EzbNzY5a8uuJPnDtUb

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions