Skip to content

FIX Allow more specific controller limits by specifying filepath, not just prefix - #278

Open
itamarst wants to merge 10 commits into
joblib:masterfrom
itamarst:custom-controller-limits
Open

itamarst wants to merge 10 commits into
joblib:masterfrom
itamarst:custom-controller-limits

Conversation

@itamarst

@itamarst itamarst commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #277

Previously prefix was used as the controlling key to identify LibControllers, but prefix is not guaranteed to be unique. Thus the switch to filepath.

While I was at it, I deleted some code that did unnecessary work of setting unused private attributes, possibly remnants of some old refactoring.

This is the companion to #275 so joblib can more reliably say "OpenBLAS+pthreads, if used, will be either 1 or cpu_count() cores and that's it". Given this change, joblib would do something like this:

def set_num_threads(num_threads):
    tp_controller = get_cached_controller()
    limits = []
    for lib_controller in tp_controller.lib_controllers:
        # filepath for threadpoolctl that has this PR, prefix for backwards compat with older versions
        limit = {"filepath": lib_controller.filepath, "prefix": lib_controller.prefix}
        if isinstance(lib_controller, OpenBLASController) and lib_controller.threading_layer != "openmp":
            cpus = cpu_count(process_wide=True)
            if num_threads >= 0.75 * cpus:
                limit["num_threads"] = 1
            else:
                limit["num_threads"] = cpus
        else:
            limit["num_threads"] = num_threads
        limits.append(limit)
    tp_controller.limit(limits=limits)

Also, remove an unnecessary attribute and correct a docstring.
@itamarst itamarst changed the title FIX More specific controller limits FIX Allow more specific controller limits by specifying filepath, not just prefix Oct 26, 2026
@itamarst
itamarst marked this pull request as ready for review October 26, 2026 17:30

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

limits= dict and list form don't allow full customizaton

1 participant