Skip to content

GH-29021: Convert _simd.c to multi-phase init (PEP 489) #29025

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

AA-Turner
Copy link

@AA-Turner AA-Turner commented May 21, 2025

@AA-Turner
Copy link
Author

s390x - baseline(Z13) is a transient failure: E: Failed to fetch http://ports.ubuntu.com/ubuntu-ports/dists/jammy-updates/main/binary-s390x/Packages.gz File has unexpected size (1313374 != 1313377). Mirror sync in progress? [IP: 185.125.190.39 80]

static struct PyModuleDef defs = {
.m_base = PyModuleDef_HEAD_INIT,
.m_name = "numpy._core._simd",
.m_size = 0,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was -1, did it become 0 on purpose?

Copy link
Author

@AA-Turner AA-Turner May 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A value of -1 indicates to Python that the module has global state (PyModuleDef.m_size). It forbids any use of subinterpreters, re-initialisation, etc.

Changing to 0 was perhaps a gamble, seeing if tests passed. To be more conservative though, we could keep m_size to -1 and defer changing it to a non-negative number to a future PR. What would you suggest?

A

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think we should change it to 0 in a future PR. Please only change it in a single PR, lets get one of these in before the mass move of all the other ones. Does the change impact performance? I would suggest redoing this in #29030, or one of the others that touch a commonly used module, and then running the benchmarks to see if there is a regression when using multi-phase init.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I've updated #29030 to keep m_size = -1.

Is there infrastructure to benchmark a PR? My laptop isn't an ideal environment & I don't have a spare computer to run the local benchmarks.

Copy link
Author

@AA-Turner AA-Turner May 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From https://github.com/numpy/numpy/actions/runs/15179012580/job/42684537494?pr=29030

m_size may not be negative for multi-phase initialization

I'll get the Python documentation updated, this isn't very obvious! In the meantime, I'll revert the change to #29030.

I believe 0 is the correct value to keep, as there is currently no per-module state and hence nothing to allocate memory for.

@rgommers
Copy link
Member

rgommers commented May 24, 2025

This change looks quite annoying:

-    import_array();
+    import_array1(-1);

import_array is public API and used by a ton of extension modules, binding generators, and docs.

At a glance from your PRs, there seems to be a larger issue in the CPython design with the normal meaning of a NULL return used for an error condition no longer being true. Returning 0 on success and -1 on error is going to lead to a lot of churn. What is going on there?

EDIT: discussion on this comment moved to #29030 (comment)

@AA-Turner AA-Turner marked this pull request as draft May 24, 2025 06:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy