So one issue with the in-place operators is that scalars are immutable. Maybe we could implement them but make them return a copy? But maybe a in-place operation returning a copy is confusing?

Implementing __complex__ makes sense. I guess the reason it's missing is there isn't a PyNumberMethod slot for it as far as I can see: https://docs.python.org/3/c-api/typeobj.html. You'd probably need to "manually" define a function named __complex__.

This is the current behavior:

In [6]: a = np.int64(3)

In [7]: id(a)
Out[7]: 2199088643840

In [8]: a += 3

In [9]: id(a)
Out[9]: 2199088641120

I think implementing __iadd__ and making it return a copy doesn't actually change this behavior. So we should do that.

I added a 2.2.0 milestone.

I think __iadd__ and __ior__ are probably the easiest since there are PyNumberMethod slots for them.

For __complex__ someone would need to define a python function (in C) named __complex__ that does the conversion to a PyComplex.

Setting up array-api-tests to run with scalars will probably be helpful as well.
It might smoke out more missing methods.

@ngoldbaum have we decided to do this ?
I wanted to give it a shot.

@Ishankoradia go ahead. Having a milestone means we're planning to do it. We also don't claim issues, just go ahead and work on it.

Gotcha !!

@ngoldbaum I am looking at the core/__init__.pyi, i can see that for method __iadd__ we only accept NDArray. I am guessing my first step would be to add a method overload to accept scalar. Am i in the right direction ?

No, in order to implement these functions you'll need to modify the C internals of NumPy.

Here is where the PyNumberMethods struct is set up for all of the NumPy scalar types:

This is inside a file that is written in NumPy's custom templating language used for codegen internally.

You can see that none of the inplace methods listed in the CPython docs are implemented. I think we need to implement all of them, not just __iadd__ and __ior__. You'd also need to add tests and I'd also double check my analysis above that defining in-place operators that return copies is actually OK. Do other array libraries do that?

For __complex__, as noted above, you'd need to add an entry and implementation for __complex__ to the PyMethodDef array for the scalar types:

If you've never touched the CPython C API before this is probably a big project, although tbh it would be a decent way to learn about the C internals of NumPy or how to work with Python C extensions using the C API directly.

Thanks a ton @ngoldbaum , this helps a lot.
I think I can do it. Although i have limited knowledge of Python C extensions , but like you said what better to learn this & numpy C internals.

I found going through https://docs.python.org/3/extending/extending.html and https://llllllllll.github.io/c-extension-tutorial/ helped immensely to understand this stuff better.

Got it !! I will read through them before I dive in.

Thank you for sharing them.

@ngoldbaum I have spent a lot of time reading the material you shared. I have good understanding of how c extensions work. I built out 2(easy ones) and was able to run them from python

I was looking at this template file you pointed out scalarmath.c.src. Its very interesting. I think all @name@ placeholders are replaced by the correct dtypes. And i also see their corresponding method implementation in the compiled file scalarmath.c. But i can figure out where is the source code for those functions (eg. @name@_add ) coming from. Could you point me to that file ?

[updated]
Ohh is this the function @name@_ctype_add for .nb_add = (binaryfunc)@name@_add, ?

Hi, sorry for taking a few days to respond, I've been on vacation.

The @name@ part of the template system numpy uses for codegen. All files with .c.src extensions use this template system.

The templating for the block of code I linked to is set up immediately above that code:

Here, the template system is saying to replace @name@ with each of the names in the comma-separated list, one for each scalar type.

In order to implement __iadd__, __ior__, and the rest of the in-place operators, you're going to need to define new templates that define implementations for the these operators (or maybe you can just re-use the existing implementations for the non-in-place operators? not sure) then add new entries to the table I linked to above for all the in-place operators like nb_inplace_add.

@ngoldbaum no problem. Thanks for getting back. (Also, hope you had a great vacation and got good time to recharge)

So my first attempt was to use the same table. I tried to add a new entry there for .nb_inplace_add like this

But the mistake I did was, I implemented a method @name@_ctype_inplace_add instead @name@_inplace_add. I assumed ctype is also some kind of a prefix stub that is needed for the methods because i see it everywhere.

I have added this now and i see the compiled file scalarmath.c that has this dummy implementation.

1. What is that ctype prefix ?
2. Right now I have just copied the dummy implementation from _ctype_add. I guess i will have to break it down once i start implementing for each data types right based on how the logic looks like ?
3. How are inputs to these methods handled here ? Can i assume in inplace operation that the first input will be self ?

If you can clear these up , it would be great. Sorry for the back & forth. Thank you for all the help in this one.

What is that ctype prefix ?

Right now I have just copied the dummy implementation from _ctype_add. I guess i will have to break it down once i start implementing for each data types right based on how the logic looks like ?

The ctype functions are used to define the "c-level" version of the operation. There's also a "python-level" version of the operation that is defined using a second level of templating to define a function named e.g. float_add. See the template function defined starting at line 1175 in scalarmath.c.src. I guess to get this to work, you'll also need to extend this to generate in-place python wrappers, along with the C-level wrappers. Although that said, maybe you can just re-use the existing wrappers that are already defined, so just set nb_inplace_add to e.g. (binaryfunc)@name@_add.

If you look up at my analysis when I originally opened the issue - numpy scalars already let you use the in-place operators, they just return a copy, so maybe in principle you can just re-use the existing implementations that are getting used already, but now by explicitly using the slot so people can call the dunder methods directly from Python.

How are inputs to these methods handled here ? Can i assume in inplace operation that the first input will be self ?

I think that's right, although instead of asking me and then waiting for a response, you should try experimenting to see for yourself. I would need to poke around with a C debugger to figure that out.

There are spin gdb and spin lldb commands. You'll also need to make sure you're building NumPy with debug symbols. There's no need to use a debug build of CPython unless you need to step through CPython, which is not needed for most things.

If you don't like debuggers, printf debugging also works :)

What is that ctype prefix ?

Right now I have just copied the dummy implementation from _ctype_add. I guess i will have to break it down once i start implementing for each data types right based on how the logic looks like ?

The ctype functions are used to define the "c-level" version of the operation. There's also a "python-level" version of the operation that is defined using a second level of templating to define a function named e.g. float_add. See the template function defined starting at line 1175 in scalarmath.c.src. I guess to get this to work, you'll also need to extend this to generate in-place python wrappers, along with the C-level wrappers. Although that said, maybe you can just re-use the existing wrappers that are already defined, so just set nb_inplace_add to e.g. (binaryfunc)@name@_add.

If you look up at my analysis when I originally opened the issue - numpy scalars already let you use the in-place operators, they just return a copy, so maybe in principle you can just re-use the existing implementations that are getting used already, but now by explicitly using the slot so people can call the dunder methods directly from Python.

How are inputs to these methods handled here ? Can i assume in inplace operation that the first input will be self ?

I think that's right, although instead of asking me and then waiting for a response, you should try experimenting to see for yourself. I would need to poke around with a C debugger to figure that out.

There are spin gdb and spin lldb commands. You'll also need to make sure you're building NumPy with debug symbols. There's no need to use a debug build of CPython unless you need to step through CPython, which is not needed for most things.

If you don't like debuggers, printf debugging also works :)

Understood. Thank you Nathan !!
This helps a lot. I will get back in 3-4 days hopefully with something implemented.

Sorry for a late side comment, but reading this are we sure that this #27305 (comment) has been followed up:

You can see that none of the inplace methods listed in the CPython docs are implemented. I think we need to implement all of them, not just iadd and ior. You'd also need to add tests and I'd also double check my analysis above that defining in-place operators that return copies is actually OK. Do other array libraries do that?

Do other array libraries in fact do that for immutable scalars? I ask because certainly python ints do not implement __iadd__ - within python at least those methods should not be used directly, but tested through simply doing a += b (which goes through a.__add__(b) if there is no __iadd__).

That is correct, inplace operators cannot and thus must not be implemented. __complex__ makes a lot of sense to be missing, precisely because it is not an nb_ slot, but rather only a Python defined method.