I think we'd have to come up with a new function that has a different return type. A few versions later, we could deprecate eig and eigvals and point people to the new function.

We can't just change the return types though IMO because then it becomes annoying to simultaneously support several NumPy versions. We also can't generate new deprecation warnings for eig and eigvals without a replacement, because then there's nothing people can do to fix it, particularly when users trigger this in libraries.

real_if_close is not that taxing for a user---and in fact in the vast majority of use cases it's just not needed.

Quite the other way around. I think this is an intrusive change and make things really complicated for linear algebra code. Because it will require checking the imaginary part every time you have an eigvals code. I would argue that this is a change not due to usability improvement but to regularize type system that I don't find sufficiently strong reason about.

I am aware of the array api work, but this should not come at the expense of giving up the convenience of a dynamic language.

eigvals(x, /) should return an array of eigenvalues in an unspecified order;

This one is also very problematic. Tons of code relies on this behavior and the behavior is pretty much written in stone at this point. Instead of changing established conventions, the other libraries should look for ways to mimic this behavior.

I am aware of the array api work, but this should not come at the expense of giving up the convenience of a dynamic language.

Array API discussion is a for context only, and is not a motivation for this request. So let's not tangle this in. [1]

it will require checking the imaginary part every time you have an eigvals code.

Will it really? To me, it's a pure math thing: eigenvalues of a real matrix are complex-valued. They can --- by accident --- have zero imaginary parts. These zero imaginary parts can --- again, by accident --- become identically zero in floating point, but in general one needs some tolerance to decide if .imag is zero or not. And that tolerance is problem-dependent. Just like a matrix rank, which is ill-defined in inexact arithmetic.

And I stand by the OP statement that the very need to decide if eigenvalues are or are not exactly on the real axis of the complex plane is rare.

That a bunch of code has possibly already been written to expect handholding from numpy is a separate matter. A small experiment running scipy and scikit-{learn,image} test suites above seems to hint that the amount of such code is not that large. We can of course question how representative that exercise is, but by all means, if we reject the change because of potential backwards compatibility concerns, let's be clear about it and not mix up any other reasons.

[1] For the record (and I'm already on record elsewhere): IMO the array API movement is in itself not a reason to change anything, full stop. It can serve as a motivation, yes, but any change initiated by array API should be weighted on its own merits, not only consistency between array backends.

IMO this is worth doing. Having consistent output types is also better for JITs like numba or AoT compilers.

Maybe it could be done by adding a new keyword argument that forces imaginary output? In a future version we could deprecate cast_to_float=True to force people to update their code if we detect a case where a cast happens by generating a warning. Later, we'd update the default for the keyword and library authors can delete the kwarg. That would allow the array API compat to work eventually.

Either way we need a story for library authors that ideally allows them to avoid adding code that depends on NumPy version in all cases. Library authors that want to jump the gun and write version-dependent code can do so, but we shouldn't generate any new deprecation warnings that force library authors to have version-dependent code.

A recent array-api discussion [2], had a question if NumPy would consider changing this value-dependent behavior and always return a complex array for eigenvalues?

I am assuming this is the reason but otherwise what's the reason to change it?

That a bunch of code has possibly already been written to expect handholding from numpy is a separate matter.

Yes and that's how Python took off with hand-holding everywhere meaning providing a convenient interface. Otherwise we would be still coding in braces or implicit nones. So trying to reshape a dynamic language to a typed thing should not be the originator of lots of changes. I can also make the counter argument if you need type safety then you are coding in the wrong place. But that won't solve the discussion at hand so let's not get in there.

The type safe way of doing this is the classical return-two-arrays in LAPACK or other compiled sources that you get one array wr for real parts and wi for imaginary parts. It is as unnecessary as writing if any(eigs.imag() != 0.0): ... everywhere but not the end of the world. Probably, we can make it work but then again what is the reasoning. Did we get complaints, is this hindering anything if not array api?

I also want to be clear about our intentions and not make up reasons to avoid the fact that this is coming from array api. So it should go both ways.

Independent from the dtype, the eigenvalue ordering, that's a major battle altogether.

I am assuming this is the reason but otherwise what's the reason to change it?

The array API discussion linked above is a context where this issue originates, yes.

NumPy uses one convention and PyTorch uses a different convention. FTR, I'm suggesting that CuPy (which is the original motivation for the whole story) follows pytorch not numpy in cupy/cupy#8980.

that's how Python took off with hand-holding everywhere

That might be, but I'd argue that everywhere should be where reasonable, as with time we've seen both kinds of interfaces.

So for me, the reason is just the math.

With my comp phys / applied maths hat on, every time I see fp_value == 0, I freak out start thinking whether it should have a finite tolerance instead.
Which begs the question, does LAPACK really guarantee exactly zero imaginary parts for real eigenvalues?

Also, what are the applications/use cases where returning complex eigenvalues with zero imaginary parts breaks something? I don't mean existing implementations (where the scipy/scikit-learn/scikit-image test suites seem to hint at an answer), I mean computational problems. I can only think of np.roots.

I also want to be clear about our intentions and not make up reasons to avoid the fact that this is coming from array api.

Sure, this is in the OP :-).

Maybe it could be done by adding a new keyword argument that forces imaginary output? In a future version we could deprecate cast_to_float=True to force people to update their code if we detect a case where a cast happens by generating a warning. Later, we'd update the default for the keyword and library authors can delete the kwarg. That would allow the array API compat to work eventually.

Either way we need a story for library authors that ideally allows them to avoid adding code that depends on NumPy version in all cases. Library authors that want to jump the gun and write version-dependent code can do so, but we shouldn't generate any new deprecation warnings that force library authors to have version-dependent code.

With a downstream library maintainer hat on, I don't see how a keyword helps. Downstream, we'll need to first add a keyword (with a version check), then silence the deprecation warnings, then remove it. It's way simpler to just add an if np.__version__ < 2.X: ... once.

With a downstream library maintainer hat on, I don't see how a keyword helps. Downstream, we'll need to first add a keyword (with a version check), then silence the deprecation warnings, then remove it. It's way simpler to just add an if np.version < 2.X: ... once.

SciPy is a very engaged downstream. I'm thinking of people with less resources, who would prefer to have a one-liner they could use, or even retain their old code - which after all isn't broken.

Another advantage of the keyword argument approach is that we can make it so the deprecation warning only triggers if you don't specify the keyword. Anyone who wants to make their code always handle imaginaries can choose to make their code depend on numpy version and use the keyword instead of casting if it's available. Anyone who wants to keep their code as-is just explicitly sets e.g. cast_to_float=True and leaves their code alone. Anyone who doesn't care about fixing deprecations also doesn't have to do anything.

In a few numpy versions we expire the deprecation. At that point the lazy people who don't care about deprecations can just set the keyword or do the migration to always handle imaginaries. Either way it's not much work and they don't need to figure out the correct way to make their code depend on numpy version.