Content-Length: 473086 | pFad | http://github.com/apache/airflow/pull/50117

28 Implement slice on LazyXComSequence by uranusjr · Pull Request #50117 · apache/airflow · GitHub
Skip to content

Implement slice on LazyXComSequence #50117

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 2, 2025

Conversation

uranusjr
Copy link
Member

@uranusjr uranusjr commented May 2, 2025

Following up #50011 (comment)

I decided to split index and slice access to their separate endpoints, instead of reusing the GetXCom endpoint. This duplicates code a bit, but the input parameters are now a lot easier to reason with. It's unfortunate FastAPI does not natively allow unions on Query(), or this could be implemented a lot nicer.

  • Fix tests on index access
  • Add unit tests for slice api
  • Add new request types to supervisor tests
  • Add unit tests for slice access in sdk
  • Cadwyn migration

@boring-cyborg boring-cyborg bot added area:API Airflow's REST/HTTP API area:task-sdk labels May 2, 2025
@uranusjr uranusjr force-pushed the dyn-map-slice-access branch from ef91af5 to 6694e1a Compare May 2, 2025 09:20
@uranusjr uranusjr force-pushed the dyn-map-slice-access branch from 6694e1a to 234e828 Compare May 12, 2025 04:22
@uranusjr uranusjr force-pushed the dyn-map-slice-access branch 4 times, most recently from 87d2547 to 7c366e1 Compare May 21, 2025 23:02
@uranusjr uranusjr marked this pull request as ready for review May 21, 2025 23:18
@uranusjr uranusjr force-pushed the dyn-map-slice-access branch from 7c366e1 to 509670c Compare May 22, 2025 00:23
@uranusjr uranusjr added this to the Airflow 3.1.0 milestone May 22, 2025
Copy link
Contributor

@amoghrajesh amoghrajesh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good, few comments

@uranusjr uranusjr force-pushed the dyn-map-slice-access branch from 7493e1f to 0154964 Compare May 28, 2025 07:14
Copy link
Contributor

@amoghrajesh amoghrajesh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on the code, we will not need the cadywn migration anymore if we plan to keep the old API as-is.

This is because:

  1. older client <=> new server will work as old API still works (no changes made there)
  2. newer client <=> new server will work as it will use new API
  3. newer client <=> old server is not a case we should be too worried about as its about forward compat, which is not of paramount importance as of now. We need to figure out task sdk, airflow core versioning even before that.

@uranusjr i think you can safely remove the cadwyn migration in this case.

Copy link
Member

@kaxil kaxil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 comment needs addressing, rest lgtm

Copy link
Contributor

@amoghrajesh amoghrajesh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM pending kaxil's comments and a simple test with old client, new server

@uranusjr uranusjr force-pushed the dyn-map-slice-access branch 2 times, most recently from d06ac9b to c4418de Compare June 1, 2025 13:42
@uranusjr uranusjr force-pushed the dyn-map-slice-access branch from c4418de to e87b4be Compare June 2, 2025 04:03
I decided to split index and slice access to their separate endpoints,
instead of reusing the GetXCom endpoint. This duplicates code a bit, but
the input parameters are now a lot easier to reason with. It's
unfortunate FastAPI does not natively allow unions on Query(), or this
could be implemented a lot nicer.
@uranusjr uranusjr force-pushed the dyn-map-slice-access branch from e87b4be to 84b36c4 Compare June 2, 2025 07:27
Copy link
Contributor

@amoghrajesh amoghrajesh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep this is good! Thanks!

@uranusjr uranusjr merged commit f8abdcd into apache:main Jun 2, 2025
97 checks passed
@uranusjr uranusjr deleted the dyn-map-slice-access branch June 2, 2025 08:13
kaxil pushed a commit that referenced this pull request Jun 3, 2025
kaxil pushed a commit that referenced this pull request Jun 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:API Airflow's REST/HTTP API area:task-sdk
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants








ApplySandwichStrip

pFad - (p)hone/(F)rame/(a)nonymizer/(d)eclutterfier!      Saves Data!


--- a PPN by Garber Painting Akron. With Image Size Reduction included!

Fetched URL: http://github.com/apache/airflow/pull/50117

Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy