Skip to content

Execution API: Improve task instance logging with structlog context #50120

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
May 2, 2025

Conversation

kaxil
Copy link
Member

@kaxil kaxil commented May 2, 2025

This change makes it easier to debug state transitions in the API-server, especially when things go wrong!

  • Added ti_id to all log contexts using structlog's bind_contextvars
  • Remove redundant ti_id from individual log messages
  • Add more detailed debug logs for important operations

Now:
image

Before:

image

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

- Added `ti_id` to all log contexts using structlog's `bind_contextvars`
- Remove redundant ti_id from individual log messages
- Add more detailed debug logs for important operations
@kaxil kaxil added the backport-to-v3-0-test Mark PR with this label to backport to v3-0-test branch label May 2, 2025
@kaxil kaxil added this to the Airflow 3.0.1 milestone May 2, 2025
@kaxil kaxil requested review from ashb and amoghrajesh as code owners May 2, 2025 12:07
@boring-cyborg boring-cyborg bot added the area:API Airflow's REST/HTTP API label May 2, 2025
Copy link
Member

@gopidesupavan gopidesupavan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice update :)

@kaxil kaxil merged commit c9b2dca into apache:main May 2, 2025
99 checks passed
@kaxil kaxil deleted the use-structlog-exec-api branch May 2, 2025 17:44
github-actions bot pushed a commit that referenced this pull request May 2, 2025
…og context (#50120)

- Added `ti_id` to all log contexts using structlog's `bind_contextvars`
- Remove redundant ti_id from individual log messages
- Add more detailed debug logs for important operations
(cherry picked from commit c9b2dca)

Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
Copy link

github-actions bot commented May 2, 2025

Backport successfully created: v3-0-test

Status Branch Result
v3-0-test PR Link

github-actions bot pushed a commit to aws-mwaa/upstream-to-airflow that referenced this pull request May 2, 2025
…og context (apache#50120)

- Added `ti_id` to all log contexts using structlog's `bind_contextvars`
- Remove redundant ti_id from individual log messages
- Add more detailed debug logs for important operations
(cherry picked from commit c9b2dca)

Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
kaxil added a commit that referenced this pull request May 2, 2025
…og context (#50120) (#50131)

- Added `ti_id` to all log contexts using structlog's `bind_contextvars`
- Remove redundant ti_id from individual log messages
- Add more detailed debug logs for important operations
(cherry picked from commit c9b2dca)

Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:API Airflow's REST/HTTP API backport-to-v3-0-test Mark PR with this label to backport to v3-0-test branch
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy