Skip to content

Mask variable values in task logs only if the variable key is sensitive #50775

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
May 21, 2025

Conversation

amoghrajesh
Copy link
Contributor

For a dag like this:

from __future__ import annotations

from airflow.models.baseoperator import BaseOperator
from airflow import DAG
from airflow.sdk import Variable

class CustomOperator(BaseOperator):
    def execute(self, context):
        v = Variable.get(key="key1")
        print("Got value", v)


with DAG("get-var-sdk", schedule=None, catchup=False) as dag:
    CustomOperator(task_id="set_var")

We shouldn't mask the value of the variable in logs unless the key is sensitive. As of now, we always mask it. This means that irrespective of the value, it will always be masked. This will block users from printing variable values in their tasks.

After changes:
image


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@amoghrajesh amoghrajesh requested review from ashb and kaxil as code owners May 19, 2025 14:54
@amoghrajesh amoghrajesh added this to the Airflow 3.0.2 milestone May 19, 2025
@amoghrajesh amoghrajesh added the backport-to-v3-0-test Mark PR with this label to backport to v3-0-test branch label May 20, 2025
@amoghrajesh amoghrajesh requested review from ashb, uranusjr and kaxil May 21, 2025 07:33
@amoghrajesh
Copy link
Contributor Author

@kaxil i just parameterised the test if something needs masking, wdyt?

@amoghrajesh amoghrajesh merged commit 4f94f22 into apache:main May 21, 2025
69 checks passed
@amoghrajesh amoghrajesh deleted the do-not-mask-variable-values branch May 21, 2025 19:03
github-actions bot pushed a commit that referenced this pull request May 21, 2025
…y is sensitive (#50775)

(cherry picked from commit 4f94f22)

Co-authored-by: Amogh Desai <amoghrajesh1999@gmail.com>
Copy link

Backport successfully created: v3-0-test

Status Branch Result
v3-0-test PR Link

github-actions bot pushed a commit to aws-mwaa/upstream-to-airflow that referenced this pull request May 21, 2025
…y is sensitive (apache#50775)

(cherry picked from commit 4f94f22)

Co-authored-by: Amogh Desai <amoghrajesh1999@gmail.com>
amoghrajesh added a commit that referenced this pull request May 22, 2025
…y is sensitive (#50775) (#50927)

(cherry picked from commit 4f94f22)

Co-authored-by: Amogh Desai <amoghrajesh1999@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:task-sdk backport-to-v3-0-test Mark PR with this label to backport to v3-0-test branch
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy