Apache Airflow Provider(s)

amazon

Versions of Apache Airflow Providers

9.7.0

Apache Airflow version

3.0.0

Operating System

PRETTY_NAME="Debian GNU/Linux 12 (bookworm)"

Deployment

Astronomer

Deployment details

We use astronomer. this seemed to work ok with local dev but not when deployed to the cluster. Perhaps that's bc of different executor being used locally vs being deployed?

What happened

After upgrading to Airflow 3.0, the EksPodOperator started failing. It appears that its hook fails to fetch a token to auth to the K8s Server. This is bc the mechanism for fetching the token runs a script in a sub process. This script import the airflow hook. Bc it's in a separate process, it runs the airflow initialization step and fails initializing the database.

What you think should happen instead

Properly fetch a token and connect to EKS

How to reproduce

Use the EKSPodOperator to run an Airflow task.

Anything else

As a workaround I created a custom hook/operator that doesn't use this script, but instead uses the aws cli python library to fetch the token. This works but requires the added dependency.

Also, I know Airflow technically isn't supported on MacOs, but the grep command used to create the Kubeconfig file uses the -P flag which isn't supported on MacOs. This made troubleshooting difficult as that was a red hearing.. but again, macOS isn't technically supported.

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct