-
Notifications
You must be signed in to change notification settings - Fork 15.1k
remote logging for s3 fails while still looking for internal logs #50866
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval. |
Which executor are you using ? |
I'm using celery executor |
hm its strange, its working for me in local with celery executor. looks like some host issue. |
It was an issue with:
I solved with adding /execution/ for So for example, In this way, both scheduler and worker can connect to webserver api, and the remote logging succeeded as well:) |
Cool would you like anything or we can close this. |
Apache Airflow version
3.0.1
If "Other Airflow 2 version" selected, which one?
No response
What happened?
After upgrading to airflow 3.0.1, remote logging is not working as intended.
I have set airflow.cfg for remote logging as below:
remote_logging = True remote_log_conn_id = s3_conn delete_local_logs = True remote_base_log_folder = s3://my-logs remote_task_handler_kwargs = {"delete_local_copy": true}
But airflow ui fails to fetch logs for both cases, triggering with ui and cli. Also the tasks are marked as fail.
["Could not read served logs: Invalid URL 'http://:8793/log/dag_id=init_test_3.0.1/run_id=manual__2025-05-21T02:10:37.485869+00:00/task_id=print_time/attempt=1.log': No host supplied"]
However, when i run airflow dags test with cli, my tasks succeed, but still fails to read logs.
["Could not read served logs: HTTPConnectionPool(host='ip-172-29-XX-XX.ap-northeast-2.compute.internal', port=8793): Max retries exceeded with url: /log/dag_id=init_test_3.0.1/run_id=manual__2025-05-21T02:15:10.539747+00:00/task_id=print_time/attempt=1.log (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0xffff73126db0>: Failed to establish a new connection: [Errno 111] Connection refused'))"]
I can see the remote logging has not been configured properly, but I don't know why.
What you think should happen instead?
I expect my tasks my tasks to succeed and the logs are fetched remotely from s3.
How to reproduce
remote_logging = True remote_log_conn_id = s3_conn delete_local_logs = True remote_base_log_folder = s3://my-logs remote_task_handler_kwargs = {"delete_local_copy": true}
Operating System
Debian GNU/Linux
Versions of Apache Airflow Providers
No response
Deployment
Other Docker-based deployment
Deployment details
I am running airflow on ECS, using apache/airflow:3.0.1-python3.12 base image
Anything else?
No response
Are you willing to submit PR?
Code of Conduct
The text was updated successfully, but these errors were encountered: