Content-Length: 274209 | pFad | https://github.com/apache/airflow/issues/48859

6A ProvidersManager: "AttributeError: 'NoneType' object has no attribute 'connection_type'" @ FSHook · Issue #48859 · apache/airflow · GitHub
Skip to content

ProvidersManager: "AttributeError: 'NoneType' object has no attribute 'connection_type'" @ FSHook #48859

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
2 tasks done
Dev-iL opened this issue Apr 7, 2025 · 2 comments
Open
2 tasks done

ProvidersManager: "AttributeError: 'NoneType' object has no attribute 'connection_type'" @ FSHook #48859

Dev-iL opened this issue Apr 7, 2025 · 2 comments
Labels
affected_version:3.0 Issues Reported for 3.0 area:CI Airflow's tests and continious integration area:core kind:bug This is a clearly a bug priority:medium Bug that should be fixed before next release but would not block a release provider:common-sql

Comments

@Dev-iL
Copy link
Contributor

Dev-iL commented Apr 7, 2025 •
edited
Loading

Apache Airflow version

3.0.0

If "Other Airflow 2 version" selected, which one?

No response

What happened?

Running a DagBag unit test raises the following:

ERROR    airflow.models.dagbag.DagBag:dagbag.py:394 Failed to import: ....
    from airflow.providers.common.sql.operators.sql import SQLExecuteQueryOperator
  File ".../.venv300/lib64/python3.12/site-packages/airflow/providers/common/sql/operators/sql.py", line 30, in <module>
    from airflow.providers.common.sql.hooks.sql import DbApiHook
  File ".../.venv300/lib64/python3.12/site-packages/airflow/providers/common/sql/hooks/sql.py", line 131, in <module>
    class DbApiHook(BaseHook):
  File ".../.venv300/lib64/python3.12/site-packages/airflow/providers/common/sql/hooks/sql.py", line 162, in DbApiHook
    _dialects: MutableMapping[str, MutableMapping] = resolve_dialects()
                                                     ^^^^^^^^^^^^^^^^^^
  File ".../.venv300/lib64/python3.12/site-packages/airflow/providers/common/sql/hooks/sql.py", line 89, in resolve_dialects
    providers_manager = ProvidersManager()
                        ^^^^^^^^^^^^^^^^^^
  File ".../.venv300/lib64/python3.12/site-packages/airflow/utils/singleton.py", line 32, in __call__
    cls._instances[cls] = super().__call__(*args, **kwargs)

What you think should happen instead?

The import should succeed.

How to reproduce

  1. Create a pytest fixture test that populates a DagBag.
  2. Create a test that uses the fixture:
    def test_dag_loaded(dagbag):
        assert dagbag.import_errors.items() == {}
  1. Have a Dag definition containing:
from airflow.providers.common.sql.operators.sql import SQLExecuteQueryOperator

Note

  • This doesn't happen when importing SQLExecuteQueryOperator in isolation via the REPL.
  • This doesn't happen when importing and creating a DagBag via the REPL.

Operating System

Rocky Linux 9.3 (Blue Onyx)

Versions of Apache Airflow Providers

apache-airflow==3.0.0rc1.post2
apache-airflow-core==3.0.0rc1.post2
apache-airflow-providers-amazon==9.5.0rc3
apache-airflow-providers-common-compat==1.6.0
apache-airflow-providers-common-io==1.5.2
apache-airflow-providers-common-sql==1.25.0rc1
apache-airflow-providers-fab==2.0.0rc3
apache-airflow-providers-ftp==3.12.3
apache-airflow-providers-google==15.0.0rc1
apache-airflow-providers-http==5.2.1
apache-airflow-providers-openlineage==2.1.2rc2
apache-airflow-providers-postgres==6.1.2rc1
apache-airflow-providers-sftp==5.1.2rc1
apache-airflow-providers-slack==9.0.3
apache-airflow-providers-smtp==2.0.2
apache-airflow-providers-sqlite==4.0.1
apache-airflow-providers-ssh==4.0.1
apache-airflow-providers-standard==0.3.0rc1
apache-airflow-task-sdk==1.0.0rc1
google-cloud-orchestration-airflow==1.17.3

Deployment

Virtualenv installation

Deployment details

Airflow Installed via

uv pip install "apache-airflow[amazon,cloudpickle,ftp,google,postgres,sftp,smtp,slack,virtualenv]" apache-airflow-providers-standard==0.3.0rc1 --pre

Anything else?

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct
@Dev-iL Dev-iL added area:core kind:bug This is a clearly a bug needs-triage label for new issues that we didn't triage yet labels Apr 7, 2025
@eladkal eladkal added priority:high High priority bug that should be patched quickly but does not require immediate new release affected_version:3.0.0rc labels Apr 7, 2025
@Dev-iL
Copy link
Contributor Author

Dev-iL commented Apr 7, 2025

Upon trying to add an MCVE, it seems that this failure only appears when treating warning as errors in pytest, i.e.

[tool.pytest.ini_options]
filterwarnings = [
    "error",
]

Which causes a various DeprecationWarnings to be raised:

  • FAB: _request_ctx_stack, _app_ctx_stack
  • marshmallow_sqlalchemy: `The 'version' attribute is deprecated and will be removed in in a future version. Use feature detection or 'importlib.metadata.version("marshmallow")' instead.
  • openlineage: DeprecationWarning: This module is deprecated. Please use openlineage.client.facet_v2`

Also, there seems to be a ProvidersManager problem related to FSHook, where hook_info.connection_type is being set on a None object:

Image

@Dev-iL Dev-iL changed the title AttributeError: 'NoneType' object has no attribute 'connection_type' ProvidersManager: "AttributeError: 'NoneType' object has no attribute 'connection_type'" @ FSHook Apr 7, 2025
@eladkal eladkal removed priority:high High priority bug that should be patched quickly but does not require immediate new release area:providers needs-triage label for new issues that we didn't triage yet labels Apr 7, 2025
@ashb ashb added the area:CI Airflow's tests and continious integration label Apr 8, 2025
@vatsrahul1001 vatsrahul1001 added priority:medium Bug that should be fixed before next release but would not block a release affected_version:3.0 Issues Reported for 3.0 and removed affected_version:3.0.0rc labels Apr 17, 2025
@Dev-iL
Copy link
Contributor Author

Dev-iL commented May 21, 2025

If anyone's looking for a workaround, these are the warning filters I'm currently using with 3.0.1/py3.12:

# pyproject.toml

[tool.pytest.ini_options]
filterwarnings = [
    "error",
    "ignore::RuntimeWarning",
    "ignore:.*MapContainer uses PyType_Spec with a metaclass that has custom tp_new.*3.14.:DeprecationWarning",
    "ignore:Timer and timing metrics publish in seconds were deprecated.*:airflow.exceptions.RemovedInAirflow4Warning:airflow",
    "ignore::DeprecationWarning:flask_appbuilder",
    "ignore::DeprecationWarning:flask_sqlalchemy",
    "ignore::DeprecationWarning:marshmallow_sqlalchemy.convert",
]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
affected_version:3.0 Issues Reported for 3.0 area:CI Airflow's tests and continious integration area:core kind:bug This is a clearly a bug priority:medium Bug that should be fixed before next release but would not block a release provider:common-sql
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@ashb @Dev-iL @vatsrahul1001 @eladkal








ApplySandwichStrip

pFad - (p)hone/(F)rame/(a)nonymizer/(d)eclutterfier!      Saves Data!


--- a PPN by Garber Painting Akron. With Image Size Reduction included!

Fetched URL: https://github.com/apache/airflow/issues/48859

Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy