Apache Airflow Provider(s)

google

Versions of Apache Airflow Providers

apache-airflow-providers-google==5.0.0

Apache Airflow version

2.1.2

Operating System

GCP Container

Deployment

Composer

Deployment details

composer-1.17.1-airflow-2.1.2

What happened

I want to use serverless export to offload the export operation from the primary instance.
https://cloud.google.com/sql/docs/mysql/import-export#serverless

Used CloudSQLExportInstanceOperator with the exportContext.offload flag to perform a serverless export operation.

I got the following warning:

{field_validator.py:266} WARNING - The field 'exportContext.offload' is in the body, but is not specified in the validation specification '[{'name': 'fileType', 'allow_empty': False}, {'name': 'uri', 'allow_empty': False}, {'name': 'databases', 'optional': True, 'type': 'list'}, {'name': 'sqlExportOptions', 'type': 'dict', 'optional': True, 'fields': [{'name': 'tables', 'optional': True, 'type': 'list'}, {'name': 'schemaOnly', 'optional': True}]}, {'name': 'csvExportOptions', 'type': 'dict', 'optional': True, 'fields': [{'name': 'selectQuery'}]}]'. This might be because you are using newer API version and new field names defined for that version. Then the warning can be safely ignored, or you might want to upgrade the operatorto the version that supports the new API version.

What you think should happen instead

I think a validation specification for exportContext.offload should be added.

How to reproduce

Try to use exportContext.offload, as in the example below.

CloudSQLExportInstanceOperator(
    task_id='export_task',
    project_id='some_project',
    instance='cloud_sql_instance',
    body={
        "exportContext": {
            "fileType": "csv",
            "uri": "gs://my-bucket/export.csv",
            "databases": ["some_db"],
            "csvExportOptions": {"selectQuery": "select * from some_table limit 10"},
            "offload": True
        }
    },
)

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct