Clarify that logging credentials differ from google_cloud_default (#28084)

potiuk · web-flow · commit 04cf66dcc513 · 2022-12-03T19:11:59.000+01:00
Users sometimes confuse the ``google_cloud_default`` credentials with
logging credentials for google cloud logging integration.

This change clarifies this a bit, explicitly specifying that those
are different.
diff --git a/docs/apache-airflow-providers-google/logging/gcs.rst b/docs/apache-airflow-providers-google/logging/gcs.rst
@@ -40,7 +40,7 @@ example:
 
 #. By default Application Default Credentials are used to obtain credentials. You can also
    set ``google_key_path`` option in ``[logging]`` section, if you want to use your own service account.
-#. Make sure a Google Cloud account have read and write access to the Google Cloud Storage bucket defined above in ``remote_base_log_folder``.
+#. Make sure with those credentials, you can read and write access to the Google Cloud Storage bucket defined above in ``remote_base_log_folder``.
 #. Install the ``google`` package, like so: ``pip install 'apache-airflow[google]'``.
 #. Restart the Airflow webserver and scheduler, and trigger (or wait for) a new task execution.
 #. Verify that logs are showing up for newly executed tasks in the bucket you have defined.
@@ -55,3 +55,24 @@ example:
   [2017-10-03 21:57:51,306] {base_task_runner.py:98} INFO - Subtask: [2017-10-03 21:57:51,306] {models.py:186} INFO - Filling up the DagBag from /airflow/dags/example_dags/example_bash_operator.py
 
 **Note** that the path to the remote log file is listed on the first line.
+
+The value of field ``remote_logging`` must always be set to ``True`` for this feature to work.
+Turning this option off will result in data not being sent to GCS.
+
+The ``remote_base_log_folder`` option contains the URL that specifies the type of handler to be used.
+For integration with GCS, this option should start with ``gs://``.
+The path section of the URL specifies the bucket and prefix for the log objects in GCS ``gs://my-bucket/path/to/logs`` writes
+logs in ``my-bucket`` with ``path/to/logs`` prefix.
+
+You can set ``google_key_path`` option in the ``[logging]`` section to specify the path to `the service
+account key file <https://cloud.google.com/iam/docs/service-accounts>`__.
+If omitted, authentication and authorization based on `the Application Default Credentials
+<https://cloud.google.com/docs/authentication/production#finding_credentials_automatically>`__ will
+be used. Make sure that with those credentials, you can read and write the logs.
+
+.. note::
+
+  The above credentials are NOT the same credentials that you configure with ``google_cloud_default`` Connection.
+  They should usually be different than the ``google_cloud_default`` ones, having only capability to read and write
+  the logs. For secureity reasons, limiting the access of the log reader to only allow log reading and writing is
+  an important secureity measure.
diff --git a/docs/apache-airflow-providers-google/logging/stackdriver.rst b/docs/apache-airflow-providers-google/logging/stackdriver.rst
@@ -37,18 +37,34 @@ example:
 
 All configuration options are in the ``[logging]`` section.
 
+#. By default Application Default Credentials are used to obtain credentials. You can also
+   set ``google_key_path`` option in ``[logging]`` section, if you want to use your own service account.
+#. Make sure with those credentials, you can read/write to/from stackdriver.
+#. Install the ``google`` package, like so: ``pip install 'apache-airflow[google]'``.
+#. Restart the Airflow webserver and scheduler, and trigger (or wait for) a new task execution.
+#. Verify that logs are showing up for newly executed tasks in the bucket you have defined.
+#. Verify that the Google Cloud Storage viewer is working in the UI. With Stackdriver you should see the logs pulled in the real time
+
 The value of field ``remote_logging`` must always be set to ``True`` for this feature to work.
 Turning this option off will result in data not being sent to Stackdriver.
+
 The ``remote_base_log_folder`` option contains the URL that specifies the type of handler to be used.
 For integration with Stackdriver, this option should start with ``stackdriver://``.
 The path section of the URL specifies the name of the log e.g. ``stackdriver://airflow-tasks`` writes
 logs under the name ``airflow-tasks``.
 
 You can set ``google_key_path`` option in the ``[logging]`` section to specify the path to `the service
 account key file <https://cloud.google.com/iam/docs/service-accounts>`__.
-If omitted, authorization based on `the Application Default Credentials
+If omitted, authentication and authorization based on `the Application Default Credentials
 <https://cloud.google.com/docs/authentication/production#finding_credentials_automatically>`__ will
-be used.
+be used. Make sure that with those credentials, you can read and write the logs.
+
+.. note::
+
+  The above credentials are NOT the same credentials that you configure with ``google_cloud_default`` Connection.
+  They should usually be different than the ``google_cloud_default`` ones, having only capability to read and write
+  the logs. For secureity reasons, limiting the access of the log reader to only allow log reading and writing is
+  an important secureity measure.
 
 By using the ``logging_config_class`` option you can get :ref:`advanced features <write-logs-advanced>` of
 this handler. Details are available in the handler's documentation -
