In the case you are not aware. There are 2 additional locations where we perform network calls in TableDataWriteChannel.java and Job.java.

Yes, I wanted to keep this PR scoped to just the BigQueryImpl file (though with the refactor that has changed). I was planning to include the calls in TableDataWriteChannel.java and Job.java in a separate PR with a few other methods, such as runWithRetries to make it easier to review and reason about.

My biggest worry with this is the attribute structure here. This PR appears to define every field of every of every API as it's own attribute, which means we're more likely to introduce conflicts for users trying to filter based on attributes downstream.

This leads me to two questions:

1. should we be putting the full contents of the responses into attributes, vs a more constrained approach (critical fields like identifiers, status, etc)?
2. if we do want to capture the API response, should it just be stringified as a single attribute?

1. I'm of the opinion that since tracing is opt-in and meant to make debugging easier, the more visibility into the data the better. It's better to have visibility into the full contents and not need it than it is to want that visibility and not have it. What do you think?
2. I'm not convinced we should be capturing the response, as it essentially just passed to the user. The user can decide to capture or log the response once received outside of the library code. This design was to capture just the inputs to the API calls. That being said, I think stringifying a response as a single attribute makes the attribute harder to parse and read when debugging, which can defeat the purpose of creating a better debug system.

My 2 cents regarding attributes:
Recall that we have 2 layers for the span. The upper level interface (BigQueryImpl.java, Job.java, etc.) and the HttpBigQueryRpc level. I believe we should capture the request and response (not in this PR). Specifically, it should be captured as a request/response* attribute at the HttpBigQueryRpc level. For the interface level (this PR), we should capture any metadata information that would help with debugging any issue that may arise and we should capture it in a structured manner so it is visible. Specifically, attributes in the options are great as they are rarely captured elsewhere. Meanwhile, we can be a little more selective with datatype attributes (TableId vs all the fields in TableInfo). And if needed, then the request attribute can be used to deep dive.

On a side note, I think we should also clearly state that "Span names and attributes are subject to change without notice" to allow us the flexibility to change the structure as needed.

• For certain responses such as table row data, we probably won't want to capture those values.

I've cleaned up the attributes that we are collecting by removing some fields that are not very useful in debugging. I also added in attributes for language and "db.name" to make the tracing more consistent with python. I've updated the span naming pattern to be more descriptive.

I think it's a good idea to include "Span names and attributes are subject to change without notice", but that seems like it would better to put in the docs rather than the code. What do you guys think?

I've gone through and removed most attributes that we were collecting before, keeping mainly just the OP identifiers and time information such as creationTime, lastModifiedTime, etc. Let me know if you think we should shave off any more

I renamed all the attributes to be snake case and gave them namespace prefixes to remove any overloading of attributes. I went with the namespace "bq" and created further namespaces depending on the objects, such as "bq.dataset", "bq.table", "bq.query", etc.

Let me know if you think a different namespace style would be better.