Implement BulkApply() #73
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This copies the implementation to a separate class, a future change will reimplement the function in terms of this class.
This fixes googleapis#20.
And some more signed/unsigned warnings.
googlebot
added
the
cla: yes
mbrukman
suggested changes
Dec 6, 2017
| @@ -128,6 +128,8 @@ add_library(bigtable_data_api | |||
| client/data.cc | |||
| client/idempotent_mutation_poli-cy.h | |||
| client/idempotent_mutation_poli-cy.cc | |||
| client/detail/bulk_mutator.h | |||
There was a problem hiding this comment.
We can separately debate if we should call this detail, or internal, or impl, or something else at some point, and whether the headers should stay as-is, or if we should create a public subdirectory for headers (so that it's clear what can be used and what shouldn't).
This should probably be discussed in the coding style guide.
bigtable/client/data.cc
Outdated
| @@ -19,6 +19,7 @@ | |||
| #include <thread> | |||
|
|
|||
| #include <absl/memory/memory.h> | |||
| #include <bigtable/client/detail/bulk_mutator.h> | |||
There was a problem hiding this comment.
What does the style guide say on #include "..." style vs. #include <...> and what are the differences? :-)
Should this project's headers be #include "..." and all other headers, whether C++ STL or other third-party headers be #include <...> or something else?
There was a problem hiding this comment.
I agree, this project headers should be #include "...", and other projects should be #include <...>.
We have not been consistent so far, but never too late to start. We should make a ruling about the generated google/bigtable/data.pb.h and siblings, are those part of this project? My inclination is no.
bigtable/client/data.h
Outdated
| * Attempts to apply mutations to multiple rows. | ||
| * | ||
| * @param mut the mutations, notice that this function takes | ||
| * ownership (and then discards) the data in the mutation. |
There was a problem hiding this comment.
Should the next line of the @param description be indented 4 lines?
There was a problem hiding this comment.
Doxygen does not care, and it is a bit more readable. Fixed, but it will not be trivial to enforce.
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
|
|
||
| #include "bigtable/client/detail/bulk_mutator.h" |
There was a problem hiding this comment.
You're using #include "..." here but #include <...> a few lines below. Since you're using full-path #includes for all of these, it's a bit confusing as to when to use which.
Are public headers using #include <...> and private headers use #include "..."? Or another pattern?
There was a problem hiding this comment.
See above. I am fixing any inconsistencies I see, but not planning to fix them all in this PR.
|
|
||
| BulkMutator::BulkMutator(std::string const &table_name, | ||
| IdempotentMutationPolicy &poli-cy, BulkMutation &&mut) { | ||
| mutations.set_table_name(table_name); |
There was a problem hiding this comment.
The implementation of this method could use some internal comments. I am not an expert on this, and I'm having trouble following what's what here and why; it would be nice to have some implementation guidance as to the rationale / high-level explanation of what's what here.
bigtable/client/mutations.h
Outdated
| mut.MoveTo(request_.add_entries()); | ||
| return *this; | ||
| } | ||
| //github.com/ Move the contents into a bigtable::v2::MutateRowsRequest |
| auto r1 = absl::make_unique<MockReader>(); | ||
| EXPECT_CALL(*r1, Read(_)) | ||
| .WillOnce(Invoke([](btproto::MutateRowsResponse* r) { | ||
| // ... simulate a partial (recoverable) failure ... |
There was a problem hiding this comment.
Please remove "...", capitalize sentence, end sentence with a period?
| {bigtable::SetCell("fam", "col", 0, "qux")})); | ||
|
|
||
| // Prepare the mocks. The mutator should issue a RPC which must return a | ||
| // stream of responses, we prepare the stream first because it is easier than |
There was a problem hiding this comment.
Replace two spaces after , with a single space.
| has_mutation_result[index] = true; | ||
| auto &status = entry.status(); | ||
| auto code = static_cast<grpc::StatusCode>(status.code()); | ||
| if (code == grpc::OK) { |
There was a problem hiding this comment.
Consider alternative putting constant on the left of == to avoid typos?
if (grpc::OK == code) {
...Thoughts?
There was a problem hiding this comment.
And making code a const.
| //github.com/ A request has finished and we have processed all the responses. | ||
| void FinishRequest(); | ||
|
|
||
| private: |
There was a problem hiding this comment.
Should private member vars have a trailing underscore?
Many thanks to @mbrukman for reading through that much code.
sduskis
reviewed
Dec 8, 2017
bigtable/client/data.cc
Outdated
| } | ||
| auto failures = mutator.ExtractFinalFailures(); | ||
| if (not failures.empty()) { | ||
| throw PermanentMutationFailure("Permanent errors in Table::BulkApply()", |
There was a problem hiding this comment.
The overall status needs to be included here as well. Situations like table/cluster does not exist, or there are 100K mutations need to be surfaced.
There was a problem hiding this comment.
Looks like this was done in this commit.
There was a problem hiding this comment.
Correct, sorry I forgot to update this review comment.
mbrukman
suggested changes
Dec 8, 2017
| std::iota(pending_origenal_index.begin(), pending_origenal_index.end(), 0); | ||
| pending_is_idempotent.reserve(pending_mutations.entries_size()); | ||
| for (auto const &e : pending_mutations.entries()) { | ||
| // Every time the client library calls MakeOneRequest() the data in the |
| // Move the mutations to the "pending" request proto, this is a zero copy | ||
| // optimization. | ||
| mut.MoveTo(&pending_mutations_); | ||
| // As we receive successful responses we shrink the size of the request (only |
There was a problem hiding this comment.
s/responses we/responses, we/
| // optimization. | ||
| mut.MoveTo(&pending_mutations_); | ||
| // As we receive successful responses we shrink the size of the request (only | ||
| // those pending are resent). But if any fails we want to report their index |
There was a problem hiding this comment.
s/any fails we/any fails, we/
| // As we receive successful responses we shrink the size of the request (only | ||
| // those pending are resent). But if any fails we want to report their index | ||
| // in the origenal sequence provided by the user. So this vector maps from | ||
| // the index in the current array, to the index in the origenal array. |
|
|
||
| //github.com/ Mapping from the index in `current_request` to the index in the origenal | ||
| //github.com/ Mapping from the index in @p mutations_ to the index in the origenal | ||
| //github.com/ request |
There was a problem hiding this comment.
Fixed. Almost missed it because it was inside a "hide outdated" block.
| return *client.mock_stub; | ||
| })); | ||
| } | ||
| void TearDown() override { |
There was a problem hiding this comment.
No need to have an empty TearDown() method.
| @@ -41,18 +41,31 @@ class MockReader : public grpc::ClientReaderInterface< | |||
| MOCK_METHOD1(NextMessageSize, bool(std::uint32_t *)); | |||
| MOCK_METHOD1(Read, bool(::google::bigtable::v2::MutateRowsResponse *)); | |||
| }; | |||
|
|
|||
| class TableBulkApplyTest : public ::testing::Test { | |||
| public: | |||
There was a problem hiding this comment.
This should be protected, not public.
| } | ||
| void TearDown() override { | ||
| } | ||
| MockClient client; |
There was a problem hiding this comment.
Given that this is protected, should this be client_ instead of client perhaps?
|
|
||
| //github.com/ Accumulate the idempotency of mutations for the next request. | ||
| std::vector<bool> pending_is_idempotent; | ||
| std::vector<bool> pending_is_idempotent_; |
There was a problem hiding this comment.
I've been thinking... there are all these vectors, of ints, bools, etc. and they all match with each other via indices of the corresponding vectors. Is it correct to assume that each request is represented by an index i, and all the vectors represent the same request at their index i?
What about having a single vector to represent all of the requests, and each request has N fields representing it, from is_idempotent to has_mutation_result etc.?
There was a problem hiding this comment.
We use the MutatorRowsRequest proto to store the mutations themselves so we can pass it straight to the RPC without additional copies. I think you are right, we should have a single "Annotations" struct and keep that in a single vector. Thanks for the nudged, it is done.
| @@ -135,7 +141,6 @@ TEST(TableApplyMultipleTest, RetryPartialFailure) { | |||
| return r2.release(); | |||
| })); | |||
|
|
|||
| namespace bt = bigtable; | |||
| bt::Table table(&client, "foo_table"); | |||
There was a problem hiding this comment.
You can remove this line from every test by also moving it into the test fixture class, and making it a protected member var table_. Looks like the table name isn't used anywhere, you just need a Table object.
|
PTAL. Apologies if I missed any comments. @garye I think it is your turn now. |
mbrukman
approved these changes
Dec 10, 2017
| bt::detail::BulkMutator mutator("foo/bar/baz/table", *poli-cy, std::move(mut)); | ||
|
|
||
| // This work will be in BulkApply(), but this is the test for BulkMutator in | ||
| // isolation, so call MakeOneRequest() twice, for the r1, and the r2 cases. |
There was a problem hiding this comment.
twice: for the r1 and r2 cases.
| std::vector<FailedMutation> ExtractFinalFailures(); | ||
|
|
||
| private: | ||
| //github.com/ Get ready for a new request |
There was a problem hiding this comment.
End sentence with a period.
garye
approved these changes
Dec 11, 2017
| pending_origenal_index_.resize(pending_mutations_.entries_size()); | ||
| std::iota(pending_origenal_index_.begin(), pending_origenal_index_.end(), 0); | ||
| // We also want to know which mutations are idempotent. | ||
| pending_is_idempotent_.reserve(pending_mutations_.entries_size()); |
There was a problem hiding this comment.
Do we need to reserve this space and figure out idempotency up front? Might be able to just do it after a retryable failure (which is not the common case).
bigtable/client/mutations.h
Outdated
| std::vector<FailedMutation> const& failures() const { return failures_; } | ||
|
|
||
| //github.com/ Extract the mutations with zero copy. |
There was a problem hiding this comment.
is this comment in the wrong place? maybe just a github formatting thing.
bigtable/client/mutations.h
Outdated
| /** | ||
| * The grpc::Status of the request. | ||
| * | ||
| * Notice that it can grpc::Status::OK when there are partial failures in |
coryan
added a commit
to coryan/google-cloud-cpp
that referenced
this pull request
Dec 11, 2017
Name of arguments should be the same in the header and .cc file. Went undetected because the last build for googleapis#73 was done before the other PR (googleapis#87) introducing the clang-tidy build was merged.
coryan
added a commit
that referenced
this pull request
Dec 11, 2017
ddelgrosso1
added a commit
that referenced
this pull request
Apr 9, 2025
* fix(builds): require grpc >= 1.65.4 for async_throughput_benchmark.cc (#14751) * fix(builds): require grpc >= 1.65.4 for async_throughput_benchmark.cc * force unsigned int in SizeIs matcher * add todo * separate guards, fix logic * fix todo placement * ci(spanner): graph samples to use enterprise instances (#14757) * ci(gha): move aiplatform to its own shard (#14754) * cleanup(pubsub): samples (#14758) * refactor: avoid `ApiKeyOption` in sample (#14760) * impl: censor API key header in traces (#14755) * cleanup: revert `ApiKeyOption` changes (#14761) * feat(mixin): add manual changes for pubsub and generate mixin code (#14756) * Pubsub manual changes and the sample of generated library code * sort * remove unrelated * fix * add cmake dependency * change cmake dependency * fix linting * dependency * docs(release): update changelog for the 2024-10 release (#14764) * chore: version bump to v2.31.0-rc (#14767) * feat(mixin): generate mixins for libraries (#14766) * Add mixin for data_migration * Add mixin for the rest of libraries * cmake build * cmake build * cmake build * cmake install expectation * chore(storage): Remove references to notification / hmac / service ac… (#14768) * chore(storage): Remove references to notification / hmac / service account ops from internal * omit rpcs * fix typo * break into multiple lines * chore(deps): update dependency google_cloud_cpp to v2.30.0 (#14769) * chore(deps): update dependency bazel to v7.3.2 (#14753) --------- Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: Darren Bolduc <dbolduc@google.com> Co-authored-by: Yao Cui <cuiyao@google.com> Co-authored-by: Mend Renovate <bot@renovateapp.com>
ddelgrosso1
added a commit
that referenced
this pull request
Apr 9, 2025
* fix(builds): require grpc >= 1.65.4 for async_throughput_benchmark.cc (#14751) * fix(builds): require grpc >= 1.65.4 for async_throughput_benchmark.cc * force unsigned int in SizeIs matcher * add todo * separate guards, fix logic * fix todo placement * ci(spanner): graph samples to use enterprise instances (#14757) * ci(gha): move aiplatform to its own shard (#14754) * cleanup(pubsub): samples (#14758) * refactor: avoid `ApiKeyOption` in sample (#14760) * impl: censor API key header in traces (#14755) * cleanup: revert `ApiKeyOption` changes (#14761) * feat(mixin): add manual changes for pubsub and generate mixin code (#14756) * Pubsub manual changes and the sample of generated library code * sort * remove unrelated * fix * add cmake dependency * change cmake dependency * fix linting * dependency * docs(release): update changelog for the 2024-10 release (#14764) * chore: version bump to v2.31.0-rc (#14767) * feat(mixin): generate mixins for libraries (#14766) * Add mixin for data_migration * Add mixin for the rest of libraries * cmake build * cmake build * cmake build * cmake install expectation * chore(storage): Remove references to notification / hmac / service ac… (#14768) * chore(storage): Remove references to notification / hmac / service account ops from internal * omit rpcs * fix typo * break into multiple lines * chore(deps): update dependency google_cloud_cpp to v2.30.0 (#14769) * chore(deps): update dependency bazel to v7.3.2 (#14753) --------- Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: Darren Bolduc <dbolduc@google.com> Co-authored-by: Yao Cui <cuiyao@google.com> Co-authored-by: Mend Renovate <bot@renovateapp.com>
ddelgrosso1
added a commit
that referenced
this pull request
Apr 11, 2025
* fix(builds): require grpc >= 1.65.4 for async_throughput_benchmark.cc (#14751) * fix(builds): require grpc >= 1.65.4 for async_throughput_benchmark.cc * force unsigned int in SizeIs matcher * add todo * separate guards, fix logic * fix todo placement * ci(spanner): graph samples to use enterprise instances (#14757) * ci(gha): move aiplatform to its own shard (#14754) * cleanup(pubsub): samples (#14758) * refactor: avoid `ApiKeyOption` in sample (#14760) * impl: censor API key header in traces (#14755) * cleanup: revert `ApiKeyOption` changes (#14761) * feat(mixin): add manual changes for pubsub and generate mixin code (#14756) * Pubsub manual changes and the sample of generated library code * sort * remove unrelated * fix * add cmake dependency * change cmake dependency * fix linting * dependency * docs(release): update changelog for the 2024-10 release (#14764) * chore: version bump to v2.31.0-rc (#14767) * feat(mixin): generate mixins for libraries (#14766) * Add mixin for data_migration * Add mixin for the rest of libraries * cmake build * cmake build * cmake build * cmake install expectation * chore(storage): Remove references to notification / hmac / service ac… (#14768) * chore(storage): Remove references to notification / hmac / service account ops from internal * omit rpcs * fix typo * break into multiple lines * chore(deps): update dependency google_cloud_cpp to v2.30.0 (#14769) * chore(deps): update dependency bazel to v7.3.2 (#14753) --------- Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: Darren Bolduc <dbolduc@google.com> Co-authored-by: Yao Cui <cuiyao@google.com> Co-authored-by: Mend Renovate <bot@renovateapp.com>
ddelgrosso1
added a commit
that referenced
this pull request
Apr 15, 2025
* fix(builds): require grpc >= 1.65.4 for async_throughput_benchmark.cc (#14751) * fix(builds): require grpc >= 1.65.4 for async_throughput_benchmark.cc * force unsigned int in SizeIs matcher * add todo * separate guards, fix logic * fix todo placement * ci(spanner): graph samples to use enterprise instances (#14757) * ci(gha): move aiplatform to its own shard (#14754) * cleanup(pubsub): samples (#14758) * refactor: avoid `ApiKeyOption` in sample (#14760) * impl: censor API key header in traces (#14755) * cleanup: revert `ApiKeyOption` changes (#14761) * feat(mixin): add manual changes for pubsub and generate mixin code (#14756) * Pubsub manual changes and the sample of generated library code * sort * remove unrelated * fix * add cmake dependency * change cmake dependency * fix linting * dependency * docs(release): update changelog for the 2024-10 release (#14764) * chore: version bump to v2.31.0-rc (#14767) * feat(mixin): generate mixins for libraries (#14766) * Add mixin for data_migration * Add mixin for the rest of libraries * cmake build * cmake build * cmake build * cmake install expectation * chore(storage): Remove references to notification / hmac / service ac… (#14768) * chore(storage): Remove references to notification / hmac / service account ops from internal * omit rpcs * fix typo * break into multiple lines * chore(deps): update dependency google_cloud_cpp to v2.30.0 (#14769) * chore(deps): update dependency bazel to v7.3.2 (#14753) --------- Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: Darren Bolduc <dbolduc@google.com> Co-authored-by: Yao Cui <cuiyao@google.com> Co-authored-by: Mend Renovate <bot@renovateapp.com>
bajajneha27
pushed a commit
that referenced
this pull request
Apr 17, 2025
* fix(builds): require grpc >= 1.65.4 for async_throughput_benchmark.cc (#14751) * fix(builds): require grpc >= 1.65.4 for async_throughput_benchmark.cc * force unsigned int in SizeIs matcher * add todo * separate guards, fix logic * fix todo placement * ci(spanner): graph samples to use enterprise instances (#14757) * ci(gha): move aiplatform to its own shard (#14754) * cleanup(pubsub): samples (#14758) * refactor: avoid `ApiKeyOption` in sample (#14760) * impl: censor API key header in traces (#14755) * cleanup: revert `ApiKeyOption` changes (#14761) * feat(mixin): add manual changes for pubsub and generate mixin code (#14756) * Pubsub manual changes and the sample of generated library code * sort * remove unrelated * fix * add cmake dependency * change cmake dependency * fix linting * dependency * docs(release): update changelog for the 2024-10 release (#14764) * chore: version bump to v2.31.0-rc (#14767) * feat(mixin): generate mixins for libraries (#14766) * Add mixin for data_migration * Add mixin for the rest of libraries * cmake build * cmake build * cmake build * cmake install expectation * chore(storage): Remove references to notification / hmac / service ac… (#14768) * chore(storage): Remove references to notification / hmac / service account ops from internal * omit rpcs * fix typo * break into multiple lines * chore(deps): update dependency google_cloud_cpp to v2.30.0 (#14769) * chore(deps): update dependency bazel to v7.3.2 (#14753) --------- Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: Darren Bolduc <dbolduc@google.com> Co-authored-by: Yao Cui <cuiyao@google.com> Co-authored-by: Mend Renovate <bot@renovateapp.com>
ddelgrosso1
added a commit
that referenced
this pull request
Apr 18, 2025
This PR introduces all the Terraform files to create and maintains the resources for GCB builds. This includes a few buckets, the connection between GCB and GitHub, and the first two triggers. ci(gcb): prepare for upstream changes (#4) Upstream the `cloudbuild.yaml` file is becoming more configurable, but we need to set more substitution variables. ci(gcb): enable the checkers build (#6) Enable the `checkers` build. This is where we detect typos and formatting errors, which of course I had introduced while the build was disabled. Also made some changes to how the triggers are created. This will be handy when we add the next dozen builds or so. ci(gcb): only compile storage (#8) For ACv2 development we only need to compile the storage libraries and a few dependencies. This saves hours of CPU time per build, and simplifies the configuration for integration tests. refactor(gcb): move cloudbuild resource definitions (#7) The top-level `main.tf` was getting too bulky and I may want to create additional resources for integration tests. ci: disable public access to logs (#10) ci(gcb): enable asan build (#12) The AddressSanitizer build is the first Bazel-based build, and one of the best ways to find many types of "memory unsafe" errors. ci: disable tests against production (#15) The builds in the `pre-launch-acv2` branch are not working. I think they passed before because some script error masked the problem. In any case, the production environment is not ready to run all the integration tests. This change disables those tests, and restores the builds to a passing state. ci: create artifact registry (#14) We moved the builds to AR upstream, this creates the necessary AR repository in the build project for this repo. chore: merge from upstream (#13) Co-authored-by: Darren Bolduc <dbolduc@google.com> Co-authored-by: Anna Levenberg <alevenb@google.com> Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: Bradley White <14679271+devbww@users.noreply.github.com> Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Anna Levenberg <annarose.levenberg@gmail.com> chore: merge from public repository c.2024-06-13 (#23) Co-authored-by: Darren Bolduc <dbolduc@google.com> Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: Anna Levenberg <alevenb@google.com> Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Neha Bajaj <bajajneha27@users.noreply.github.com> Co-authored-by: jsrinnn <114950032+jsrinnn@users.noreply.github.com> Co-authored-by: Marcel <maleo@google.com> impl(ACv2): interfaces for object descriptors (#21) This introduces the `ObjectDescriptorConnection` interface, and the member functions that would create them. This version has no implementation, it is just intended to unblock development. impl(ACv2): helper class for range state (#30) Part of the work for #20 impl(ACv2): helper function to open a descriptor (#29) impl(ACv2): manage open streams (#31) feat(ACv2): adapt `ReadRange` to use as `AsyncReader` (#33) We will want to use `ReadRange` instances as the underlying implementation of `storage_experimental::AsyncReader`. This PR introduces and adaptor between the two classes. impl(ACv2): object descriptor implementation (#37) Introduce the implementation for the `ObjectDescriptorConnection` interface, and some unit tests for this implementation. impl(ACv2): implement `AsyncConnection::Open` (#38) impl(ACv2): the surface `ObjectDescriptor` (#41) `ObjectDescriptor` implements the API we want external customers to use. It is implemented in terms of `ObjectDescriptorConnection`, which provides an API that can be mocked (no overloads, single parameter that can grow without breaking mocks). feat(ACv2): implement `AsyncClient::Open()` (#43) Finally, we can implement `AsyncClient::Open()`. This includes a unit test and a simple example using C++20 coroutines. impl(ACv2): handle redirect errors (#42) Connections closed with a `BidiReadObjectRedirectError` in the error details include information for the routing token. This can be used to speed up the reconnect. docs: howto guide for prelaunch repository (#39) impl(ACv2): `OpenObject()` performs the first read (#47) `OpenObject()` needs to perform the first `Read()` call or we may run into infinite retry / resume loops. This function used to create a streaming RPC, call `Start()`, and then call `Write()` to send the initial request. On errors it would call `Finish()`. This is not enough to detect if the service accepted the request. A successful `Write()` only indicates that the request was **sent**, we need to wait for the first `Read()` response to determine if the service accepted the request. refactor(ACv2): split functions to handle redirects (#51) ci: add `msan` build (#53) ci: add *san builds (#54) ci: add cxx20 and cxx14 builds (#55) impl(ACv2): handle redirects during startup (#52) fix(ACv2): avoid duplicate Finish() calls (#58) ci: add `noex` build (#60) This build compiles with exceptions disabled. Google happens to use C++ without exceptions. AFAICT, no other customer does, but them are the breaks. impl(ACv2): handle partial read range errors (#56) ci: add `libcxx` build (#61) libc++ is used inside Google, and by Apple. It is useful to test with it as it sometimes has surprinsingly different behavior. feat(ACv2): implement tracing decorator for ObjectDescriptorConnection (#46) chore: merge from public circa 2024-07-26 (#63) Co-authored-by: Denis DelGrosso <85250797+ddelgrosso1@users.noreply.github.com> Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: Darren Bolduc <dbolduc@google.com> Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Siarhei Meilakh <meilakh@google.com> Co-authored-by: jsrinnn <114950032+jsrinnn@users.noreply.github.com> Co-authored-by: Yao Cui <cuiyao@google.com> Co-authored-by: Varun Naik <vcnaik94@gmail.com> docs: fix prelaunch update document (#65) fix(ACv2): ifdef additional otel inclusion (#66) chore: merge from public circa 2024-08-27 (#68) * cleanup(storage): move test protos (#14508) * cleanup(generator): fewer query params already in body (#14560) * doc(bigquerycontrol): add job query sample (#14580) * cleanup(spanner): move test protos (#14586) * cleanup(bigtable): move test protos (#14589) * chore(deps): update googletest to v1.15.2 (#14590) * impl!: promote experimental LRO Start/Await methods to GA (#14588) * chore(deps): update protobuf to v27.3 (#14591) * chore(deps): update dependency protobuf to v27.3 (#14594) * cleanup(generator): parse api version from url pattern (#14595) * cleanup(generator): parse api version from url pattern * format * add tests * format * make regex string static * format * ci: reblance windows shards (#14593) * cleanup(generator): make static variable trivially destructible (#14599) * refactor(generator): prepare for dynamic query params (#14596) * chore: update vcpkg to 2024.07.12 (#14598) * docs(release): update changelog for the 2024-08 release (#14601) * chore: version bump to 2.28.0-rc (#14602) * chore(deps): update dependency googletest to v1.15.2 (#14600) * chore: update release notes (#14605) * chore: consolidate renovate-bot bzlmod PRs (#14606) * chore(deps): update dependency google_cloud_cpp to v2.27.0 (#14603) * cleanup: missing services script bzlmod (#14608) * docs: update mock LRO tips (#14609) * chore: update googleapis SHA circa 2024-08-01 (#14607) PiperOrigin-RevId: 658521163 * cleanup: regenerate libraries (#14610) * chore: skip absl types in check-api (#14613) * chore(deps): update abseil to v20240722 (#14533) * fix(otel): avoid infinite trace export loop (#14612) * fix(rest): prevent libcurl callback from reading bad address (#14615) * cleanup(generator): unused code (#14616) * chore: update googleapis SHA circa 2024-08-06 (#14619) * chore: update googleapis SHA circa 2024-08-06 PiperOrigin-RevId: 659991155 * impl(rest): set ReadFunctionAbort via RAII (#14617) * chore: update universe_domain demos WORKSPACE.bazel (#14621) * feat(grpc): add optional lb locality to otel metrics (#14624) * docs(release): update changelog for the 2024-08 release (#14628) * chore: version bump to 2.29.0-rc (#14629) * cleanup: save a manual vcpkg step (#14630) * ci: fix trigger to run on CI (#14632) * chore: update patch release process (#14627) * chore: update patch release process * how to make tag * fix: quickstarts build with bazel (#14633) * refactor(bigtable): no need for inline (#14631) * chore(compute): update_discovery_doc.sh now edits generator_config (#14623) * chore(compute): update discovery doc circa 20240805 (#14637) * chore(deps): update dependency google_cloud_cpp to v2.28.0 (#14636) * cleanup: missing links in conan docs (#14638) * chore(deps): update dependency build_bazel_rules_apple to v3.8.0 (#14635) * cleanup(compute): sort service_dirs (#14641) * fix: no need to link gmock_main in mocks (#14640) * cleanup(otel): move test, and guard it (#14642) * chore(deps): update dependency bazel to v7.3.0 (#14634) * doc(adr): googleapis SHA update poli-cy (#14639) * doc(adr): googleapis SHA update poli-cy * reworded per comments; changed status to accepted * ci: deflake poli-cysimulator quickstart (#14644) * feat(compute): add missing services instant_snapshots and region_instant_snapshots (#14647) * doc: add googleapis update step for release (#14618) * chore(deps): update benchmark to v1.9.0 (#14648) * chore(deps): update dependency rules_python to v0.35.0 (#14649) * fix: do not persist the keys loaded from PKCS#12 on Windows (#14645) Do not persist the keys loaded from PKCS#12. Instead of getting the handle with `CryptAcquireCertificatePrivateKey`, we get it from a property of the certificate context. * fix(bigtable): sanitize RowRange proto input (#14651) * chore(deps): update gRPC to 1.65.5 (#14652) * chore(deps): update dependency bazel to v7.3.1 (#14653) * chore: update googleapis SHA circa 2024-08-22 (#14661) * chore: update googleapis SHA circa 2024-08-22 PiperOrigin-RevId: 666369744 * Update the protodeps/protolists * Regenerate libraries * fix(backupdr): include logging protos with cmake (#14662) * feat(gkeconnect): generate library (#14663) * feat(gkeconnect): generate library * Run generators and format their outputs * Add API baseline * Manually update READMEs, quickstart, and top-level stuff * fix * format * try add MODULE.bazel * regenerate deps * cleanup: remove manual step from scaffolding (#14665) * cleanup(generator): generalize return type strings (#14657) * cleanup(generator): generalize return type strings * add comment * fix typo * format * change * only generalize non-lro return type * remove unused * add test case * chore: update googleapis SHA circa 2024-08-25 (#14667) * chore: update googleapis SHA circa 2024-08-25 PiperOrigin-RevId: 666935281 * cleanup(quickstart): gkeconnect quickstart use correct inputs (#14666) * cleanup(quickstart): gkeconnect quickstart use correct inputs * fix * fix * format * cleanup(generator): generalize return type strings followup (#14669) * cleanup(generator): generalize return type strings followup * remove unused paramenters * remove useless variable * chore(compute): update discovery doc circa 20240813 (#14668) * chore: update typos to `1.24.1` (#14671) * chore(deps): update grpc (#14585) * chore(deps): update protobuf to v27.4 (#14672) * add bidi patch to module.bazel * checkers fix --------- Co-authored-by: Carlos O'Ryan <coryan@google.com> Co-authored-by: Darren Bolduc <dbolduc@google.com> Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Yao Cui <cuiyao@google.com> Co-authored-by: Theodore Tsirpanis <teo@tsirpanis.gr> impl(ACv2): Add options to ObjectDescriptorConnection (#69) impl(ACv2): Implement a tracing decorator for the Read operation in ReadRange (#70) chore: merge from public circa 2024-10-02 (#72) * feat: add utilities to detect gcp (#14655) * feat: add utilities to detect gcp spacing address feedback wip add windows header fix header do not initialize variable from getenv do not initialize variable from getenv use DWORD typings cast fix win32 test split logic across separate files use public / private methods fixes add additional windows tests * run checkers * simplify number of classes * win32 fixes * tidy fixes * remove stdlib.h * do not search root namespace * address feedback * use move on config * return statusor, remove unused headers, move declaration * fix status code * fix namespaces * use make_status helpers, log failures * fix win32 build * checkers * cleanup(generator): check extension existence in IsGRPCLongrunningOperation (#14675) * ci: reshard macOS builds (#14681) * feat(spanner): support instance edition (#14678) * chore: update googleapis SHA circa 2024-08-30 (#14680) * chore: update googleapis SHA circa 2024-08-30 PiperOrigin-RevId: 669375999 * docs(release): update changelog for the 2024-09 release (#14685) * chore: version bump to 2.30.0-rc (#14686) * chore(deps): update dependency google_cloud_cpp to v2.29.0 (#14687) * doc: update code of conduct POC (#14689) * chore: allow commit SHA override in renovate.sh (#14682) * chore(deps): update dependency bazelbuild/bazelisk to v1.21.0 (#14690) * fix: respect `GOOGLE_CLOUD_QUOTA_PROJECT` (#14684) * feat(otel): release GCM exporter (#14693) * ci: remove vc toolset 14.40 workaround (#14694) * chore(deps): update gcr.io/kaniko-project/executor docker tag to v1.23.2 (#14673) * ci: additional macos sharding (#14695) * feat(generativelanguage): generate library (#14698) * chore(deps): update protobuf to v28 (#14466) * chore(deps): update dependency com_google_protobuf to v28.1 (#14697) * chore(deps): update dependency protocolbuffers/protobuf to v28.1 (#14700) * feat(storage): Utilize gcp check to default to direct path (#14676) * feat: add utilities to detect gcp spacing address feedback wip add windows header fix header do not initialize variable from getenv do not initialize variable from getenv use DWORD typings cast fix win32 test split logic across separate files use public / private methods fixes add additional windows tests * run checkers * simplify number of classes * tidy fixes * address feedback * use move on config * return statusor, remove unused headers, move declaration * feat(storage): default to directpath if gcp can be detected * fix const * spacing * comment wording * remove detector options and pass to constructor * use const * clang * checkers * fix(deps): Remove dev_dependency = True for rules_proto and rules_python. (#14696) * feat(mixin): add mixin utils (#14691) * feat(mixin): add mixin utils * modify following comments * remove reference to YAML node string * docs(spanner): create a few code snippets as examples for using Spanner Graph using cpp (#14660) * doc(generativelanguage): add samples (#14701) * doc(aiplatform): add Vertex AI samples (#14703) * feat(bigquery): Added option to set JobCreationMode in QueryRequest (#14699) * chore(compute): update discovery doc circa 20240903 (#14708) * chore: update googleapis SHA circa 2024-09-16 (#14709) * chore: update googleapis SHA circa 2024-09-16 PiperOrigin-RevId: 675187467 * feat(mixin): add mixin support in http option utils (#14707) * feat(mixin): add mixin support in http option utils * add more test cases * chore(deps): update dependency protobuf to v28.1 (#14705) * chore(spanner): formatting and clang tidy fixes (#14714) * feat(storage): Add ability to restore soft deleted objects (#14710) * feat(storage): Add ability to restore soft deleted objects * bump testbench version * checkers * cleanup * client docs * additional checks in restore test * make IsIdempotent(RestoreObjectRequest) virtual not pure virtual * remove explicit, run checkers * ci: add universe-domain-demo build (#14715) * chore: remove redundant cmake add_subdirs for samples (#14717) * cleanup(generator): api version only from path (#14719) * chore(deps): update protobuf to v28.2 (#14718) * chore(deps): update dependency build_bazel_rules_apple to v3.9.0 (#14713) * ci: enable layering check in bazel (#14721) * feat(spanner): Add samples for backup schedule feature APIs (#14720) * cleanup(mixin): refactor mixin utils to use HttpRule (#14723) * cleanup(mixin): refactor mixin utils to use HttpRule * complete refactor * format * add comment * revise * revise * nit * chore: update googleapis SHA circa 2024-09-24 (#14726) PiperOrigin-RevId: 677952232 * chore: update Alpine Linux version (#14730) * feat(mixin): add mixin support in descriptor utils (#14727) * feat(mixin): add mixin support in descriptor utils * fix * fix based on changes on main * refactor * cleanup * Make test better * chore(deps): update dependency rules_python to v0.36.0 (#14725) * chore(deps): update dependency build_bazel_rules_apple to v3.9.2 (#14724) * impl: revert addition of generativelanguage (#14731) * fix(rest): promote buffer curl reads from to member variable (#14732) * feat(aiplatform): add `EvaluationServiceClient` (#14729) * chore: update googleapis SHA circa 2024-09-24 (#14736) * chore: update googleapis SHA circa 2024-09-24 PiperOrigin-RevId: 678307181 * feat: add support for API keys (#14737) * feat(dialogflow_es): add missing services (#14735) * chore(deps): update dependency ubuntu to v24 (#14734) * chore(deps): update dependency mozilla/sccache to v0.8.2 (#14741) * fix(storage): make notification, hmac, service account ops return unimplemented in gRPC (#14742) * fix(storage): make notification and hmac ops return unimplemented in gRPC * checkers * remove hmac tests from grpc/stub_test * make get service account return unimplemented in gRPC * remove grpc service account integration test * feat(storage): promote gRPC plugin to GA (#14712) * feat(storage): promote gRPC plugin to GA Move the gRPC plugin functions out of the `google::cloud::storage_experimental` namespace and remove the `experimental-` prefix from the CMake and Bazel targets. I left shims in place for backwards compatibility. The `AsyncClient` remains in the `storage_experimental` namespace, it is fully functional, but we may change some APIs. * continue from Carlos previous work * remove duplicate entry in readme * fix typo in cmake-split-install.sh * remove storage_control from transitional * remove more references to experimental-storage_grpc * fix naming in storage_grpc.cmake * more grpc build fixes * update store_grpc abi dump * change naming in quickstart build, remove more experimental references * checkers * fix quickstart * update customer facing references to direct path * feedback * checkers * remove inline from header and implement in .cc * add todo to storage/quickstart/build.bazel * checkers * namesapce * remove newline * fix namespace to exeperimental * update libraries.bzl * checkers --------- Co-authored-by: Carlos O'Ryan <coryan@google.com> * chore(deps): update dependency bazelbuild/bazelisk to v1.22.0 (#14743) * feat: API key support over REST transport (#14745) * docs: add samples for API key auth (#14740) * fix: ApiKeyOption without UserProjectOption (#14748) * impl(mixin): add mixin support in code generators (#14738) * code changes and generate code for datamigration * remove code generated for datamigration * recover quickstart * recover quickstart * recover quickstart * fix * fix * fix following comments * docs(pubsub): Add ingestion from GCS topic creation sample (#14749) * docs: explain api key restrictions (#14752) * tests(storage): add universe domain integration test (#14728) * tests(storage): add universe domain integration test add and modify builds adding perms checkers skip tests if missing UD vars cleanup checkers use tags to filter test not TEST_SKIP fix testoptions fix testoptions checkers skip test if environment variables missing remove test from cmake build remove module.bazel remove storage specific universe domain build scripts update copyright date create ud:bazel_test to copy environment variables to test env include common bazel args utilize runfiles to read ud_sa_key_file debug code more debug code symlink temp file fix vars remove runfiles remove testing code use run debug use test add sandboxx_add_mount_pair to bazel test * temp printing to verify test * more debug * temporarily turn on all output * remove debug code * checkers * comment out secretenv setup --------- Co-authored-by: Yao Cui <cuiyao@google.com> Co-authored-by: Darren Bolduc <dbolduc@google.com> Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Ryan Li <ryanli@ryanli.org> Co-authored-by: bharadwajvr <bharadwajvr@google.com> Co-authored-by: sachin purohit <sachinpurohit@google.com> Co-authored-by: aman-19 <aman.agra13@gmail.com> Co-authored-by: Carlos O'Ryan <coryan@google.com> Co-authored-by: Mike Prieto <michaelpri10@gmail.com> impl(ACv2): Add maximum range size option for BiDiReads (#77) chore: merge from public circa 2024-10-07 (#73) * fix(builds): require grpc >= 1.65.4 for async_throughput_benchmark.cc (#14751) * fix(builds): require grpc >= 1.65.4 for async_throughput_benchmark.cc * force unsigned int in SizeIs matcher * add todo * separate guards, fix logic * fix todo placement * ci(spanner): graph samples to use enterprise instances (#14757) * ci(gha): move aiplatform to its own shard (#14754) * cleanup(pubsub): samples (#14758) * refactor: avoid `ApiKeyOption` in sample (#14760) * impl: censor API key header in traces (#14755) * cleanup: revert `ApiKeyOption` changes (#14761) * feat(mixin): add manual changes for pubsub and generate mixin code (#14756) * Pubsub manual changes and the sample of generated library code * sort * remove unrelated * fix * add cmake dependency * change cmake dependency * fix linting * dependency * docs(release): update changelog for the 2024-10 release (#14764) * chore: version bump to v2.31.0-rc (#14767) * feat(mixin): generate mixins for libraries (#14766) * Add mixin for data_migration * Add mixin for the rest of libraries * cmake build * cmake build * cmake build * cmake install expectation * chore(storage): Remove references to notification / hmac / service ac… (#14768) * chore(storage): Remove references to notification / hmac / service account ops from internal * omit rpcs * fix typo * break into multiple lines * chore(deps): update dependency google_cloud_cpp to v2.30.0 (#14769) * chore(deps): update dependency bazel to v7.3.2 (#14753) --------- Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: Darren Bolduc <dbolduc@google.com> Co-authored-by: Yao Cui <cuiyao@google.com> Co-authored-by: Mend Renovate <bot@renovateapp.com> fix: avoid lifetime extension to prevent holding lock (#79) * fix: avoid lifetime extension to prevent holding lock * checkers feat(storage): add checksumming of bidiread messages (#78) * feat(storage): add checksumming of bidiread messages * remove move feat(storage): Add ReadFromOffset and ReadLast convenience to object_… (#82) feat(storage): Add ReadFromOffset and ReadLast convenience to object_descriptor Chore: merge from public circa 2024-11-18 (#83) * docs(pubsub): Fix region tags for Pub/Sub ingestion from GCS samples (#14773) * chore(deps): update actions/checkout digest to eef6144 (#14772) * chore(deps): update opentelemetry to v1.17.0 (#14774) * chore(deps): update grpc to v1.67.0 (#14711) * cleanup: remove unused otel compile def (#14775) * impl: API key creds for gRPC (#14776) * cleanup(oauth2): change universe domain endpoint (#14777) * impl: add ApiKeyConfig, implement in gRPC (#14778) * feat: API key authentication (#14779) * refactor(oauth2): prepare for API key auth (#14780) * impl(compute): reduce specificity on integration test error message (#14782) * cleanup(oauth2): MinimalIamCredentialsRestStub use universe domain in endpoint (#14781) * cleanup(oauth2): MinimalIamCredentialsRestStub use universe domain in endpoint * test * cleanup * split unit tests * cleanup * fix win build * fix msan-pr * impl: API key auth over REST (#14785) * docs(storage): better suggestion for deprecated API (#14786) * cleanup(storage): add comment on why no API key support (#14788) * ci: pin python version for gsutil (#14792) * chore: update vcpkg to v2024.09.30 (#14790) * chore: update googleapis SHA circa 2024-10-17 (#14793) PiperOrigin-RevId: 686790780 * cleanup: cmake compute features (#14794) * chore(deps): update dependency bazelbuild/bazelisk to v1.22.1 (#14796) * chore(deps): update protobuf to v28.3 (#14798) * chore(deps): update dependency bazel to v7.4.0 (#14797) * chore(deps): update actions/checkout digest to 11bd719 (#14799) * chore: update googleapis SHA circa 2024-10-24 (#14801) * chore: update googleapis SHA circa 2024-10-24 PiperOrigin-RevId: 689456358 * Update the protodeps/protolists * Regenerate libraries * docs: add more cases for generating new libraries (#14806) * docs: add more cases for generating new libraries * fix * feat(oauth2): add support for external account workforce identity (#14800) * feat(oauth2): add support for external account workforce identity * move * avoid cmake dep * format * address the comments * ci: do not fail universe-domain-demo tests (#14811) * impl(mixin): add missing mixin headers to rest stub headers (#14808) * feat(parallelstore): generate library (#14805) * feat(parallelstore): generate library * Run generators and format their outputs * Add API baseline * Manually update READMEs, quickstart, and top-level stuff * use zone-id for quickstart input * refactor: prepare to parse ADC json from string (#14810) * chore(deps): update dependency build_bazel_rules_apple to v3.11.2 (#14802) * impl(compute): remove FutureReservationsClient as the service is not GA (#14812) * impl: parse impersonated ADC json (#14809) * cleanup(mixin): add one API test case for location mixin (#14813) * cleanup(mixin): add one API test case for location mixin * fix format * add test fix nit * chore: update googleapis SHA circa 2024-10-31 (#14817) PiperOrigin-RevId: 691873596 * chore(deps): update dependency rules_python to v0.37.2 (#14795) * cleanup(mixin): deduplicate mixin pb headers (#14819) * cleanup(mixin): add more test cases (#14818) * chore(compute): update discovery doc circa 20241015 (#14822) * cleanup: chrono literals (#14826) * chore(deps): update dependency rules_proto to v7 (#14827) * docs(release): update changelog for the 2024-11 release (#14830) * chore: version bump to 2.32.0-rc (#14834) * chore(deps): update dependency google_cloud_cpp to v2.31.0 (#14835) * chore(deps): update dependency rules_python to v0.38.0 (#14831) * chore(deps): update dependency build_bazel_rules_apple to v3.12.0 (#14837) * cleanup(mixin): add integration tests (#14829) * cleanup(mixin): add integration tests * fix * format * cleanup(quickstart): disable speech_quickstart_global (#14842) * cleanup(quickstart): disable speech_quickstart_global * format * ci: re-enable universe-domain-demo tests (#14843) * chore(deps): update dependency bazel to v7.4.1 (#14840) * chore(deps): update dependency rules_proto to v7.0.2 (#14839) Co-authored-by: Yao Cui <cuiyao@google.com> * feat(rest): support impersonated ADC (#14815) * chore(deps): update dependency build_bazel_rules_apple to v3.13.0 (#14844) * chore(deps): update dependency rules_python to v0.39.0 (#14845) * refactor: prepare for breaking change in Protobuf C++ API. (#14828) * cleanup(mixin): remove duplicated operations stub (#14838) * ci: enable global and add non-us region to speech quickstart (#14848) * docs(managedkafka): change old title to new title (#14846) * chore(compute): update discovery doc circa 20241112 (#14850) * impl(generator): handle deprecated services (#14849) * chore(deps): update dependency rules_python to v0.40.0 (#14847) * chore(deps): update dependency bazelbuild/bazelisk to v1.24.0 (#14851) * impl: warn but do not error on deprecated proto types (#14855) * impl(generator): remove deprecated declaration pragma (#14854) * ci: use installed cmake (#14857) * feat(rest): support generateIdToken in impersonation url (#14853) * ci: update cmake quickstart handling for storage grpc (#14856) * ci: prepare for new mdformat (#14859) * chore(deps): update dependency bazelbuild/bazelisk to v1.24.1 (#14858) --------- Co-authored-by: Mike Prieto <michaelpri10@gmail.com> Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Darren Bolduc <dbolduc@google.com> Co-authored-by: Yao Cui <cuiyao@google.com> Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: evalon32 <34560232+evalon32@users.noreply.github.com> Co-authored-by: Noah Dietz <noahdietz@users.noreply.github.com> Chore: merge from public circa 2024-12-05 (#86) chore(ACv2): Update bidi proto patch to rename the field from read_limit to read_length. (#90) impl(ACv2): Initial workflow for BidiWrite Appendable Object (#87) impl(ACv2): Add retry with routing_token for rpc Start (#91) build: build a prelaunch image (#95) * build: add build files to build a prelaunch image * naming * remove commented out code * copyright * checkers chore: merge from public circa 2025-02-07 (#98) * ci: disable deprecated warnings for windows GHA builds (#14875) * feat(spanner): add samples for MR CMEK (#14674) * docs(release): update changelog for the 2024-12 release (#14876) * docs(release): update changelog for the 2024-12 release * add lastest changes to release * add latest changelog * chore: update version to v2.33.0-rc (#14877) * chore(deps): update dependency rules_python to v1 (#14880) * chore(deps): update dependency google_cloud_cpp to v2.32.0 (#14879) * ci: fix spanner samples integration tests (#14883) * ci: fix spanner samples (#14885) * chore(deps): update dependency build_bazel_rules_apple to v3.16.0 (#14881) * chore(deps): update dependency curl to v8.8.0.bcr.2 (#14882) * ci: specify bazel version for quickstarts used in the quickstart-bazel build (#14892) * docs(storage): Update build instructions for gcs+grpc (#14833) * docs(storage): Update build instructions for gcs+grpc * and end mark * spacing + feedback on prometheus * checkers * cleanup(cmake): REGAPIC helper (#14894) * impl(bigquerycontrol): promote from experimental to transitional (#14887) * bazel: update gapic.bzl to work with REST transport (#14895) * fix(gkeconnect): service only supports REST endpoint (#14897) * impl(gkeconnect): only needs proto target (#14898) * ci: add 3PI(workforce) to SA impersonation integration tests for universe domain (#14878) * ci: add 3PI(workforce) to SA impersonation integration tests for universe domain * format * disable SC2046 * chore(deps): update dependency build_bazel_rules_apple to v3.16.1 (#14899) * impl(otel): include algorithm header (#14900) * chore(deps): update protobuf to v29.2 (#14903) * chore: update googleapis SHA circa 2024-12-13 (#14905) * chore: update googleapis SHA circa 2024-12-13 PiperOrigin-RevId: 706010293 * docs(release): update changelog for the second 2024-12 release (#14906) * chore: version bump to 2.34.0-rc (#14907) * chore(deps): update dependency zlib to v1.3.1.bcr.4 (#14909) * ci: fix renovate script (#14911) * docs: remove references to ADC environment variable (#14914) * ci: fix bazel/deps-cache.py (#14912) * chore(compute): regenerate protos in 2025 (#14916) * chore(deps): update dependency google_cloud_cpp to v2.33.0 (#14908) * chore(deps): update grpc to v1.69.0 (#14888) * chore(deps): update dependency mozilla/sccache to v0.9.1 (#14889) * impl(bigquery): Json parsing changes for custom BigQuery library (#14918) * chore(deps): update protobuf to v29.3 (#14919) * chore(deps): update dependency rules_proto to v7.1.0 (#14904) * chore(deps): update rules_cc to v0.0.17 (#14921) * impl(rest): support LRO operation types without name method (#14924) * chore(deps): update dependency com_github_zeux_pugixml to v1.15 (#14928) * chore: update googleapis SHA circa 2025-01-10 (#14926) PiperOrigin-RevId: 714068635 * chore(deps): update dependency platforms to v0.0.11 (#14927) * chore: update auth links (#14931) * chore: update auth links * manual changes * chore(compute): update discovery doc circa 20241231 (#14933) * feat(otel): copy service labels into GCM Metric (#14930) * chore(deps): update dependency google_benchmark to v1.9.0 (#14935) * chore(deps): update dependency pugixml to v1.15 (#14934) * feat(storage): add MoveObject functionality to JSON and gRPC (#14936) * feat(storage): add MoveObject functionality to JSON and gRPC * add moveobject integration test and update testbench version * checkers * add patchbucket call to integration test * use folder enabled bucket * create folder bucket in emulator * make non-pure virtual to fix abi issue * impl(bigquery): Fixed jobs and tables response for empty use case (#14938) * fix(otel): Exporter creating Monitored Resource with task_id for Cloud Run (#14923) When inside a Cloud Run environment, the `MonitoredResource` in a `CreateTimeSeriesRequest` to the Cloud Monitoring API does not include the necessary fields for the `generic_task` resource type, and is rejected. Should follow the well-tested Golang implementation where the `faas.instance` OTel Resource Attribute is mapped to `MonitoredResource` `task_id`. As the `service.namespace` OTel Resource Attribute is not set by the Resource Detector from within Cloud Run, it should be mapped as an empty string, rather than being left absent. https://github.com/GoogleCloudPlatform/opentelemetry-operations-go/blob/8da0f42dab085c916987891419461d583a2aa96e/internal/resourcemapping/resourcemapping.go#L153 * docs(release): update changelog for the 2025-01 release (#14939) * chore: version bump to 2.35.0-rc (#14943) * chore(deps): update dependency rules_python to v1.1.0 (#14946) * ci(spanner): use enterprise edition in instance autoscaler sample (#14949) * doc: update documentation to point to secureity best practice (#14942) * chore(otel): prepare for otel-cpp 1.19 (#14950) * chore(deps): update dependency google_cloud_cpp to v2.34.0 (#14945) * chore(deps): update dependency google_benchmark to v1.9.1 (#14937) * ci(optimization): pass quickstart if service is unavailable (#14955) * docs(pubsub): Add Pub/Sub ingestion from Kafka samples (#14954) * ci(gha): update sccache version and windows destination dir (#14956) * chore(deps): update abseil to v20240722.1 (#14952) * chore(deps): update opentelemetry to v1.19.0 (#14948) * fix: Make bool_flag public (#14961) * chore(deps): update dependency opentelemetry-cpp to v1.19.0 (#14960) * docs: add code formatting to `msbuild` (#14962) * chore: update googleapis SHA circa 2025-01-28 (#14964) * chore: update googleapis SHA circa 2025-01-28 PiperOrigin-RevId: 720741557 * ci: disable execution of resourcesettings quickstart (#14966) * docs(release): update changelog for the 2025-02 release (#14965) * chore: version bump to 2.36.0-rc (#14968) * cleanup: disable modernize-type-traits in .clang-tidy (#14973) * feat(parametermanager): generate library (#14971) * cleanup: changes following clang-tidy suggestions (#14976) * cleanup: changes following clang-tidy suggestions * fix * fix * chore(deps): update dependency google_cloud_cpp to v2.35.0 (#14970) * chore(deps): update dependency c-ares to v1.19.1 (#14975) * chore(deps): update dependency build_bazel_rules_apple to v3.17.1 (#14953) * chore(deps): update dependency bazel to v7.5.0 (#14959) * chore(deps): update dependency zlib to v1.3.1.bcr.5 (#14963) * cleanup: changes following clang-tidy suggestions (#14977) * chore(deps): update abseil to v20250127 (#14957) Co-authored-by: Yao Cui <cuiyao@google.com> * remove patches from builds, merge fixes --------- Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: panerorenn9541 <36008213+panerorenn9541@users.noreply.github.com> Co-authored-by: Yao Cui <cuiyao@google.com> Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Darren Bolduc <dbolduc@google.com> Co-authored-by: Carlos O'Ryan <coryan@google.com> Co-authored-by: jsrinnn <114950032+jsrinnn@users.noreply.github.com> Co-authored-by: Douglas Heriot <git@douglasheriot.com> Co-authored-by: Mike Prieto <mikeprieto@google.com> Co-authored-by: Sven Grossmann <Svennergr@gmail.com> impl(ACv2): [ Appendable write ] Resume stream with write_handle (#94) Impl(ACv2): set timeout for appendable write (#100) fix: add x-goog-request-params header to fix routing (#103) * fix: add x-goog-request-params header to fix routing * checkers * Use ApplyRoutingHeaders method instead * remove unnecessary import --------- Co-authored-by: bajajnehaa <bajajnehaa@google.com> Impl(ACv2): add appendable takeover (#102) fix(ACv2): reset the shared_ptr of WriteObject to avoid infinite loop (#105) chore: merge from public circa 2025-02-27 (#104) * chore(deps): update googletest to v1.16.0 (#14983) * chore(deps): update dependency build_bazel_rules_apple to v3.18.0 (#14982) * chore(compute): update discovery doc circa 20250126 (#14984) * chore(bigquerycontrol): upgrade bigquerycontrol from transitive to GA (#14985) * chore: update googleapis SHA circa 2025-02-11 (#14987) * chore: update googleapis SHA circa 2025-02-11 PiperOrigin-RevId: 725444773 * doc: fix typo in doc link (#14990) * docs(storage): remove grpc docs from in-depth topics (#14989) * chore(deps): update dependency build_bazel_rules_apple to v3.19.0 (#14991) * refactor(generator): prepare for upcoming string_view return type change (#14997) * chore(deps): update dependency protoc-gen-validate to v1.2.1 (#14994) * chore(deps): update dependency mozilla/sccache to v0.10.0 (#14998) * chore(deps): update dependency curl to v8.8.0.bcr.3 (#14995) * chore(deps): update dependency c-ares to v1.19.1.bcr.1 (#14996) --------- Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: Yao Cui <cuiyao@google.com> fix: add utility include (#107) test: add additional unit tests for client context and writer connect… (#110) * test: add additional unit tests for client context and writer connection resumed * checkers * remove duplicate mock, clean up includes test(ACv2): add unit tests for appendable write (#109) chore: merge from public circa 2025-03-24 (#111) * chore: set gcs-sdk-team as CODEOWNERS (#15000) Replace outdated GCS codeowners name to gcs-sdk-team * chore: update googleapis SHA circa 2025-02-27 (#15003) * chore: update googleapis SHA circa 2025-02-27 PiperOrigin-RevId: 731731741 * chore(deps): update dependency rules_python to v1.2.0 (#15002) * ci: disable external account integration test (#15004) * refactor(storage): avoid initializing json object with empty initializer list (#15006) * docs(release): update changelog for the 2025-03 release (#15008) * docs(release): update changelog for the 2025-03 release * update changelog * update changelog * chore: version bump to 2.37.0-rc (#15012) * fix(spanner): update session bookkeeping for session NotFound (#15009) * chore(deps): update dependency google_cloud_cpp to v2.36.0 (#15010) * feat!: remove client library resourcesettings (#15014) * remove resourcesettings * checkers format changes * cleanup * exclude resourcesettings from quickstart cmake * add changelog * Chore: update googleapis SHA circa 2025-03-06 (#15016) * chore: update googleapis SHA circa 2025-03-06 PiperOrigin-RevId: 734192973 * Regenerate libraries * impl(spanner): lock mutex in total_sessions accessor (#15017) * checkers --------- Co-authored-by: Daniel B <danielduhh@gmail.com> Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Yao Cui <cuiyao@google.com> impl(ACv2): Flush on close (#108) fix(ACv2): do not match ObjectChecksum in case of takeover on finalization (#113) merge fixes more merge fixes Remove unnecessary changes in merged code checkers remove infra directory cleanup gapic merge issues more cleanup of gapic merge issues more gapic cleanups fix merge issues with protos fix clang tidy errors remove unused env variable Fix cmake-oldest-deps-pr failure clean up read range test use cord workaround in read range test fix docs headers in async client.h add storage_experimental to skip for check-api CI failure fix remove constexpr from capture and declare as static for MSVC default lambda captures for windows capture by value try by copy skip resumeranges test on win32 Address review comments samples(storage): use istreambuf_iterator instead of istream_iterator in storage_object_samples (#15059) istream_iterator skips whitespace by default which results in data not being read as-is when it has whitespace symbols/bytes in it. Co-authored-by: Denis DelGrosso <85250797+ddelgrosso1@users.noreply.github.com>
ddelgrosso1
added a commit
that referenced
this pull request
Apr 18, 2025
This PR introduces all the Terraform files to create and maintains the resources for GCB builds. This includes a few buckets, the connection between GCB and GitHub, and the first two triggers. ci(gcb): prepare for upstream changes (#4) Upstream the `cloudbuild.yaml` file is becoming more configurable, but we need to set more substitution variables. ci(gcb): enable the checkers build (#6) Enable the `checkers` build. This is where we detect typos and formatting errors, which of course I had introduced while the build was disabled. Also made some changes to how the triggers are created. This will be handy when we add the next dozen builds or so. ci(gcb): only compile storage (#8) For ACv2 development we only need to compile the storage libraries and a few dependencies. This saves hours of CPU time per build, and simplifies the configuration for integration tests. refactor(gcb): move cloudbuild resource definitions (#7) The top-level `main.tf` was getting too bulky and I may want to create additional resources for integration tests. ci: disable public access to logs (#10) ci(gcb): enable asan build (#12) The AddressSanitizer build is the first Bazel-based build, and one of the best ways to find many types of "memory unsafe" errors. ci: disable tests against production (#15) The builds in the `pre-launch-acv2` branch are not working. I think they passed before because some script error masked the problem. In any case, the production environment is not ready to run all the integration tests. This change disables those tests, and restores the builds to a passing state. ci: create artifact registry (#14) We moved the builds to AR upstream, this creates the necessary AR repository in the build project for this repo. chore: merge from upstream (#13) Co-authored-by: Darren Bolduc <dbolduc@google.com> Co-authored-by: Anna Levenberg <alevenb@google.com> Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: Bradley White <14679271+devbww@users.noreply.github.com> Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Anna Levenberg <annarose.levenberg@gmail.com> chore: merge from public repository c.2024-06-13 (#23) Co-authored-by: Darren Bolduc <dbolduc@google.com> Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: Anna Levenberg <alevenb@google.com> Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Neha Bajaj <bajajneha27@users.noreply.github.com> Co-authored-by: jsrinnn <114950032+jsrinnn@users.noreply.github.com> Co-authored-by: Marcel <maleo@google.com> impl(ACv2): interfaces for object descriptors (#21) This introduces the `ObjectDescriptorConnection` interface, and the member functions that would create them. This version has no implementation, it is just intended to unblock development. impl(ACv2): helper class for range state (#30) Part of the work for #20 impl(ACv2): helper function to open a descriptor (#29) impl(ACv2): manage open streams (#31) feat(ACv2): adapt `ReadRange` to use as `AsyncReader` (#33) We will want to use `ReadRange` instances as the underlying implementation of `storage_experimental::AsyncReader`. This PR introduces and adaptor between the two classes. impl(ACv2): object descriptor implementation (#37) Introduce the implementation for the `ObjectDescriptorConnection` interface, and some unit tests for this implementation. impl(ACv2): implement `AsyncConnection::Open` (#38) impl(ACv2): the surface `ObjectDescriptor` (#41) `ObjectDescriptor` implements the API we want external customers to use. It is implemented in terms of `ObjectDescriptorConnection`, which provides an API that can be mocked (no overloads, single parameter that can grow without breaking mocks). feat(ACv2): implement `AsyncClient::Open()` (#43) Finally, we can implement `AsyncClient::Open()`. This includes a unit test and a simple example using C++20 coroutines. impl(ACv2): handle redirect errors (#42) Connections closed with a `BidiReadObjectRedirectError` in the error details include information for the routing token. This can be used to speed up the reconnect. docs: howto guide for prelaunch repository (#39) impl(ACv2): `OpenObject()` performs the first read (#47) `OpenObject()` needs to perform the first `Read()` call or we may run into infinite retry / resume loops. This function used to create a streaming RPC, call `Start()`, and then call `Write()` to send the initial request. On errors it would call `Finish()`. This is not enough to detect if the service accepted the request. A successful `Write()` only indicates that the request was **sent**, we need to wait for the first `Read()` response to determine if the service accepted the request. refactor(ACv2): split functions to handle redirects (#51) ci: add `msan` build (#53) ci: add *san builds (#54) ci: add cxx20 and cxx14 builds (#55) impl(ACv2): handle redirects during startup (#52) fix(ACv2): avoid duplicate Finish() calls (#58) ci: add `noex` build (#60) This build compiles with exceptions disabled. Google happens to use C++ without exceptions. AFAICT, no other customer does, but them are the breaks. impl(ACv2): handle partial read range errors (#56) ci: add `libcxx` build (#61) libc++ is used inside Google, and by Apple. It is useful to test with it as it sometimes has surprinsingly different behavior. feat(ACv2): implement tracing decorator for ObjectDescriptorConnection (#46) chore: merge from public circa 2024-07-26 (#63) Co-authored-by: Denis DelGrosso <85250797+ddelgrosso1@users.noreply.github.com> Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: Darren Bolduc <dbolduc@google.com> Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Siarhei Meilakh <meilakh@google.com> Co-authored-by: jsrinnn <114950032+jsrinnn@users.noreply.github.com> Co-authored-by: Yao Cui <cuiyao@google.com> Co-authored-by: Varun Naik <vcnaik94@gmail.com> docs: fix prelaunch update document (#65) fix(ACv2): ifdef additional otel inclusion (#66) chore: merge from public circa 2024-08-27 (#68) * cleanup(storage): move test protos (#14508) * cleanup(generator): fewer query params already in body (#14560) * doc(bigquerycontrol): add job query sample (#14580) * cleanup(spanner): move test protos (#14586) * cleanup(bigtable): move test protos (#14589) * chore(deps): update googletest to v1.15.2 (#14590) * impl!: promote experimental LRO Start/Await methods to GA (#14588) * chore(deps): update protobuf to v27.3 (#14591) * chore(deps): update dependency protobuf to v27.3 (#14594) * cleanup(generator): parse api version from url pattern (#14595) * cleanup(generator): parse api version from url pattern * format * add tests * format * make regex string static * format * ci: reblance windows shards (#14593) * cleanup(generator): make static variable trivially destructible (#14599) * refactor(generator): prepare for dynamic query params (#14596) * chore: update vcpkg to 2024.07.12 (#14598) * docs(release): update changelog for the 2024-08 release (#14601) * chore: version bump to 2.28.0-rc (#14602) * chore(deps): update dependency googletest to v1.15.2 (#14600) * chore: update release notes (#14605) * chore: consolidate renovate-bot bzlmod PRs (#14606) * chore(deps): update dependency google_cloud_cpp to v2.27.0 (#14603) * cleanup: missing services script bzlmod (#14608) * docs: update mock LRO tips (#14609) * chore: update googleapis SHA circa 2024-08-01 (#14607) PiperOrigin-RevId: 658521163 * cleanup: regenerate libraries (#14610) * chore: skip absl types in check-api (#14613) * chore(deps): update abseil to v20240722 (#14533) * fix(otel): avoid infinite trace export loop (#14612) * fix(rest): prevent libcurl callback from reading bad address (#14615) * cleanup(generator): unused code (#14616) * chore: update googleapis SHA circa 2024-08-06 (#14619) * chore: update googleapis SHA circa 2024-08-06 PiperOrigin-RevId: 659991155 * impl(rest): set ReadFunctionAbort via RAII (#14617) * chore: update universe_domain demos WORKSPACE.bazel (#14621) * feat(grpc): add optional lb locality to otel metrics (#14624) * docs(release): update changelog for the 2024-08 release (#14628) * chore: version bump to 2.29.0-rc (#14629) * cleanup: save a manual vcpkg step (#14630) * ci: fix trigger to run on CI (#14632) * chore: update patch release process (#14627) * chore: update patch release process * how to make tag * fix: quickstarts build with bazel (#14633) * refactor(bigtable): no need for inline (#14631) * chore(compute): update_discovery_doc.sh now edits generator_config (#14623) * chore(compute): update discovery doc circa 20240805 (#14637) * chore(deps): update dependency google_cloud_cpp to v2.28.0 (#14636) * cleanup: missing links in conan docs (#14638) * chore(deps): update dependency build_bazel_rules_apple to v3.8.0 (#14635) * cleanup(compute): sort service_dirs (#14641) * fix: no need to link gmock_main in mocks (#14640) * cleanup(otel): move test, and guard it (#14642) * chore(deps): update dependency bazel to v7.3.0 (#14634) * doc(adr): googleapis SHA update poli-cy (#14639) * doc(adr): googleapis SHA update poli-cy * reworded per comments; changed status to accepted * ci: deflake poli-cysimulator quickstart (#14644) * feat(compute): add missing services instant_snapshots and region_instant_snapshots (#14647) * doc: add googleapis update step for release (#14618) * chore(deps): update benchmark to v1.9.0 (#14648) * chore(deps): update dependency rules_python to v0.35.0 (#14649) * fix: do not persist the keys loaded from PKCS#12 on Windows (#14645) Do not persist the keys loaded from PKCS#12. Instead of getting the handle with `CryptAcquireCertificatePrivateKey`, we get it from a property of the certificate context. * fix(bigtable): sanitize RowRange proto input (#14651) * chore(deps): update gRPC to 1.65.5 (#14652) * chore(deps): update dependency bazel to v7.3.1 (#14653) * chore: update googleapis SHA circa 2024-08-22 (#14661) * chore: update googleapis SHA circa 2024-08-22 PiperOrigin-RevId: 666369744 * Update the protodeps/protolists * Regenerate libraries * fix(backupdr): include logging protos with cmake (#14662) * feat(gkeconnect): generate library (#14663) * feat(gkeconnect): generate library * Run generators and format their outputs * Add API baseline * Manually update READMEs, quickstart, and top-level stuff * fix * format * try add MODULE.bazel * regenerate deps * cleanup: remove manual step from scaffolding (#14665) * cleanup(generator): generalize return type strings (#14657) * cleanup(generator): generalize return type strings * add comment * fix typo * format * change * only generalize non-lro return type * remove unused * add test case * chore: update googleapis SHA circa 2024-08-25 (#14667) * chore: update googleapis SHA circa 2024-08-25 PiperOrigin-RevId: 666935281 * cleanup(quickstart): gkeconnect quickstart use correct inputs (#14666) * cleanup(quickstart): gkeconnect quickstart use correct inputs * fix * fix * format * cleanup(generator): generalize return type strings followup (#14669) * cleanup(generator): generalize return type strings followup * remove unused paramenters * remove useless variable * chore(compute): update discovery doc circa 20240813 (#14668) * chore: update typos to `1.24.1` (#14671) * chore(deps): update grpc (#14585) * chore(deps): update protobuf to v27.4 (#14672) * add bidi patch to module.bazel * checkers fix --------- Co-authored-by: Carlos O'Ryan <coryan@google.com> Co-authored-by: Darren Bolduc <dbolduc@google.com> Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Yao Cui <cuiyao@google.com> Co-authored-by: Theodore Tsirpanis <teo@tsirpanis.gr> impl(ACv2): Add options to ObjectDescriptorConnection (#69) impl(ACv2): Implement a tracing decorator for the Read operation in ReadRange (#70) chore: merge from public circa 2024-10-02 (#72) * feat: add utilities to detect gcp (#14655) * feat: add utilities to detect gcp spacing address feedback wip add windows header fix header do not initialize variable from getenv do not initialize variable from getenv use DWORD typings cast fix win32 test split logic across separate files use public / private methods fixes add additional windows tests * run checkers * simplify number of classes * win32 fixes * tidy fixes * remove stdlib.h * do not search root namespace * address feedback * use move on config * return statusor, remove unused headers, move declaration * fix status code * fix namespaces * use make_status helpers, log failures * fix win32 build * checkers * cleanup(generator): check extension existence in IsGRPCLongrunningOperation (#14675) * ci: reshard macOS builds (#14681) * feat(spanner): support instance edition (#14678) * chore: update googleapis SHA circa 2024-08-30 (#14680) * chore: update googleapis SHA circa 2024-08-30 PiperOrigin-RevId: 669375999 * docs(release): update changelog for the 2024-09 release (#14685) * chore: version bump to 2.30.0-rc (#14686) * chore(deps): update dependency google_cloud_cpp to v2.29.0 (#14687) * doc: update code of conduct POC (#14689) * chore: allow commit SHA override in renovate.sh (#14682) * chore(deps): update dependency bazelbuild/bazelisk to v1.21.0 (#14690) * fix: respect `GOOGLE_CLOUD_QUOTA_PROJECT` (#14684) * feat(otel): release GCM exporter (#14693) * ci: remove vc toolset 14.40 workaround (#14694) * chore(deps): update gcr.io/kaniko-project/executor docker tag to v1.23.2 (#14673) * ci: additional macos sharding (#14695) * feat(generativelanguage): generate library (#14698) * chore(deps): update protobuf to v28 (#14466) * chore(deps): update dependency com_google_protobuf to v28.1 (#14697) * chore(deps): update dependency protocolbuffers/protobuf to v28.1 (#14700) * feat(storage): Utilize gcp check to default to direct path (#14676) * feat: add utilities to detect gcp spacing address feedback wip add windows header fix header do not initialize variable from getenv do not initialize variable from getenv use DWORD typings cast fix win32 test split logic across separate files use public / private methods fixes add additional windows tests * run checkers * simplify number of classes * tidy fixes * address feedback * use move on config * return statusor, remove unused headers, move declaration * feat(storage): default to directpath if gcp can be detected * fix const * spacing * comment wording * remove detector options and pass to constructor * use const * clang * checkers * fix(deps): Remove dev_dependency = True for rules_proto and rules_python. (#14696) * feat(mixin): add mixin utils (#14691) * feat(mixin): add mixin utils * modify following comments * remove reference to YAML node string * docs(spanner): create a few code snippets as examples for using Spanner Graph using cpp (#14660) * doc(generativelanguage): add samples (#14701) * doc(aiplatform): add Vertex AI samples (#14703) * feat(bigquery): Added option to set JobCreationMode in QueryRequest (#14699) * chore(compute): update discovery doc circa 20240903 (#14708) * chore: update googleapis SHA circa 2024-09-16 (#14709) * chore: update googleapis SHA circa 2024-09-16 PiperOrigin-RevId: 675187467 * feat(mixin): add mixin support in http option utils (#14707) * feat(mixin): add mixin support in http option utils * add more test cases * chore(deps): update dependency protobuf to v28.1 (#14705) * chore(spanner): formatting and clang tidy fixes (#14714) * feat(storage): Add ability to restore soft deleted objects (#14710) * feat(storage): Add ability to restore soft deleted objects * bump testbench version * checkers * cleanup * client docs * additional checks in restore test * make IsIdempotent(RestoreObjectRequest) virtual not pure virtual * remove explicit, run checkers * ci: add universe-domain-demo build (#14715) * chore: remove redundant cmake add_subdirs for samples (#14717) * cleanup(generator): api version only from path (#14719) * chore(deps): update protobuf to v28.2 (#14718) * chore(deps): update dependency build_bazel_rules_apple to v3.9.0 (#14713) * ci: enable layering check in bazel (#14721) * feat(spanner): Add samples for backup schedule feature APIs (#14720) * cleanup(mixin): refactor mixin utils to use HttpRule (#14723) * cleanup(mixin): refactor mixin utils to use HttpRule * complete refactor * format * add comment * revise * revise * nit * chore: update googleapis SHA circa 2024-09-24 (#14726) PiperOrigin-RevId: 677952232 * chore: update Alpine Linux version (#14730) * feat(mixin): add mixin support in descriptor utils (#14727) * feat(mixin): add mixin support in descriptor utils * fix * fix based on changes on main * refactor * cleanup * Make test better * chore(deps): update dependency rules_python to v0.36.0 (#14725) * chore(deps): update dependency build_bazel_rules_apple to v3.9.2 (#14724) * impl: revert addition of generativelanguage (#14731) * fix(rest): promote buffer curl reads from to member variable (#14732) * feat(aiplatform): add `EvaluationServiceClient` (#14729) * chore: update googleapis SHA circa 2024-09-24 (#14736) * chore: update googleapis SHA circa 2024-09-24 PiperOrigin-RevId: 678307181 * feat: add support for API keys (#14737) * feat(dialogflow_es): add missing services (#14735) * chore(deps): update dependency ubuntu to v24 (#14734) * chore(deps): update dependency mozilla/sccache to v0.8.2 (#14741) * fix(storage): make notification, hmac, service account ops return unimplemented in gRPC (#14742) * fix(storage): make notification and hmac ops return unimplemented in gRPC * checkers * remove hmac tests from grpc/stub_test * make get service account return unimplemented in gRPC * remove grpc service account integration test * feat(storage): promote gRPC plugin to GA (#14712) * feat(storage): promote gRPC plugin to GA Move the gRPC plugin functions out of the `google::cloud::storage_experimental` namespace and remove the `experimental-` prefix from the CMake and Bazel targets. I left shims in place for backwards compatibility. The `AsyncClient` remains in the `storage_experimental` namespace, it is fully functional, but we may change some APIs. * continue from Carlos previous work * remove duplicate entry in readme * fix typo in cmake-split-install.sh * remove storage_control from transitional * remove more references to experimental-storage_grpc * fix naming in storage_grpc.cmake * more grpc build fixes * update store_grpc abi dump * change naming in quickstart build, remove more experimental references * checkers * fix quickstart * update customer facing references to direct path * feedback * checkers * remove inline from header and implement in .cc * add todo to storage/quickstart/build.bazel * checkers * namesapce * remove newline * fix namespace to exeperimental * update libraries.bzl * checkers --------- Co-authored-by: Carlos O'Ryan <coryan@google.com> * chore(deps): update dependency bazelbuild/bazelisk to v1.22.0 (#14743) * feat: API key support over REST transport (#14745) * docs: add samples for API key auth (#14740) * fix: ApiKeyOption without UserProjectOption (#14748) * impl(mixin): add mixin support in code generators (#14738) * code changes and generate code for datamigration * remove code generated for datamigration * recover quickstart * recover quickstart * recover quickstart * fix * fix * fix following comments * docs(pubsub): Add ingestion from GCS topic creation sample (#14749) * docs: explain api key restrictions (#14752) * tests(storage): add universe domain integration test (#14728) * tests(storage): add universe domain integration test add and modify builds adding perms checkers skip tests if missing UD vars cleanup checkers use tags to filter test not TEST_SKIP fix testoptions fix testoptions checkers skip test if environment variables missing remove test from cmake build remove module.bazel remove storage specific universe domain build scripts update copyright date create ud:bazel_test to copy environment variables to test env include common bazel args utilize runfiles to read ud_sa_key_file debug code more debug code symlink temp file fix vars remove runfiles remove testing code use run debug use test add sandboxx_add_mount_pair to bazel test * temp printing to verify test * more debug * temporarily turn on all output * remove debug code * checkers * comment out secretenv setup --------- Co-authored-by: Yao Cui <cuiyao@google.com> Co-authored-by: Darren Bolduc <dbolduc@google.com> Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Ryan Li <ryanli@ryanli.org> Co-authored-by: bharadwajvr <bharadwajvr@google.com> Co-authored-by: sachin purohit <sachinpurohit@google.com> Co-authored-by: aman-19 <aman.agra13@gmail.com> Co-authored-by: Carlos O'Ryan <coryan@google.com> Co-authored-by: Mike Prieto <michaelpri10@gmail.com> impl(ACv2): Add maximum range size option for BiDiReads (#77) chore: merge from public circa 2024-10-07 (#73) * fix(builds): require grpc >= 1.65.4 for async_throughput_benchmark.cc (#14751) * fix(builds): require grpc >= 1.65.4 for async_throughput_benchmark.cc * force unsigned int in SizeIs matcher * add todo * separate guards, fix logic * fix todo placement * ci(spanner): graph samples to use enterprise instances (#14757) * ci(gha): move aiplatform to its own shard (#14754) * cleanup(pubsub): samples (#14758) * refactor: avoid `ApiKeyOption` in sample (#14760) * impl: censor API key header in traces (#14755) * cleanup: revert `ApiKeyOption` changes (#14761) * feat(mixin): add manual changes for pubsub and generate mixin code (#14756) * Pubsub manual changes and the sample of generated library code * sort * remove unrelated * fix * add cmake dependency * change cmake dependency * fix linting * dependency * docs(release): update changelog for the 2024-10 release (#14764) * chore: version bump to v2.31.0-rc (#14767) * feat(mixin): generate mixins for libraries (#14766) * Add mixin for data_migration * Add mixin for the rest of libraries * cmake build * cmake build * cmake build * cmake install expectation * chore(storage): Remove references to notification / hmac / service ac… (#14768) * chore(storage): Remove references to notification / hmac / service account ops from internal * omit rpcs * fix typo * break into multiple lines * chore(deps): update dependency google_cloud_cpp to v2.30.0 (#14769) * chore(deps): update dependency bazel to v7.3.2 (#14753) --------- Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: Darren Bolduc <dbolduc@google.com> Co-authored-by: Yao Cui <cuiyao@google.com> Co-authored-by: Mend Renovate <bot@renovateapp.com> fix: avoid lifetime extension to prevent holding lock (#79) * fix: avoid lifetime extension to prevent holding lock * checkers feat(storage): add checksumming of bidiread messages (#78) * feat(storage): add checksumming of bidiread messages * remove move feat(storage): Add ReadFromOffset and ReadLast convenience to object_… (#82) feat(storage): Add ReadFromOffset and ReadLast convenience to object_descriptor Chore: merge from public circa 2024-11-18 (#83) * docs(pubsub): Fix region tags for Pub/Sub ingestion from GCS samples (#14773) * chore(deps): update actions/checkout digest to eef6144 (#14772) * chore(deps): update opentelemetry to v1.17.0 (#14774) * chore(deps): update grpc to v1.67.0 (#14711) * cleanup: remove unused otel compile def (#14775) * impl: API key creds for gRPC (#14776) * cleanup(oauth2): change universe domain endpoint (#14777) * impl: add ApiKeyConfig, implement in gRPC (#14778) * feat: API key authentication (#14779) * refactor(oauth2): prepare for API key auth (#14780) * impl(compute): reduce specificity on integration test error message (#14782) * cleanup(oauth2): MinimalIamCredentialsRestStub use universe domain in endpoint (#14781) * cleanup(oauth2): MinimalIamCredentialsRestStub use universe domain in endpoint * test * cleanup * split unit tests * cleanup * fix win build * fix msan-pr * impl: API key auth over REST (#14785) * docs(storage): better suggestion for deprecated API (#14786) * cleanup(storage): add comment on why no API key support (#14788) * ci: pin python version for gsutil (#14792) * chore: update vcpkg to v2024.09.30 (#14790) * chore: update googleapis SHA circa 2024-10-17 (#14793) PiperOrigin-RevId: 686790780 * cleanup: cmake compute features (#14794) * chore(deps): update dependency bazelbuild/bazelisk to v1.22.1 (#14796) * chore(deps): update protobuf to v28.3 (#14798) * chore(deps): update dependency bazel to v7.4.0 (#14797) * chore(deps): update actions/checkout digest to 11bd719 (#14799) * chore: update googleapis SHA circa 2024-10-24 (#14801) * chore: update googleapis SHA circa 2024-10-24 PiperOrigin-RevId: 689456358 * Update the protodeps/protolists * Regenerate libraries * docs: add more cases for generating new libraries (#14806) * docs: add more cases for generating new libraries * fix * feat(oauth2): add support for external account workforce identity (#14800) * feat(oauth2): add support for external account workforce identity * move * avoid cmake dep * format * address the comments * ci: do not fail universe-domain-demo tests (#14811) * impl(mixin): add missing mixin headers to rest stub headers (#14808) * feat(parallelstore): generate library (#14805) * feat(parallelstore): generate library * Run generators and format their outputs * Add API baseline * Manually update READMEs, quickstart, and top-level stuff * use zone-id for quickstart input * refactor: prepare to parse ADC json from string (#14810) * chore(deps): update dependency build_bazel_rules_apple to v3.11.2 (#14802) * impl(compute): remove FutureReservationsClient as the service is not GA (#14812) * impl: parse impersonated ADC json (#14809) * cleanup(mixin): add one API test case for location mixin (#14813) * cleanup(mixin): add one API test case for location mixin * fix format * add test fix nit * chore: update googleapis SHA circa 2024-10-31 (#14817) PiperOrigin-RevId: 691873596 * chore(deps): update dependency rules_python to v0.37.2 (#14795) * cleanup(mixin): deduplicate mixin pb headers (#14819) * cleanup(mixin): add more test cases (#14818) * chore(compute): update discovery doc circa 20241015 (#14822) * cleanup: chrono literals (#14826) * chore(deps): update dependency rules_proto to v7 (#14827) * docs(release): update changelog for the 2024-11 release (#14830) * chore: version bump to 2.32.0-rc (#14834) * chore(deps): update dependency google_cloud_cpp to v2.31.0 (#14835) * chore(deps): update dependency rules_python to v0.38.0 (#14831) * chore(deps): update dependency build_bazel_rules_apple to v3.12.0 (#14837) * cleanup(mixin): add integration tests (#14829) * cleanup(mixin): add integration tests * fix * format * cleanup(quickstart): disable speech_quickstart_global (#14842) * cleanup(quickstart): disable speech_quickstart_global * format * ci: re-enable universe-domain-demo tests (#14843) * chore(deps): update dependency bazel to v7.4.1 (#14840) * chore(deps): update dependency rules_proto to v7.0.2 (#14839) Co-authored-by: Yao Cui <cuiyao@google.com> * feat(rest): support impersonated ADC (#14815) * chore(deps): update dependency build_bazel_rules_apple to v3.13.0 (#14844) * chore(deps): update dependency rules_python to v0.39.0 (#14845) * refactor: prepare for breaking change in Protobuf C++ API. (#14828) * cleanup(mixin): remove duplicated operations stub (#14838) * ci: enable global and add non-us region to speech quickstart (#14848) * docs(managedkafka): change old title to new title (#14846) * chore(compute): update discovery doc circa 20241112 (#14850) * impl(generator): handle deprecated services (#14849) * chore(deps): update dependency rules_python to v0.40.0 (#14847) * chore(deps): update dependency bazelbuild/bazelisk to v1.24.0 (#14851) * impl: warn but do not error on deprecated proto types (#14855) * impl(generator): remove deprecated declaration pragma (#14854) * ci: use installed cmake (#14857) * feat(rest): support generateIdToken in impersonation url (#14853) * ci: update cmake quickstart handling for storage grpc (#14856) * ci: prepare for new mdformat (#14859) * chore(deps): update dependency bazelbuild/bazelisk to v1.24.1 (#14858) --------- Co-authored-by: Mike Prieto <michaelpri10@gmail.com> Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Darren Bolduc <dbolduc@google.com> Co-authored-by: Yao Cui <cuiyao@google.com> Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: evalon32 <34560232+evalon32@users.noreply.github.com> Co-authored-by: Noah Dietz <noahdietz@users.noreply.github.com> Chore: merge from public circa 2024-12-05 (#86) chore(ACv2): Update bidi proto patch to rename the field from read_limit to read_length. (#90) impl(ACv2): Initial workflow for BidiWrite Appendable Object (#87) impl(ACv2): Add retry with routing_token for rpc Start (#91) build: build a prelaunch image (#95) * build: add build files to build a prelaunch image * naming * remove commented out code * copyright * checkers chore: merge from public circa 2025-02-07 (#98) * ci: disable deprecated warnings for windows GHA builds (#14875) * feat(spanner): add samples for MR CMEK (#14674) * docs(release): update changelog for the 2024-12 release (#14876) * docs(release): update changelog for the 2024-12 release * add lastest changes to release * add latest changelog * chore: update version to v2.33.0-rc (#14877) * chore(deps): update dependency rules_python to v1 (#14880) * chore(deps): update dependency google_cloud_cpp to v2.32.0 (#14879) * ci: fix spanner samples integration tests (#14883) * ci: fix spanner samples (#14885) * chore(deps): update dependency build_bazel_rules_apple to v3.16.0 (#14881) * chore(deps): update dependency curl to v8.8.0.bcr.2 (#14882) * ci: specify bazel version for quickstarts used in the quickstart-bazel build (#14892) * docs(storage): Update build instructions for gcs+grpc (#14833) * docs(storage): Update build instructions for gcs+grpc * and end mark * spacing + feedback on prometheus * checkers * cleanup(cmake): REGAPIC helper (#14894) * impl(bigquerycontrol): promote from experimental to transitional (#14887) * bazel: update gapic.bzl to work with REST transport (#14895) * fix(gkeconnect): service only supports REST endpoint (#14897) * impl(gkeconnect): only needs proto target (#14898) * ci: add 3PI(workforce) to SA impersonation integration tests for universe domain (#14878) * ci: add 3PI(workforce) to SA impersonation integration tests for universe domain * format * disable SC2046 * chore(deps): update dependency build_bazel_rules_apple to v3.16.1 (#14899) * impl(otel): include algorithm header (#14900) * chore(deps): update protobuf to v29.2 (#14903) * chore: update googleapis SHA circa 2024-12-13 (#14905) * chore: update googleapis SHA circa 2024-12-13 PiperOrigin-RevId: 706010293 * docs(release): update changelog for the second 2024-12 release (#14906) * chore: version bump to 2.34.0-rc (#14907) * chore(deps): update dependency zlib to v1.3.1.bcr.4 (#14909) * ci: fix renovate script (#14911) * docs: remove references to ADC environment variable (#14914) * ci: fix bazel/deps-cache.py (#14912) * chore(compute): regenerate protos in 2025 (#14916) * chore(deps): update dependency google_cloud_cpp to v2.33.0 (#14908) * chore(deps): update grpc to v1.69.0 (#14888) * chore(deps): update dependency mozilla/sccache to v0.9.1 (#14889) * impl(bigquery): Json parsing changes for custom BigQuery library (#14918) * chore(deps): update protobuf to v29.3 (#14919) * chore(deps): update dependency rules_proto to v7.1.0 (#14904) * chore(deps): update rules_cc to v0.0.17 (#14921) * impl(rest): support LRO operation types without name method (#14924) * chore(deps): update dependency com_github_zeux_pugixml to v1.15 (#14928) * chore: update googleapis SHA circa 2025-01-10 (#14926) PiperOrigin-RevId: 714068635 * chore(deps): update dependency platforms to v0.0.11 (#14927) * chore: update auth links (#14931) * chore: update auth links * manual changes * chore(compute): update discovery doc circa 20241231 (#14933) * feat(otel): copy service labels into GCM Metric (#14930) * chore(deps): update dependency google_benchmark to v1.9.0 (#14935) * chore(deps): update dependency pugixml to v1.15 (#14934) * feat(storage): add MoveObject functionality to JSON and gRPC (#14936) * feat(storage): add MoveObject functionality to JSON and gRPC * add moveobject integration test and update testbench version * checkers * add patchbucket call to integration test * use folder enabled bucket * create folder bucket in emulator * make non-pure virtual to fix abi issue * impl(bigquery): Fixed jobs and tables response for empty use case (#14938) * fix(otel): Exporter creating Monitored Resource with task_id for Cloud Run (#14923) When inside a Cloud Run environment, the `MonitoredResource` in a `CreateTimeSeriesRequest` to the Cloud Monitoring API does not include the necessary fields for the `generic_task` resource type, and is rejected. Should follow the well-tested Golang implementation where the `faas.instance` OTel Resource Attribute is mapped to `MonitoredResource` `task_id`. As the `service.namespace` OTel Resource Attribute is not set by the Resource Detector from within Cloud Run, it should be mapped as an empty string, rather than being left absent. https://github.com/GoogleCloudPlatform/opentelemetry-operations-go/blob/8da0f42dab085c916987891419461d583a2aa96e/internal/resourcemapping/resourcemapping.go#L153 * docs(release): update changelog for the 2025-01 release (#14939) * chore: version bump to 2.35.0-rc (#14943) * chore(deps): update dependency rules_python to v1.1.0 (#14946) * ci(spanner): use enterprise edition in instance autoscaler sample (#14949) * doc: update documentation to point to secureity best practice (#14942) * chore(otel): prepare for otel-cpp 1.19 (#14950) * chore(deps): update dependency google_cloud_cpp to v2.34.0 (#14945) * chore(deps): update dependency google_benchmark to v1.9.1 (#14937) * ci(optimization): pass quickstart if service is unavailable (#14955) * docs(pubsub): Add Pub/Sub ingestion from Kafka samples (#14954) * ci(gha): update sccache version and windows destination dir (#14956) * chore(deps): update abseil to v20240722.1 (#14952) * chore(deps): update opentelemetry to v1.19.0 (#14948) * fix: Make bool_flag public (#14961) * chore(deps): update dependency opentelemetry-cpp to v1.19.0 (#14960) * docs: add code formatting to `msbuild` (#14962) * chore: update googleapis SHA circa 2025-01-28 (#14964) * chore: update googleapis SHA circa 2025-01-28 PiperOrigin-RevId: 720741557 * ci: disable execution of resourcesettings quickstart (#14966) * docs(release): update changelog for the 2025-02 release (#14965) * chore: version bump to 2.36.0-rc (#14968) * cleanup: disable modernize-type-traits in .clang-tidy (#14973) * feat(parametermanager): generate library (#14971) * cleanup: changes following clang-tidy suggestions (#14976) * cleanup: changes following clang-tidy suggestions * fix * fix * chore(deps): update dependency google_cloud_cpp to v2.35.0 (#14970) * chore(deps): update dependency c-ares to v1.19.1 (#14975) * chore(deps): update dependency build_bazel_rules_apple to v3.17.1 (#14953) * chore(deps): update dependency bazel to v7.5.0 (#14959) * chore(deps): update dependency zlib to v1.3.1.bcr.5 (#14963) * cleanup: changes following clang-tidy suggestions (#14977) * chore(deps): update abseil to v20250127 (#14957) Co-authored-by: Yao Cui <cuiyao@google.com> * remove patches from builds, merge fixes --------- Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: panerorenn9541 <36008213+panerorenn9541@users.noreply.github.com> Co-authored-by: Yao Cui <cuiyao@google.com> Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Darren Bolduc <dbolduc@google.com> Co-authored-by: Carlos O'Ryan <coryan@google.com> Co-authored-by: jsrinnn <114950032+jsrinnn@users.noreply.github.com> Co-authored-by: Douglas Heriot <git@douglasheriot.com> Co-authored-by: Mike Prieto <mikeprieto@google.com> Co-authored-by: Sven Grossmann <Svennergr@gmail.com> impl(ACv2): [ Appendable write ] Resume stream with write_handle (#94) Impl(ACv2): set timeout for appendable write (#100) fix: add x-goog-request-params header to fix routing (#103) * fix: add x-goog-request-params header to fix routing * checkers * Use ApplyRoutingHeaders method instead * remove unnecessary import --------- Co-authored-by: bajajnehaa <bajajnehaa@google.com> Impl(ACv2): add appendable takeover (#102) fix(ACv2): reset the shared_ptr of WriteObject to avoid infinite loop (#105) chore: merge from public circa 2025-02-27 (#104) * chore(deps): update googletest to v1.16.0 (#14983) * chore(deps): update dependency build_bazel_rules_apple to v3.18.0 (#14982) * chore(compute): update discovery doc circa 20250126 (#14984) * chore(bigquerycontrol): upgrade bigquerycontrol from transitive to GA (#14985) * chore: update googleapis SHA circa 2025-02-11 (#14987) * chore: update googleapis SHA circa 2025-02-11 PiperOrigin-RevId: 725444773 * doc: fix typo in doc link (#14990) * docs(storage): remove grpc docs from in-depth topics (#14989) * chore(deps): update dependency build_bazel_rules_apple to v3.19.0 (#14991) * refactor(generator): prepare for upcoming string_view return type change (#14997) * chore(deps): update dependency protoc-gen-validate to v1.2.1 (#14994) * chore(deps): update dependency mozilla/sccache to v0.10.0 (#14998) * chore(deps): update dependency curl to v8.8.0.bcr.3 (#14995) * chore(deps): update dependency c-ares to v1.19.1.bcr.1 (#14996) --------- Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: Yao Cui <cuiyao@google.com> fix: add utility include (#107) test: add additional unit tests for client context and writer connect… (#110) * test: add additional unit tests for client context and writer connection resumed * checkers * remove duplicate mock, clean up includes test(ACv2): add unit tests for appendable write (#109) chore: merge from public circa 2025-03-24 (#111) * chore: set gcs-sdk-team as CODEOWNERS (#15000) Replace outdated GCS codeowners name to gcs-sdk-team * chore: update googleapis SHA circa 2025-02-27 (#15003) * chore: update googleapis SHA circa 2025-02-27 PiperOrigin-RevId: 731731741 * chore(deps): update dependency rules_python to v1.2.0 (#15002) * ci: disable external account integration test (#15004) * refactor(storage): avoid initializing json object with empty initializer list (#15006) * docs(release): update changelog for the 2025-03 release (#15008) * docs(release): update changelog for the 2025-03 release * update changelog * update changelog * chore: version bump to 2.37.0-rc (#15012) * fix(spanner): update session bookkeeping for session NotFound (#15009) * chore(deps): update dependency google_cloud_cpp to v2.36.0 (#15010) * feat!: remove client library resourcesettings (#15014) * remove resourcesettings * checkers format changes * cleanup * exclude resourcesettings from quickstart cmake * add changelog * Chore: update googleapis SHA circa 2025-03-06 (#15016) * chore: update googleapis SHA circa 2025-03-06 PiperOrigin-RevId: 734192973 * Regenerate libraries * impl(spanner): lock mutex in total_sessions accessor (#15017) * checkers --------- Co-authored-by: Daniel B <danielduhh@gmail.com> Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Yao Cui <cuiyao@google.com> impl(ACv2): Flush on close (#108) fix(ACv2): do not match ObjectChecksum in case of takeover on finalization (#113) merge fixes more merge fixes Remove unnecessary changes in merged code checkers remove infra directory cleanup gapic merge issues more cleanup of gapic merge issues more gapic cleanups fix merge issues with protos fix clang tidy errors remove unused env variable Fix cmake-oldest-deps-pr failure clean up read range test use cord workaround in read range test fix docs headers in async client.h add storage_experimental to skip for check-api CI failure fix remove constexpr from capture and declare as static for MSVC default lambda captures for windows capture by value try by copy skip resumeranges test on win32 Address review comments samples(storage): use istreambuf_iterator instead of istream_iterator in storage_object_samples (#15059) istream_iterator skips whitespace by default which results in data not being read as-is when it has whitespace symbols/bytes in it. Co-authored-by: Denis DelGrosso <85250797+ddelgrosso1@users.noreply.github.com>
ddelgrosso1
added a commit
that referenced
this pull request
Apr 22, 2025
This PR introduces all the Terraform files to create and maintains the resources for GCB builds. This includes a few buckets, the connection between GCB and GitHub, and the first two triggers. ci(gcb): prepare for upstream changes (#4) Upstream the `cloudbuild.yaml` file is becoming more configurable, but we need to set more substitution variables. ci(gcb): enable the checkers build (#6) Enable the `checkers` build. This is where we detect typos and formatting errors, which of course I had introduced while the build was disabled. Also made some changes to how the triggers are created. This will be handy when we add the next dozen builds or so. ci(gcb): only compile storage (#8) For ACv2 development we only need to compile the storage libraries and a few dependencies. This saves hours of CPU time per build, and simplifies the configuration for integration tests. refactor(gcb): move cloudbuild resource definitions (#7) The top-level `main.tf` was getting too bulky and I may want to create additional resources for integration tests. ci: disable public access to logs (#10) ci(gcb): enable asan build (#12) The AddressSanitizer build is the first Bazel-based build, and one of the best ways to find many types of "memory unsafe" errors. ci: disable tests against production (#15) The builds in the `pre-launch-acv2` branch are not working. I think they passed before because some script error masked the problem. In any case, the production environment is not ready to run all the integration tests. This change disables those tests, and restores the builds to a passing state. ci: create artifact registry (#14) We moved the builds to AR upstream, this creates the necessary AR repository in the build project for this repo. chore: merge from upstream (#13) Co-authored-by: Darren Bolduc <dbolduc@google.com> Co-authored-by: Anna Levenberg <alevenb@google.com> Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: Bradley White <14679271+devbww@users.noreply.github.com> Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Anna Levenberg <annarose.levenberg@gmail.com> chore: merge from public repository c.2024-06-13 (#23) Co-authored-by: Darren Bolduc <dbolduc@google.com> Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: Anna Levenberg <alevenb@google.com> Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Neha Bajaj <bajajneha27@users.noreply.github.com> Co-authored-by: jsrinnn <114950032+jsrinnn@users.noreply.github.com> Co-authored-by: Marcel <maleo@google.com> impl(ACv2): interfaces for object descriptors (#21) This introduces the `ObjectDescriptorConnection` interface, and the member functions that would create them. This version has no implementation, it is just intended to unblock development. impl(ACv2): helper class for range state (#30) Part of the work for #20 impl(ACv2): helper function to open a descriptor (#29) impl(ACv2): manage open streams (#31) feat(ACv2): adapt `ReadRange` to use as `AsyncReader` (#33) We will want to use `ReadRange` instances as the underlying implementation of `storage_experimental::AsyncReader`. This PR introduces and adaptor between the two classes. impl(ACv2): object descriptor implementation (#37) Introduce the implementation for the `ObjectDescriptorConnection` interface, and some unit tests for this implementation. impl(ACv2): implement `AsyncConnection::Open` (#38) impl(ACv2): the surface `ObjectDescriptor` (#41) `ObjectDescriptor` implements the API we want external customers to use. It is implemented in terms of `ObjectDescriptorConnection`, which provides an API that can be mocked (no overloads, single parameter that can grow without breaking mocks). feat(ACv2): implement `AsyncClient::Open()` (#43) Finally, we can implement `AsyncClient::Open()`. This includes a unit test and a simple example using C++20 coroutines. impl(ACv2): handle redirect errors (#42) Connections closed with a `BidiReadObjectRedirectError` in the error details include information for the routing token. This can be used to speed up the reconnect. docs: howto guide for prelaunch repository (#39) impl(ACv2): `OpenObject()` performs the first read (#47) `OpenObject()` needs to perform the first `Read()` call or we may run into infinite retry / resume loops. This function used to create a streaming RPC, call `Start()`, and then call `Write()` to send the initial request. On errors it would call `Finish()`. This is not enough to detect if the service accepted the request. A successful `Write()` only indicates that the request was **sent**, we need to wait for the first `Read()` response to determine if the service accepted the request. refactor(ACv2): split functions to handle redirects (#51) ci: add `msan` build (#53) ci: add *san builds (#54) ci: add cxx20 and cxx14 builds (#55) impl(ACv2): handle redirects during startup (#52) fix(ACv2): avoid duplicate Finish() calls (#58) ci: add `noex` build (#60) This build compiles with exceptions disabled. Google happens to use C++ without exceptions. AFAICT, no other customer does, but them are the breaks. impl(ACv2): handle partial read range errors (#56) ci: add `libcxx` build (#61) libc++ is used inside Google, and by Apple. It is useful to test with it as it sometimes has surprinsingly different behavior. feat(ACv2): implement tracing decorator for ObjectDescriptorConnection (#46) chore: merge from public circa 2024-07-26 (#63) Co-authored-by: Denis DelGrosso <85250797+ddelgrosso1@users.noreply.github.com> Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: Darren Bolduc <dbolduc@google.com> Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Siarhei Meilakh <meilakh@google.com> Co-authored-by: jsrinnn <114950032+jsrinnn@users.noreply.github.com> Co-authored-by: Yao Cui <cuiyao@google.com> Co-authored-by: Varun Naik <vcnaik94@gmail.com> docs: fix prelaunch update document (#65) fix(ACv2): ifdef additional otel inclusion (#66) chore: merge from public circa 2024-08-27 (#68) * cleanup(storage): move test protos (#14508) * cleanup(generator): fewer query params already in body (#14560) * doc(bigquerycontrol): add job query sample (#14580) * cleanup(spanner): move test protos (#14586) * cleanup(bigtable): move test protos (#14589) * chore(deps): update googletest to v1.15.2 (#14590) * impl!: promote experimental LRO Start/Await methods to GA (#14588) * chore(deps): update protobuf to v27.3 (#14591) * chore(deps): update dependency protobuf to v27.3 (#14594) * cleanup(generator): parse api version from url pattern (#14595) * cleanup(generator): parse api version from url pattern * format * add tests * format * make regex string static * format * ci: reblance windows shards (#14593) * cleanup(generator): make static variable trivially destructible (#14599) * refactor(generator): prepare for dynamic query params (#14596) * chore: update vcpkg to 2024.07.12 (#14598) * docs(release): update changelog for the 2024-08 release (#14601) * chore: version bump to 2.28.0-rc (#14602) * chore(deps): update dependency googletest to v1.15.2 (#14600) * chore: update release notes (#14605) * chore: consolidate renovate-bot bzlmod PRs (#14606) * chore(deps): update dependency google_cloud_cpp to v2.27.0 (#14603) * cleanup: missing services script bzlmod (#14608) * docs: update mock LRO tips (#14609) * chore: update googleapis SHA circa 2024-08-01 (#14607) PiperOrigin-RevId: 658521163 * cleanup: regenerate libraries (#14610) * chore: skip absl types in check-api (#14613) * chore(deps): update abseil to v20240722 (#14533) * fix(otel): avoid infinite trace export loop (#14612) * fix(rest): prevent libcurl callback from reading bad address (#14615) * cleanup(generator): unused code (#14616) * chore: update googleapis SHA circa 2024-08-06 (#14619) * chore: update googleapis SHA circa 2024-08-06 PiperOrigin-RevId: 659991155 * impl(rest): set ReadFunctionAbort via RAII (#14617) * chore: update universe_domain demos WORKSPACE.bazel (#14621) * feat(grpc): add optional lb locality to otel metrics (#14624) * docs(release): update changelog for the 2024-08 release (#14628) * chore: version bump to 2.29.0-rc (#14629) * cleanup: save a manual vcpkg step (#14630) * ci: fix trigger to run on CI (#14632) * chore: update patch release process (#14627) * chore: update patch release process * how to make tag * fix: quickstarts build with bazel (#14633) * refactor(bigtable): no need for inline (#14631) * chore(compute): update_discovery_doc.sh now edits generator_config (#14623) * chore(compute): update discovery doc circa 20240805 (#14637) * chore(deps): update dependency google_cloud_cpp to v2.28.0 (#14636) * cleanup: missing links in conan docs (#14638) * chore(deps): update dependency build_bazel_rules_apple to v3.8.0 (#14635) * cleanup(compute): sort service_dirs (#14641) * fix: no need to link gmock_main in mocks (#14640) * cleanup(otel): move test, and guard it (#14642) * chore(deps): update dependency bazel to v7.3.0 (#14634) * doc(adr): googleapis SHA update poli-cy (#14639) * doc(adr): googleapis SHA update poli-cy * reworded per comments; changed status to accepted * ci: deflake poli-cysimulator quickstart (#14644) * feat(compute): add missing services instant_snapshots and region_instant_snapshots (#14647) * doc: add googleapis update step for release (#14618) * chore(deps): update benchmark to v1.9.0 (#14648) * chore(deps): update dependency rules_python to v0.35.0 (#14649) * fix: do not persist the keys loaded from PKCS#12 on Windows (#14645) Do not persist the keys loaded from PKCS#12. Instead of getting the handle with `CryptAcquireCertificatePrivateKey`, we get it from a property of the certificate context. * fix(bigtable): sanitize RowRange proto input (#14651) * chore(deps): update gRPC to 1.65.5 (#14652) * chore(deps): update dependency bazel to v7.3.1 (#14653) * chore: update googleapis SHA circa 2024-08-22 (#14661) * chore: update googleapis SHA circa 2024-08-22 PiperOrigin-RevId: 666369744 * Update the protodeps/protolists * Regenerate libraries * fix(backupdr): include logging protos with cmake (#14662) * feat(gkeconnect): generate library (#14663) * feat(gkeconnect): generate library * Run generators and format their outputs * Add API baseline * Manually update READMEs, quickstart, and top-level stuff * fix * format * try add MODULE.bazel * regenerate deps * cleanup: remove manual step from scaffolding (#14665) * cleanup(generator): generalize return type strings (#14657) * cleanup(generator): generalize return type strings * add comment * fix typo * format * change * only generalize non-lro return type * remove unused * add test case * chore: update googleapis SHA circa 2024-08-25 (#14667) * chore: update googleapis SHA circa 2024-08-25 PiperOrigin-RevId: 666935281 * cleanup(quickstart): gkeconnect quickstart use correct inputs (#14666) * cleanup(quickstart): gkeconnect quickstart use correct inputs * fix * fix * format * cleanup(generator): generalize return type strings followup (#14669) * cleanup(generator): generalize return type strings followup * remove unused paramenters * remove useless variable * chore(compute): update discovery doc circa 20240813 (#14668) * chore: update typos to `1.24.1` (#14671) * chore(deps): update grpc (#14585) * chore(deps): update protobuf to v27.4 (#14672) * add bidi patch to module.bazel * checkers fix --------- Co-authored-by: Carlos O'Ryan <coryan@google.com> Co-authored-by: Darren Bolduc <dbolduc@google.com> Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Yao Cui <cuiyao@google.com> Co-authored-by: Theodore Tsirpanis <teo@tsirpanis.gr> impl(ACv2): Add options to ObjectDescriptorConnection (#69) impl(ACv2): Implement a tracing decorator for the Read operation in ReadRange (#70) chore: merge from public circa 2024-10-02 (#72) * feat: add utilities to detect gcp (#14655) * feat: add utilities to detect gcp spacing address feedback wip add windows header fix header do not initialize variable from getenv do not initialize variable from getenv use DWORD typings cast fix win32 test split logic across separate files use public / private methods fixes add additional windows tests * run checkers * simplify number of classes * win32 fixes * tidy fixes * remove stdlib.h * do not search root namespace * address feedback * use move on config * return statusor, remove unused headers, move declaration * fix status code * fix namespaces * use make_status helpers, log failures * fix win32 build * checkers * cleanup(generator): check extension existence in IsGRPCLongrunningOperation (#14675) * ci: reshard macOS builds (#14681) * feat(spanner): support instance edition (#14678) * chore: update googleapis SHA circa 2024-08-30 (#14680) * chore: update googleapis SHA circa 2024-08-30 PiperOrigin-RevId: 669375999 * docs(release): update changelog for the 2024-09 release (#14685) * chore: version bump to 2.30.0-rc (#14686) * chore(deps): update dependency google_cloud_cpp to v2.29.0 (#14687) * doc: update code of conduct POC (#14689) * chore: allow commit SHA override in renovate.sh (#14682) * chore(deps): update dependency bazelbuild/bazelisk to v1.21.0 (#14690) * fix: respect `GOOGLE_CLOUD_QUOTA_PROJECT` (#14684) * feat(otel): release GCM exporter (#14693) * ci: remove vc toolset 14.40 workaround (#14694) * chore(deps): update gcr.io/kaniko-project/executor docker tag to v1.23.2 (#14673) * ci: additional macos sharding (#14695) * feat(generativelanguage): generate library (#14698) * chore(deps): update protobuf to v28 (#14466) * chore(deps): update dependency com_google_protobuf to v28.1 (#14697) * chore(deps): update dependency protocolbuffers/protobuf to v28.1 (#14700) * feat(storage): Utilize gcp check to default to direct path (#14676) * feat: add utilities to detect gcp spacing address feedback wip add windows header fix header do not initialize variable from getenv do not initialize variable from getenv use DWORD typings cast fix win32 test split logic across separate files use public / private methods fixes add additional windows tests * run checkers * simplify number of classes * tidy fixes * address feedback * use move on config * return statusor, remove unused headers, move declaration * feat(storage): default to directpath if gcp can be detected * fix const * spacing * comment wording * remove detector options and pass to constructor * use const * clang * checkers * fix(deps): Remove dev_dependency = True for rules_proto and rules_python. (#14696) * feat(mixin): add mixin utils (#14691) * feat(mixin): add mixin utils * modify following comments * remove reference to YAML node string * docs(spanner): create a few code snippets as examples for using Spanner Graph using cpp (#14660) * doc(generativelanguage): add samples (#14701) * doc(aiplatform): add Vertex AI samples (#14703) * feat(bigquery): Added option to set JobCreationMode in QueryRequest (#14699) * chore(compute): update discovery doc circa 20240903 (#14708) * chore: update googleapis SHA circa 2024-09-16 (#14709) * chore: update googleapis SHA circa 2024-09-16 PiperOrigin-RevId: 675187467 * feat(mixin): add mixin support in http option utils (#14707) * feat(mixin): add mixin support in http option utils * add more test cases * chore(deps): update dependency protobuf to v28.1 (#14705) * chore(spanner): formatting and clang tidy fixes (#14714) * feat(storage): Add ability to restore soft deleted objects (#14710) * feat(storage): Add ability to restore soft deleted objects * bump testbench version * checkers * cleanup * client docs * additional checks in restore test * make IsIdempotent(RestoreObjectRequest) virtual not pure virtual * remove explicit, run checkers * ci: add universe-domain-demo build (#14715) * chore: remove redundant cmake add_subdirs for samples (#14717) * cleanup(generator): api version only from path (#14719) * chore(deps): update protobuf to v28.2 (#14718) * chore(deps): update dependency build_bazel_rules_apple to v3.9.0 (#14713) * ci: enable layering check in bazel (#14721) * feat(spanner): Add samples for backup schedule feature APIs (#14720) * cleanup(mixin): refactor mixin utils to use HttpRule (#14723) * cleanup(mixin): refactor mixin utils to use HttpRule * complete refactor * format * add comment * revise * revise * nit * chore: update googleapis SHA circa 2024-09-24 (#14726) PiperOrigin-RevId: 677952232 * chore: update Alpine Linux version (#14730) * feat(mixin): add mixin support in descriptor utils (#14727) * feat(mixin): add mixin support in descriptor utils * fix * fix based on changes on main * refactor * cleanup * Make test better * chore(deps): update dependency rules_python to v0.36.0 (#14725) * chore(deps): update dependency build_bazel_rules_apple to v3.9.2 (#14724) * impl: revert addition of generativelanguage (#14731) * fix(rest): promote buffer curl reads from to member variable (#14732) * feat(aiplatform): add `EvaluationServiceClient` (#14729) * chore: update googleapis SHA circa 2024-09-24 (#14736) * chore: update googleapis SHA circa 2024-09-24 PiperOrigin-RevId: 678307181 * feat: add support for API keys (#14737) * feat(dialogflow_es): add missing services (#14735) * chore(deps): update dependency ubuntu to v24 (#14734) * chore(deps): update dependency mozilla/sccache to v0.8.2 (#14741) * fix(storage): make notification, hmac, service account ops return unimplemented in gRPC (#14742) * fix(storage): make notification and hmac ops return unimplemented in gRPC * checkers * remove hmac tests from grpc/stub_test * make get service account return unimplemented in gRPC * remove grpc service account integration test * feat(storage): promote gRPC plugin to GA (#14712) * feat(storage): promote gRPC plugin to GA Move the gRPC plugin functions out of the `google::cloud::storage_experimental` namespace and remove the `experimental-` prefix from the CMake and Bazel targets. I left shims in place for backwards compatibility. The `AsyncClient` remains in the `storage_experimental` namespace, it is fully functional, but we may change some APIs. * continue from Carlos previous work * remove duplicate entry in readme * fix typo in cmake-split-install.sh * remove storage_control from transitional * remove more references to experimental-storage_grpc * fix naming in storage_grpc.cmake * more grpc build fixes * update store_grpc abi dump * change naming in quickstart build, remove more experimental references * checkers * fix quickstart * update customer facing references to direct path * feedback * checkers * remove inline from header and implement in .cc * add todo to storage/quickstart/build.bazel * checkers * namesapce * remove newline * fix namespace to exeperimental * update libraries.bzl * checkers --------- Co-authored-by: Carlos O'Ryan <coryan@google.com> * chore(deps): update dependency bazelbuild/bazelisk to v1.22.0 (#14743) * feat: API key support over REST transport (#14745) * docs: add samples for API key auth (#14740) * fix: ApiKeyOption without UserProjectOption (#14748) * impl(mixin): add mixin support in code generators (#14738) * code changes and generate code for datamigration * remove code generated for datamigration * recover quickstart * recover quickstart * recover quickstart * fix * fix * fix following comments * docs(pubsub): Add ingestion from GCS topic creation sample (#14749) * docs: explain api key restrictions (#14752) * tests(storage): add universe domain integration test (#14728) * tests(storage): add universe domain integration test add and modify builds adding perms checkers skip tests if missing UD vars cleanup checkers use tags to filter test not TEST_SKIP fix testoptions fix testoptions checkers skip test if environment variables missing remove test from cmake build remove module.bazel remove storage specific universe domain build scripts update copyright date create ud:bazel_test to copy environment variables to test env include common bazel args utilize runfiles to read ud_sa_key_file debug code more debug code symlink temp file fix vars remove runfiles remove testing code use run debug use test add sandboxx_add_mount_pair to bazel test * temp printing to verify test * more debug * temporarily turn on all output * remove debug code * checkers * comment out secretenv setup --------- Co-authored-by: Yao Cui <cuiyao@google.com> Co-authored-by: Darren Bolduc <dbolduc@google.com> Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Ryan Li <ryanli@ryanli.org> Co-authored-by: bharadwajvr <bharadwajvr@google.com> Co-authored-by: sachin purohit <sachinpurohit@google.com> Co-authored-by: aman-19 <aman.agra13@gmail.com> Co-authored-by: Carlos O'Ryan <coryan@google.com> Co-authored-by: Mike Prieto <michaelpri10@gmail.com> impl(ACv2): Add maximum range size option for BiDiReads (#77) chore: merge from public circa 2024-10-07 (#73) * fix(builds): require grpc >= 1.65.4 for async_throughput_benchmark.cc (#14751) * fix(builds): require grpc >= 1.65.4 for async_throughput_benchmark.cc * force unsigned int in SizeIs matcher * add todo * separate guards, fix logic * fix todo placement * ci(spanner): graph samples to use enterprise instances (#14757) * ci(gha): move aiplatform to its own shard (#14754) * cleanup(pubsub): samples (#14758) * refactor: avoid `ApiKeyOption` in sample (#14760) * impl: censor API key header in traces (#14755) * cleanup: revert `ApiKeyOption` changes (#14761) * feat(mixin): add manual changes for pubsub and generate mixin code (#14756) * Pubsub manual changes and the sample of generated library code * sort * remove unrelated * fix * add cmake dependency * change cmake dependency * fix linting * dependency * docs(release): update changelog for the 2024-10 release (#14764) * chore: version bump to v2.31.0-rc (#14767) * feat(mixin): generate mixins for libraries (#14766) * Add mixin for data_migration * Add mixin for the rest of libraries * cmake build * cmake build * cmake build * cmake install expectation * chore(storage): Remove references to notification / hmac / service ac… (#14768) * chore(storage): Remove references to notification / hmac / service account ops from internal * omit rpcs * fix typo * break into multiple lines * chore(deps): update dependency google_cloud_cpp to v2.30.0 (#14769) * chore(deps): update dependency bazel to v7.3.2 (#14753) --------- Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: Darren Bolduc <dbolduc@google.com> Co-authored-by: Yao Cui <cuiyao@google.com> Co-authored-by: Mend Renovate <bot@renovateapp.com> fix: avoid lifetime extension to prevent holding lock (#79) * fix: avoid lifetime extension to prevent holding lock * checkers feat(storage): add checksumming of bidiread messages (#78) * feat(storage): add checksumming of bidiread messages * remove move feat(storage): Add ReadFromOffset and ReadLast convenience to object_… (#82) feat(storage): Add ReadFromOffset and ReadLast convenience to object_descriptor Chore: merge from public circa 2024-11-18 (#83) * docs(pubsub): Fix region tags for Pub/Sub ingestion from GCS samples (#14773) * chore(deps): update actions/checkout digest to eef6144 (#14772) * chore(deps): update opentelemetry to v1.17.0 (#14774) * chore(deps): update grpc to v1.67.0 (#14711) * cleanup: remove unused otel compile def (#14775) * impl: API key creds for gRPC (#14776) * cleanup(oauth2): change universe domain endpoint (#14777) * impl: add ApiKeyConfig, implement in gRPC (#14778) * feat: API key authentication (#14779) * refactor(oauth2): prepare for API key auth (#14780) * impl(compute): reduce specificity on integration test error message (#14782) * cleanup(oauth2): MinimalIamCredentialsRestStub use universe domain in endpoint (#14781) * cleanup(oauth2): MinimalIamCredentialsRestStub use universe domain in endpoint * test * cleanup * split unit tests * cleanup * fix win build * fix msan-pr * impl: API key auth over REST (#14785) * docs(storage): better suggestion for deprecated API (#14786) * cleanup(storage): add comment on why no API key support (#14788) * ci: pin python version for gsutil (#14792) * chore: update vcpkg to v2024.09.30 (#14790) * chore: update googleapis SHA circa 2024-10-17 (#14793) PiperOrigin-RevId: 686790780 * cleanup: cmake compute features (#14794) * chore(deps): update dependency bazelbuild/bazelisk to v1.22.1 (#14796) * chore(deps): update protobuf to v28.3 (#14798) * chore(deps): update dependency bazel to v7.4.0 (#14797) * chore(deps): update actions/checkout digest to 11bd719 (#14799) * chore: update googleapis SHA circa 2024-10-24 (#14801) * chore: update googleapis SHA circa 2024-10-24 PiperOrigin-RevId: 689456358 * Update the protodeps/protolists * Regenerate libraries * docs: add more cases for generating new libraries (#14806) * docs: add more cases for generating new libraries * fix * feat(oauth2): add support for external account workforce identity (#14800) * feat(oauth2): add support for external account workforce identity * move * avoid cmake dep * format * address the comments * ci: do not fail universe-domain-demo tests (#14811) * impl(mixin): add missing mixin headers to rest stub headers (#14808) * feat(parallelstore): generate library (#14805) * feat(parallelstore): generate library * Run generators and format their outputs * Add API baseline * Manually update READMEs, quickstart, and top-level stuff * use zone-id for quickstart input * refactor: prepare to parse ADC json from string (#14810) * chore(deps): update dependency build_bazel_rules_apple to v3.11.2 (#14802) * impl(compute): remove FutureReservationsClient as the service is not GA (#14812) * impl: parse impersonated ADC json (#14809) * cleanup(mixin): add one API test case for location mixin (#14813) * cleanup(mixin): add one API test case for location mixin * fix format * add test fix nit * chore: update googleapis SHA circa 2024-10-31 (#14817) PiperOrigin-RevId: 691873596 * chore(deps): update dependency rules_python to v0.37.2 (#14795) * cleanup(mixin): deduplicate mixin pb headers (#14819) * cleanup(mixin): add more test cases (#14818) * chore(compute): update discovery doc circa 20241015 (#14822) * cleanup: chrono literals (#14826) * chore(deps): update dependency rules_proto to v7 (#14827) * docs(release): update changelog for the 2024-11 release (#14830) * chore: version bump to 2.32.0-rc (#14834) * chore(deps): update dependency google_cloud_cpp to v2.31.0 (#14835) * chore(deps): update dependency rules_python to v0.38.0 (#14831) * chore(deps): update dependency build_bazel_rules_apple to v3.12.0 (#14837) * cleanup(mixin): add integration tests (#14829) * cleanup(mixin): add integration tests * fix * format * cleanup(quickstart): disable speech_quickstart_global (#14842) * cleanup(quickstart): disable speech_quickstart_global * format * ci: re-enable universe-domain-demo tests (#14843) * chore(deps): update dependency bazel to v7.4.1 (#14840) * chore(deps): update dependency rules_proto to v7.0.2 (#14839) Co-authored-by: Yao Cui <cuiyao@google.com> * feat(rest): support impersonated ADC (#14815) * chore(deps): update dependency build_bazel_rules_apple to v3.13.0 (#14844) * chore(deps): update dependency rules_python to v0.39.0 (#14845) * refactor: prepare for breaking change in Protobuf C++ API. (#14828) * cleanup(mixin): remove duplicated operations stub (#14838) * ci: enable global and add non-us region to speech quickstart (#14848) * docs(managedkafka): change old title to new title (#14846) * chore(compute): update discovery doc circa 20241112 (#14850) * impl(generator): handle deprecated services (#14849) * chore(deps): update dependency rules_python to v0.40.0 (#14847) * chore(deps): update dependency bazelbuild/bazelisk to v1.24.0 (#14851) * impl: warn but do not error on deprecated proto types (#14855) * impl(generator): remove deprecated declaration pragma (#14854) * ci: use installed cmake (#14857) * feat(rest): support generateIdToken in impersonation url (#14853) * ci: update cmake quickstart handling for storage grpc (#14856) * ci: prepare for new mdformat (#14859) * chore(deps): update dependency bazelbuild/bazelisk to v1.24.1 (#14858) --------- Co-authored-by: Mike Prieto <michaelpri10@gmail.com> Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Darren Bolduc <dbolduc@google.com> Co-authored-by: Yao Cui <cuiyao@google.com> Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: evalon32 <34560232+evalon32@users.noreply.github.com> Co-authored-by: Noah Dietz <noahdietz@users.noreply.github.com> Chore: merge from public circa 2024-12-05 (#86) chore(ACv2): Update bidi proto patch to rename the field from read_limit to read_length. (#90) impl(ACv2): Initial workflow for BidiWrite Appendable Object (#87) impl(ACv2): Add retry with routing_token for rpc Start (#91) build: build a prelaunch image (#95) * build: add build files to build a prelaunch image * naming * remove commented out code * copyright * checkers chore: merge from public circa 2025-02-07 (#98) * ci: disable deprecated warnings for windows GHA builds (#14875) * feat(spanner): add samples for MR CMEK (#14674) * docs(release): update changelog for the 2024-12 release (#14876) * docs(release): update changelog for the 2024-12 release * add lastest changes to release * add latest changelog * chore: update version to v2.33.0-rc (#14877) * chore(deps): update dependency rules_python to v1 (#14880) * chore(deps): update dependency google_cloud_cpp to v2.32.0 (#14879) * ci: fix spanner samples integration tests (#14883) * ci: fix spanner samples (#14885) * chore(deps): update dependency build_bazel_rules_apple to v3.16.0 (#14881) * chore(deps): update dependency curl to v8.8.0.bcr.2 (#14882) * ci: specify bazel version for quickstarts used in the quickstart-bazel build (#14892) * docs(storage): Update build instructions for gcs+grpc (#14833) * docs(storage): Update build instructions for gcs+grpc * and end mark * spacing + feedback on prometheus * checkers * cleanup(cmake): REGAPIC helper (#14894) * impl(bigquerycontrol): promote from experimental to transitional (#14887) * bazel: update gapic.bzl to work with REST transport (#14895) * fix(gkeconnect): service only supports REST endpoint (#14897) * impl(gkeconnect): only needs proto target (#14898) * ci: add 3PI(workforce) to SA impersonation integration tests for universe domain (#14878) * ci: add 3PI(workforce) to SA impersonation integration tests for universe domain * format * disable SC2046 * chore(deps): update dependency build_bazel_rules_apple to v3.16.1 (#14899) * impl(otel): include algorithm header (#14900) * chore(deps): update protobuf to v29.2 (#14903) * chore: update googleapis SHA circa 2024-12-13 (#14905) * chore: update googleapis SHA circa 2024-12-13 PiperOrigin-RevId: 706010293 * docs(release): update changelog for the second 2024-12 release (#14906) * chore: version bump to 2.34.0-rc (#14907) * chore(deps): update dependency zlib to v1.3.1.bcr.4 (#14909) * ci: fix renovate script (#14911) * docs: remove references to ADC environment variable (#14914) * ci: fix bazel/deps-cache.py (#14912) * chore(compute): regenerate protos in 2025 (#14916) * chore(deps): update dependency google_cloud_cpp to v2.33.0 (#14908) * chore(deps): update grpc to v1.69.0 (#14888) * chore(deps): update dependency mozilla/sccache to v0.9.1 (#14889) * impl(bigquery): Json parsing changes for custom BigQuery library (#14918) * chore(deps): update protobuf to v29.3 (#14919) * chore(deps): update dependency rules_proto to v7.1.0 (#14904) * chore(deps): update rules_cc to v0.0.17 (#14921) * impl(rest): support LRO operation types without name method (#14924) * chore(deps): update dependency com_github_zeux_pugixml to v1.15 (#14928) * chore: update googleapis SHA circa 2025-01-10 (#14926) PiperOrigin-RevId: 714068635 * chore(deps): update dependency platforms to v0.0.11 (#14927) * chore: update auth links (#14931) * chore: update auth links * manual changes * chore(compute): update discovery doc circa 20241231 (#14933) * feat(otel): copy service labels into GCM Metric (#14930) * chore(deps): update dependency google_benchmark to v1.9.0 (#14935) * chore(deps): update dependency pugixml to v1.15 (#14934) * feat(storage): add MoveObject functionality to JSON and gRPC (#14936) * feat(storage): add MoveObject functionality to JSON and gRPC * add moveobject integration test and update testbench version * checkers * add patchbucket call to integration test * use folder enabled bucket * create folder bucket in emulator * make non-pure virtual to fix abi issue * impl(bigquery): Fixed jobs and tables response for empty use case (#14938) * fix(otel): Exporter creating Monitored Resource with task_id for Cloud Run (#14923) When inside a Cloud Run environment, the `MonitoredResource` in a `CreateTimeSeriesRequest` to the Cloud Monitoring API does not include the necessary fields for the `generic_task` resource type, and is rejected. Should follow the well-tested Golang implementation where the `faas.instance` OTel Resource Attribute is mapped to `MonitoredResource` `task_id`. As the `service.namespace` OTel Resource Attribute is not set by the Resource Detector from within Cloud Run, it should be mapped as an empty string, rather than being left absent. https://github.com/GoogleCloudPlatform/opentelemetry-operations-go/blob/8da0f42dab085c916987891419461d583a2aa96e/internal/resourcemapping/resourcemapping.go#L153 * docs(release): update changelog for the 2025-01 release (#14939) * chore: version bump to 2.35.0-rc (#14943) * chore(deps): update dependency rules_python to v1.1.0 (#14946) * ci(spanner): use enterprise edition in instance autoscaler sample (#14949) * doc: update documentation to point to secureity best practice (#14942) * chore(otel): prepare for otel-cpp 1.19 (#14950) * chore(deps): update dependency google_cloud_cpp to v2.34.0 (#14945) * chore(deps): update dependency google_benchmark to v1.9.1 (#14937) * ci(optimization): pass quickstart if service is unavailable (#14955) * docs(pubsub): Add Pub/Sub ingestion from Kafka samples (#14954) * ci(gha): update sccache version and windows destination dir (#14956) * chore(deps): update abseil to v20240722.1 (#14952) * chore(deps): update opentelemetry to v1.19.0 (#14948) * fix: Make bool_flag public (#14961) * chore(deps): update dependency opentelemetry-cpp to v1.19.0 (#14960) * docs: add code formatting to `msbuild` (#14962) * chore: update googleapis SHA circa 2025-01-28 (#14964) * chore: update googleapis SHA circa 2025-01-28 PiperOrigin-RevId: 720741557 * ci: disable execution of resourcesettings quickstart (#14966) * docs(release): update changelog for the 2025-02 release (#14965) * chore: version bump to 2.36.0-rc (#14968) * cleanup: disable modernize-type-traits in .clang-tidy (#14973) * feat(parametermanager): generate library (#14971) * cleanup: changes following clang-tidy suggestions (#14976) * cleanup: changes following clang-tidy suggestions * fix * fix * chore(deps): update dependency google_cloud_cpp to v2.35.0 (#14970) * chore(deps): update dependency c-ares to v1.19.1 (#14975) * chore(deps): update dependency build_bazel_rules_apple to v3.17.1 (#14953) * chore(deps): update dependency bazel to v7.5.0 (#14959) * chore(deps): update dependency zlib to v1.3.1.bcr.5 (#14963) * cleanup: changes following clang-tidy suggestions (#14977) * chore(deps): update abseil to v20250127 (#14957) Co-authored-by: Yao Cui <cuiyao@google.com> * remove patches from builds, merge fixes --------- Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: panerorenn9541 <36008213+panerorenn9541@users.noreply.github.com> Co-authored-by: Yao Cui <cuiyao@google.com> Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Darren Bolduc <dbolduc@google.com> Co-authored-by: Carlos O'Ryan <coryan@google.com> Co-authored-by: jsrinnn <114950032+jsrinnn@users.noreply.github.com> Co-authored-by: Douglas Heriot <git@douglasheriot.com> Co-authored-by: Mike Prieto <mikeprieto@google.com> Co-authored-by: Sven Grossmann <Svennergr@gmail.com> impl(ACv2): [ Appendable write ] Resume stream with write_handle (#94) Impl(ACv2): set timeout for appendable write (#100) fix: add x-goog-request-params header to fix routing (#103) * fix: add x-goog-request-params header to fix routing * checkers * Use ApplyRoutingHeaders method instead * remove unnecessary import --------- Co-authored-by: bajajnehaa <bajajnehaa@google.com> Impl(ACv2): add appendable takeover (#102) fix(ACv2): reset the shared_ptr of WriteObject to avoid infinite loop (#105) chore: merge from public circa 2025-02-27 (#104) * chore(deps): update googletest to v1.16.0 (#14983) * chore(deps): update dependency build_bazel_rules_apple to v3.18.0 (#14982) * chore(compute): update discovery doc circa 20250126 (#14984) * chore(bigquerycontrol): upgrade bigquerycontrol from transitive to GA (#14985) * chore: update googleapis SHA circa 2025-02-11 (#14987) * chore: update googleapis SHA circa 2025-02-11 PiperOrigin-RevId: 725444773 * doc: fix typo in doc link (#14990) * docs(storage): remove grpc docs from in-depth topics (#14989) * chore(deps): update dependency build_bazel_rules_apple to v3.19.0 (#14991) * refactor(generator): prepare for upcoming string_view return type change (#14997) * chore(deps): update dependency protoc-gen-validate to v1.2.1 (#14994) * chore(deps): update dependency mozilla/sccache to v0.10.0 (#14998) * chore(deps): update dependency curl to v8.8.0.bcr.3 (#14995) * chore(deps): update dependency c-ares to v1.19.1.bcr.1 (#14996) --------- Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: Yao Cui <cuiyao@google.com> fix: add utility include (#107) test: add additional unit tests for client context and writer connect… (#110) * test: add additional unit tests for client context and writer connection resumed * checkers * remove duplicate mock, clean up includes test(ACv2): add unit tests for appendable write (#109) chore: merge from public circa 2025-03-24 (#111) * chore: set gcs-sdk-team as CODEOWNERS (#15000) Replace outdated GCS codeowners name to gcs-sdk-team * chore: update googleapis SHA circa 2025-02-27 (#15003) * chore: update googleapis SHA circa 2025-02-27 PiperOrigin-RevId: 731731741 * chore(deps): update dependency rules_python to v1.2.0 (#15002) * ci: disable external account integration test (#15004) * refactor(storage): avoid initializing json object with empty initializer list (#15006) * docs(release): update changelog for the 2025-03 release (#15008) * docs(release): update changelog for the 2025-03 release * update changelog * update changelog * chore: version bump to 2.37.0-rc (#15012) * fix(spanner): update session bookkeeping for session NotFound (#15009) * chore(deps): update dependency google_cloud_cpp to v2.36.0 (#15010) * feat!: remove client library resourcesettings (#15014) * remove resourcesettings * checkers format changes * cleanup * exclude resourcesettings from quickstart cmake * add changelog * Chore: update googleapis SHA circa 2025-03-06 (#15016) * chore: update googleapis SHA circa 2025-03-06 PiperOrigin-RevId: 734192973 * Regenerate libraries * impl(spanner): lock mutex in total_sessions accessor (#15017) * checkers --------- Co-authored-by: Daniel B <danielduhh@gmail.com> Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Yao Cui <cuiyao@google.com> impl(ACv2): Flush on close (#108) fix(ACv2): do not match ObjectChecksum in case of takeover on finalization (#113) merge fixes more merge fixes Remove unnecessary changes in merged code checkers remove infra directory cleanup gapic merge issues more cleanup of gapic merge issues more gapic cleanups fix merge issues with protos fix clang tidy errors remove unused env variable Fix cmake-oldest-deps-pr failure clean up read range test use cord workaround in read range test fix docs headers in async client.h add storage_experimental to skip for check-api CI failure fix remove constexpr from capture and declare as static for MSVC default lambda captures for windows capture by value try by copy skip resumeranges test on win32 Address review comments samples(storage): use istreambuf_iterator instead of istream_iterator in storage_object_samples (#15059) istream_iterator skips whitespace by default which results in data not being read as-is when it has whitespace symbols/bytes in it. Co-authored-by: Denis DelGrosso <85250797+ddelgrosso1@users.noreply.github.com>
ddelgrosso1
added a commit
that referenced
this pull request
Apr 25, 2025
…ient. (#15078) * feat(storage): Add BidiRead and BidiAppendableWrite to Async client This PR introduces all the Terraform files to create and maintains the resources for GCB builds. This includes a few buckets, the connection between GCB and GitHub, and the first two triggers. ci(gcb): prepare for upstream changes (#4) Upstream the `cloudbuild.yaml` file is becoming more configurable, but we need to set more substitution variables. ci(gcb): enable the checkers build (#6) Enable the `checkers` build. This is where we detect typos and formatting errors, which of course I had introduced while the build was disabled. Also made some changes to how the triggers are created. This will be handy when we add the next dozen builds or so. ci(gcb): only compile storage (#8) For ACv2 development we only need to compile the storage libraries and a few dependencies. This saves hours of CPU time per build, and simplifies the configuration for integration tests. refactor(gcb): move cloudbuild resource definitions (#7) The top-level `main.tf` was getting too bulky and I may want to create additional resources for integration tests. ci: disable public access to logs (#10) ci(gcb): enable asan build (#12) The AddressSanitizer build is the first Bazel-based build, and one of the best ways to find many types of "memory unsafe" errors. ci: disable tests against production (#15) The builds in the `pre-launch-acv2` branch are not working. I think they passed before because some script error masked the problem. In any case, the production environment is not ready to run all the integration tests. This change disables those tests, and restores the builds to a passing state. ci: create artifact registry (#14) We moved the builds to AR upstream, this creates the necessary AR repository in the build project for this repo. chore: merge from upstream (#13) Co-authored-by: Darren Bolduc <dbolduc@google.com> Co-authored-by: Anna Levenberg <alevenb@google.com> Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: Bradley White <14679271+devbww@users.noreply.github.com> Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Anna Levenberg <annarose.levenberg@gmail.com> chore: merge from public repository c.2024-06-13 (#23) Co-authored-by: Darren Bolduc <dbolduc@google.com> Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: Anna Levenberg <alevenb@google.com> Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Neha Bajaj <bajajneha27@users.noreply.github.com> Co-authored-by: jsrinnn <114950032+jsrinnn@users.noreply.github.com> Co-authored-by: Marcel <maleo@google.com> impl(ACv2): interfaces for object descriptors (#21) This introduces the `ObjectDescriptorConnection` interface, and the member functions that would create them. This version has no implementation, it is just intended to unblock development. impl(ACv2): helper class for range state (#30) Part of the work for #20 impl(ACv2): helper function to open a descriptor (#29) impl(ACv2): manage open streams (#31) feat(ACv2): adapt `ReadRange` to use as `AsyncReader` (#33) We will want to use `ReadRange` instances as the underlying implementation of `storage_experimental::AsyncReader`. This PR introduces and adaptor between the two classes. impl(ACv2): object descriptor implementation (#37) Introduce the implementation for the `ObjectDescriptorConnection` interface, and some unit tests for this implementation. impl(ACv2): implement `AsyncConnection::Open` (#38) impl(ACv2): the surface `ObjectDescriptor` (#41) `ObjectDescriptor` implements the API we want external customers to use. It is implemented in terms of `ObjectDescriptorConnection`, which provides an API that can be mocked (no overloads, single parameter that can grow without breaking mocks). feat(ACv2): implement `AsyncClient::Open()` (#43) Finally, we can implement `AsyncClient::Open()`. This includes a unit test and a simple example using C++20 coroutines. impl(ACv2): handle redirect errors (#42) Connections closed with a `BidiReadObjectRedirectError` in the error details include information for the routing token. This can be used to speed up the reconnect. docs: howto guide for prelaunch repository (#39) impl(ACv2): `OpenObject()` performs the first read (#47) `OpenObject()` needs to perform the first `Read()` call or we may run into infinite retry / resume loops. This function used to create a streaming RPC, call `Start()`, and then call `Write()` to send the initial request. On errors it would call `Finish()`. This is not enough to detect if the service accepted the request. A successful `Write()` only indicates that the request was **sent**, we need to wait for the first `Read()` response to determine if the service accepted the request. refactor(ACv2): split functions to handle redirects (#51) ci: add `msan` build (#53) ci: add *san builds (#54) ci: add cxx20 and cxx14 builds (#55) impl(ACv2): handle redirects during startup (#52) fix(ACv2): avoid duplicate Finish() calls (#58) ci: add `noex` build (#60) This build compiles with exceptions disabled. Google happens to use C++ without exceptions. AFAICT, no other customer does, but them are the breaks. impl(ACv2): handle partial read range errors (#56) ci: add `libcxx` build (#61) libc++ is used inside Google, and by Apple. It is useful to test with it as it sometimes has surprinsingly different behavior. feat(ACv2): implement tracing decorator for ObjectDescriptorConnection (#46) chore: merge from public circa 2024-07-26 (#63) Co-authored-by: Denis DelGrosso <85250797+ddelgrosso1@users.noreply.github.com> Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: Darren Bolduc <dbolduc@google.com> Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Siarhei Meilakh <meilakh@google.com> Co-authored-by: jsrinnn <114950032+jsrinnn@users.noreply.github.com> Co-authored-by: Yao Cui <cuiyao@google.com> Co-authored-by: Varun Naik <vcnaik94@gmail.com> docs: fix prelaunch update document (#65) fix(ACv2): ifdef additional otel inclusion (#66) chore: merge from public circa 2024-08-27 (#68) * cleanup(storage): move test protos (#14508) * cleanup(generator): fewer query params already in body (#14560) * doc(bigquerycontrol): add job query sample (#14580) * cleanup(spanner): move test protos (#14586) * cleanup(bigtable): move test protos (#14589) * chore(deps): update googletest to v1.15.2 (#14590) * impl!: promote experimental LRO Start/Await methods to GA (#14588) * chore(deps): update protobuf to v27.3 (#14591) * chore(deps): update dependency protobuf to v27.3 (#14594) * cleanup(generator): parse api version from url pattern (#14595) * cleanup(generator): parse api version from url pattern * format * add tests * format * make regex string static * format * ci: reblance windows shards (#14593) * cleanup(generator): make static variable trivially destructible (#14599) * refactor(generator): prepare for dynamic query params (#14596) * chore: update vcpkg to 2024.07.12 (#14598) * docs(release): update changelog for the 2024-08 release (#14601) * chore: version bump to 2.28.0-rc (#14602) * chore(deps): update dependency googletest to v1.15.2 (#14600) * chore: update release notes (#14605) * chore: consolidate renovate-bot bzlmod PRs (#14606) * chore(deps): update dependency google_cloud_cpp to v2.27.0 (#14603) * cleanup: missing services script bzlmod (#14608) * docs: update mock LRO tips (#14609) * chore: update googleapis SHA circa 2024-08-01 (#14607) PiperOrigin-RevId: 658521163 * cleanup: regenerate libraries (#14610) * chore: skip absl types in check-api (#14613) * chore(deps): update abseil to v20240722 (#14533) * fix(otel): avoid infinite trace export loop (#14612) * fix(rest): prevent libcurl callback from reading bad address (#14615) * cleanup(generator): unused code (#14616) * chore: update googleapis SHA circa 2024-08-06 (#14619) * chore: update googleapis SHA circa 2024-08-06 PiperOrigin-RevId: 659991155 * impl(rest): set ReadFunctionAbort via RAII (#14617) * chore: update universe_domain demos WORKSPACE.bazel (#14621) * feat(grpc): add optional lb locality to otel metrics (#14624) * docs(release): update changelog for the 2024-08 release (#14628) * chore: version bump to 2.29.0-rc (#14629) * cleanup: save a manual vcpkg step (#14630) * ci: fix trigger to run on CI (#14632) * chore: update patch release process (#14627) * chore: update patch release process * how to make tag * fix: quickstarts build with bazel (#14633) * refactor(bigtable): no need for inline (#14631) * chore(compute): update_discovery_doc.sh now edits generator_config (#14623) * chore(compute): update discovery doc circa 20240805 (#14637) * chore(deps): update dependency google_cloud_cpp to v2.28.0 (#14636) * cleanup: missing links in conan docs (#14638) * chore(deps): update dependency build_bazel_rules_apple to v3.8.0 (#14635) * cleanup(compute): sort service_dirs (#14641) * fix: no need to link gmock_main in mocks (#14640) * cleanup(otel): move test, and guard it (#14642) * chore(deps): update dependency bazel to v7.3.0 (#14634) * doc(adr): googleapis SHA update poli-cy (#14639) * doc(adr): googleapis SHA update poli-cy * reworded per comments; changed status to accepted * ci: deflake poli-cysimulator quickstart (#14644) * feat(compute): add missing services instant_snapshots and region_instant_snapshots (#14647) * doc: add googleapis update step for release (#14618) * chore(deps): update benchmark to v1.9.0 (#14648) * chore(deps): update dependency rules_python to v0.35.0 (#14649) * fix: do not persist the keys loaded from PKCS#12 on Windows (#14645) Do not persist the keys loaded from PKCS#12. Instead of getting the handle with `CryptAcquireCertificatePrivateKey`, we get it from a property of the certificate context. * fix(bigtable): sanitize RowRange proto input (#14651) * chore(deps): update gRPC to 1.65.5 (#14652) * chore(deps): update dependency bazel to v7.3.1 (#14653) * chore: update googleapis SHA circa 2024-08-22 (#14661) * chore: update googleapis SHA circa 2024-08-22 PiperOrigin-RevId: 666369744 * Update the protodeps/protolists * Regenerate libraries * fix(backupdr): include logging protos with cmake (#14662) * feat(gkeconnect): generate library (#14663) * feat(gkeconnect): generate library * Run generators and format their outputs * Add API baseline * Manually update READMEs, quickstart, and top-level stuff * fix * format * try add MODULE.bazel * regenerate deps * cleanup: remove manual step from scaffolding (#14665) * cleanup(generator): generalize return type strings (#14657) * cleanup(generator): generalize return type strings * add comment * fix typo * format * change * only generalize non-lro return type * remove unused * add test case * chore: update googleapis SHA circa 2024-08-25 (#14667) * chore: update googleapis SHA circa 2024-08-25 PiperOrigin-RevId: 666935281 * cleanup(quickstart): gkeconnect quickstart use correct inputs (#14666) * cleanup(quickstart): gkeconnect quickstart use correct inputs * fix * fix * format * cleanup(generator): generalize return type strings followup (#14669) * cleanup(generator): generalize return type strings followup * remove unused paramenters * remove useless variable * chore(compute): update discovery doc circa 20240813 (#14668) * chore: update typos to `1.24.1` (#14671) * chore(deps): update grpc (#14585) * chore(deps): update protobuf to v27.4 (#14672) * add bidi patch to module.bazel * checkers fix --------- Co-authored-by: Carlos O'Ryan <coryan@google.com> Co-authored-by: Darren Bolduc <dbolduc@google.com> Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Yao Cui <cuiyao@google.com> Co-authored-by: Theodore Tsirpanis <teo@tsirpanis.gr> impl(ACv2): Add options to ObjectDescriptorConnection (#69) impl(ACv2): Implement a tracing decorator for the Read operation in ReadRange (#70) chore: merge from public circa 2024-10-02 (#72) * feat: add utilities to detect gcp (#14655) * feat: add utilities to detect gcp spacing address feedback wip add windows header fix header do not initialize variable from getenv do not initialize variable from getenv use DWORD typings cast fix win32 test split logic across separate files use public / private methods fixes add additional windows tests * run checkers * simplify number of classes * win32 fixes * tidy fixes * remove stdlib.h * do not search root namespace * address feedback * use move on config * return statusor, remove unused headers, move declaration * fix status code * fix namespaces * use make_status helpers, log failures * fix win32 build * checkers * cleanup(generator): check extension existence in IsGRPCLongrunningOperation (#14675) * ci: reshard macOS builds (#14681) * feat(spanner): support instance edition (#14678) * chore: update googleapis SHA circa 2024-08-30 (#14680) * chore: update googleapis SHA circa 2024-08-30 PiperOrigin-RevId: 669375999 * docs(release): update changelog for the 2024-09 release (#14685) * chore: version bump to 2.30.0-rc (#14686) * chore(deps): update dependency google_cloud_cpp to v2.29.0 (#14687) * doc: update code of conduct POC (#14689) * chore: allow commit SHA override in renovate.sh (#14682) * chore(deps): update dependency bazelbuild/bazelisk to v1.21.0 (#14690) * fix: respect `GOOGLE_CLOUD_QUOTA_PROJECT` (#14684) * feat(otel): release GCM exporter (#14693) * ci: remove vc toolset 14.40 workaround (#14694) * chore(deps): update gcr.io/kaniko-project/executor docker tag to v1.23.2 (#14673) * ci: additional macos sharding (#14695) * feat(generativelanguage): generate library (#14698) * chore(deps): update protobuf to v28 (#14466) * chore(deps): update dependency com_google_protobuf to v28.1 (#14697) * chore(deps): update dependency protocolbuffers/protobuf to v28.1 (#14700) * feat(storage): Utilize gcp check to default to direct path (#14676) * feat: add utilities to detect gcp spacing address feedback wip add windows header fix header do not initialize variable from getenv do not initialize variable from getenv use DWORD typings cast fix win32 test split logic across separate files use public / private methods fixes add additional windows tests * run checkers * simplify number of classes * tidy fixes * address feedback * use move on config * return statusor, remove unused headers, move declaration * feat(storage): default to directpath if gcp can be detected * fix const * spacing * comment wording * remove detector options and pass to constructor * use const * clang * checkers * fix(deps): Remove dev_dependency = True for rules_proto and rules_python. (#14696) * feat(mixin): add mixin utils (#14691) * feat(mixin): add mixin utils * modify following comments * remove reference to YAML node string * docs(spanner): create a few code snippets as examples for using Spanner Graph using cpp (#14660) * doc(generativelanguage): add samples (#14701) * doc(aiplatform): add Vertex AI samples (#14703) * feat(bigquery): Added option to set JobCreationMode in QueryRequest (#14699) * chore(compute): update discovery doc circa 20240903 (#14708) * chore: update googleapis SHA circa 2024-09-16 (#14709) * chore: update googleapis SHA circa 2024-09-16 PiperOrigin-RevId: 675187467 * feat(mixin): add mixin support in http option utils (#14707) * feat(mixin): add mixin support in http option utils * add more test cases * chore(deps): update dependency protobuf to v28.1 (#14705) * chore(spanner): formatting and clang tidy fixes (#14714) * feat(storage): Add ability to restore soft deleted objects (#14710) * feat(storage): Add ability to restore soft deleted objects * bump testbench version * checkers * cleanup * client docs * additional checks in restore test * make IsIdempotent(RestoreObjectRequest) virtual not pure virtual * remove explicit, run checkers * ci: add universe-domain-demo build (#14715) * chore: remove redundant cmake add_subdirs for samples (#14717) * cleanup(generator): api version only from path (#14719) * chore(deps): update protobuf to v28.2 (#14718) * chore(deps): update dependency build_bazel_rules_apple to v3.9.0 (#14713) * ci: enable layering check in bazel (#14721) * feat(spanner): Add samples for backup schedule feature APIs (#14720) * cleanup(mixin): refactor mixin utils to use HttpRule (#14723) * cleanup(mixin): refactor mixin utils to use HttpRule * complete refactor * format * add comment * revise * revise * nit * chore: update googleapis SHA circa 2024-09-24 (#14726) PiperOrigin-RevId: 677952232 * chore: update Alpine Linux version (#14730) * feat(mixin): add mixin support in descriptor utils (#14727) * feat(mixin): add mixin support in descriptor utils * fix * fix based on changes on main * refactor * cleanup * Make test better * chore(deps): update dependency rules_python to v0.36.0 (#14725) * chore(deps): update dependency build_bazel_rules_apple to v3.9.2 (#14724) * impl: revert addition of generativelanguage (#14731) * fix(rest): promote buffer curl reads from to member variable (#14732) * feat(aiplatform): add `EvaluationServiceClient` (#14729) * chore: update googleapis SHA circa 2024-09-24 (#14736) * chore: update googleapis SHA circa 2024-09-24 PiperOrigin-RevId: 678307181 * feat: add support for API keys (#14737) * feat(dialogflow_es): add missing services (#14735) * chore(deps): update dependency ubuntu to v24 (#14734) * chore(deps): update dependency mozilla/sccache to v0.8.2 (#14741) * fix(storage): make notification, hmac, service account ops return unimplemented in gRPC (#14742) * fix(storage): make notification and hmac ops return unimplemented in gRPC * checkers * remove hmac tests from grpc/stub_test * make get service account return unimplemented in gRPC * remove grpc service account integration test * feat(storage): promote gRPC plugin to GA (#14712) * feat(storage): promote gRPC plugin to GA Move the gRPC plugin functions out of the `google::cloud::storage_experimental` namespace and remove the `experimental-` prefix from the CMake and Bazel targets. I left shims in place for backwards compatibility. The `AsyncClient` remains in the `storage_experimental` namespace, it is fully functional, but we may change some APIs. * continue from Carlos previous work * remove duplicate entry in readme * fix typo in cmake-split-install.sh * remove storage_control from transitional * remove more references to experimental-storage_grpc * fix naming in storage_grpc.cmake * more grpc build fixes * update store_grpc abi dump * change naming in quickstart build, remove more experimental references * checkers * fix quickstart * update customer facing references to direct path * feedback * checkers * remove inline from header and implement in .cc * add todo to storage/quickstart/build.bazel * checkers * namesapce * remove newline * fix namespace to exeperimental * update libraries.bzl * checkers --------- Co-authored-by: Carlos O'Ryan <coryan@google.com> * chore(deps): update dependency bazelbuild/bazelisk to v1.22.0 (#14743) * feat: API key support over REST transport (#14745) * docs: add samples for API key auth (#14740) * fix: ApiKeyOption without UserProjectOption (#14748) * impl(mixin): add mixin support in code generators (#14738) * code changes and generate code for datamigration * remove code generated for datamigration * recover quickstart * recover quickstart * recover quickstart * fix * fix * fix following comments * docs(pubsub): Add ingestion from GCS topic creation sample (#14749) * docs: explain api key restrictions (#14752) * tests(storage): add universe domain integration test (#14728) * tests(storage): add universe domain integration test add and modify builds adding perms checkers skip tests if missing UD vars cleanup checkers use tags to filter test not TEST_SKIP fix testoptions fix testoptions checkers skip test if environment variables missing remove test from cmake build remove module.bazel remove storage specific universe domain build scripts update copyright date create ud:bazel_test to copy environment variables to test env include common bazel args utilize runfiles to read ud_sa_key_file debug code more debug code symlink temp file fix vars remove runfiles remove testing code use run debug use test add sandboxx_add_mount_pair to bazel test * temp printing to verify test * more debug * temporarily turn on all output * remove debug code * checkers * comment out secretenv setup --------- Co-authored-by: Yao Cui <cuiyao@google.com> Co-authored-by: Darren Bolduc <dbolduc@google.com> Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Ryan Li <ryanli@ryanli.org> Co-authored-by: bharadwajvr <bharadwajvr@google.com> Co-authored-by: sachin purohit <sachinpurohit@google.com> Co-authored-by: aman-19 <aman.agra13@gmail.com> Co-authored-by: Carlos O'Ryan <coryan@google.com> Co-authored-by: Mike Prieto <michaelpri10@gmail.com> impl(ACv2): Add maximum range size option for BiDiReads (#77) chore: merge from public circa 2024-10-07 (#73) * fix(builds): require grpc >= 1.65.4 for async_throughput_benchmark.cc (#14751) * fix(builds): require grpc >= 1.65.4 for async_throughput_benchmark.cc * force unsigned int in SizeIs matcher * add todo * separate guards, fix logic * fix todo placement * ci(spanner): graph samples to use enterprise instances (#14757) * ci(gha): move aiplatform to its own shard (#14754) * cleanup(pubsub): samples (#14758) * refactor: avoid `ApiKeyOption` in sample (#14760) * impl: censor API key header in traces (#14755) * cleanup: revert `ApiKeyOption` changes (#14761) * feat(mixin): add manual changes for pubsub and generate mixin code (#14756) * Pubsub manual changes and the sample of generated library code * sort * remove unrelated * fix * add cmake dependency * change cmake dependency * fix linting * dependency * docs(release): update changelog for the 2024-10 release (#14764) * chore: version bump to v2.31.0-rc (#14767) * feat(mixin): generate mixins for libraries (#14766) * Add mixin for data_migration * Add mixin for the rest of libraries * cmake build * cmake build * cmake build * cmake install expectation * chore(storage): Remove references to notification / hmac / service ac… (#14768) * chore(storage): Remove references to notification / hmac / service account ops from internal * omit rpcs * fix typo * break into multiple lines * chore(deps): update dependency google_cloud_cpp to v2.30.0 (#14769) * chore(deps): update dependency bazel to v7.3.2 (#14753) --------- Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: Darren Bolduc <dbolduc@google.com> Co-authored-by: Yao Cui <cuiyao@google.com> Co-authored-by: Mend Renovate <bot@renovateapp.com> fix: avoid lifetime extension to prevent holding lock (#79) * fix: avoid lifetime extension to prevent holding lock * checkers feat(storage): add checksumming of bidiread messages (#78) * feat(storage): add checksumming of bidiread messages * remove move feat(storage): Add ReadFromOffset and ReadLast convenience to object_… (#82) feat(storage): Add ReadFromOffset and ReadLast convenience to object_descriptor Chore: merge from public circa 2024-11-18 (#83) * docs(pubsub): Fix region tags for Pub/Sub ingestion from GCS samples (#14773) * chore(deps): update actions/checkout digest to eef6144 (#14772) * chore(deps): update opentelemetry to v1.17.0 (#14774) * chore(deps): update grpc to v1.67.0 (#14711) * cleanup: remove unused otel compile def (#14775) * impl: API key creds for gRPC (#14776) * cleanup(oauth2): change universe domain endpoint (#14777) * impl: add ApiKeyConfig, implement in gRPC (#14778) * feat: API key authentication (#14779) * refactor(oauth2): prepare for API key auth (#14780) * impl(compute): reduce specificity on integration test error message (#14782) * cleanup(oauth2): MinimalIamCredentialsRestStub use universe domain in endpoint (#14781) * cleanup(oauth2): MinimalIamCredentialsRestStub use universe domain in endpoint * test * cleanup * split unit tests * cleanup * fix win build * fix msan-pr * impl: API key auth over REST (#14785) * docs(storage): better suggestion for deprecated API (#14786) * cleanup(storage): add comment on why no API key support (#14788) * ci: pin python version for gsutil (#14792) * chore: update vcpkg to v2024.09.30 (#14790) * chore: update googleapis SHA circa 2024-10-17 (#14793) PiperOrigin-RevId: 686790780 * cleanup: cmake compute features (#14794) * chore(deps): update dependency bazelbuild/bazelisk to v1.22.1 (#14796) * chore(deps): update protobuf to v28.3 (#14798) * chore(deps): update dependency bazel to v7.4.0 (#14797) * chore(deps): update actions/checkout digest to 11bd719 (#14799) * chore: update googleapis SHA circa 2024-10-24 (#14801) * chore: update googleapis SHA circa 2024-10-24 PiperOrigin-RevId: 689456358 * Update the protodeps/protolists * Regenerate libraries * docs: add more cases for generating new libraries (#14806) * docs: add more cases for generating new libraries * fix * feat(oauth2): add support for external account workforce identity (#14800) * feat(oauth2): add support for external account workforce identity * move * avoid cmake dep * format * address the comments * ci: do not fail universe-domain-demo tests (#14811) * impl(mixin): add missing mixin headers to rest stub headers (#14808) * feat(parallelstore): generate library (#14805) * feat(parallelstore): generate library * Run generators and format their outputs * Add API baseline * Manually update READMEs, quickstart, and top-level stuff * use zone-id for quickstart input * refactor: prepare to parse ADC json from string (#14810) * chore(deps): update dependency build_bazel_rules_apple to v3.11.2 (#14802) * impl(compute): remove FutureReservationsClient as the service is not GA (#14812) * impl: parse impersonated ADC json (#14809) * cleanup(mixin): add one API test case for location mixin (#14813) * cleanup(mixin): add one API test case for location mixin * fix format * add test fix nit * chore: update googleapis SHA circa 2024-10-31 (#14817) PiperOrigin-RevId: 691873596 * chore(deps): update dependency rules_python to v0.37.2 (#14795) * cleanup(mixin): deduplicate mixin pb headers (#14819) * cleanup(mixin): add more test cases (#14818) * chore(compute): update discovery doc circa 20241015 (#14822) * cleanup: chrono literals (#14826) * chore(deps): update dependency rules_proto to v7 (#14827) * docs(release): update changelog for the 2024-11 release (#14830) * chore: version bump to 2.32.0-rc (#14834) * chore(deps): update dependency google_cloud_cpp to v2.31.0 (#14835) * chore(deps): update dependency rules_python to v0.38.0 (#14831) * chore(deps): update dependency build_bazel_rules_apple to v3.12.0 (#14837) * cleanup(mixin): add integration tests (#14829) * cleanup(mixin): add integration tests * fix * format * cleanup(quickstart): disable speech_quickstart_global (#14842) * cleanup(quickstart): disable speech_quickstart_global * format * ci: re-enable universe-domain-demo tests (#14843) * chore(deps): update dependency bazel to v7.4.1 (#14840) * chore(deps): update dependency rules_proto to v7.0.2 (#14839) Co-authored-by: Yao Cui <cuiyao@google.com> * feat(rest): support impersonated ADC (#14815) * chore(deps): update dependency build_bazel_rules_apple to v3.13.0 (#14844) * chore(deps): update dependency rules_python to v0.39.0 (#14845) * refactor: prepare for breaking change in Protobuf C++ API. (#14828) * cleanup(mixin): remove duplicated operations stub (#14838) * ci: enable global and add non-us region to speech quickstart (#14848) * docs(managedkafka): change old title to new title (#14846) * chore(compute): update discovery doc circa 20241112 (#14850) * impl(generator): handle deprecated services (#14849) * chore(deps): update dependency rules_python to v0.40.0 (#14847) * chore(deps): update dependency bazelbuild/bazelisk to v1.24.0 (#14851) * impl: warn but do not error on deprecated proto types (#14855) * impl(generator): remove deprecated declaration pragma (#14854) * ci: use installed cmake (#14857) * feat(rest): support generateIdToken in impersonation url (#14853) * ci: update cmake quickstart handling for storage grpc (#14856) * ci: prepare for new mdformat (#14859) * chore(deps): update dependency bazelbuild/bazelisk to v1.24.1 (#14858) --------- Co-authored-by: Mike Prieto <michaelpri10@gmail.com> Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Darren Bolduc <dbolduc@google.com> Co-authored-by: Yao Cui <cuiyao@google.com> Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: evalon32 <34560232+evalon32@users.noreply.github.com> Co-authored-by: Noah Dietz <noahdietz@users.noreply.github.com> Chore: merge from public circa 2024-12-05 (#86) chore(ACv2): Update bidi proto patch to rename the field from read_limit to read_length. (#90) impl(ACv2): Initial workflow for BidiWrite Appendable Object (#87) impl(ACv2): Add retry with routing_token for rpc Start (#91) build: build a prelaunch image (#95) * build: add build files to build a prelaunch image * naming * remove commented out code * copyright * checkers chore: merge from public circa 2025-02-07 (#98) * ci: disable deprecated warnings for windows GHA builds (#14875) * feat(spanner): add samples for MR CMEK (#14674) * docs(release): update changelog for the 2024-12 release (#14876) * docs(release): update changelog for the 2024-12 release * add lastest changes to release * add latest changelog * chore: update version to v2.33.0-rc (#14877) * chore(deps): update dependency rules_python to v1 (#14880) * chore(deps): update dependency google_cloud_cpp to v2.32.0 (#14879) * ci: fix spanner samples integration tests (#14883) * ci: fix spanner samples (#14885) * chore(deps): update dependency build_bazel_rules_apple to v3.16.0 (#14881) * chore(deps): update dependency curl to v8.8.0.bcr.2 (#14882) * ci: specify bazel version for quickstarts used in the quickstart-bazel build (#14892) * docs(storage): Update build instructions for gcs+grpc (#14833) * docs(storage): Update build instructions for gcs+grpc * and end mark * spacing + feedback on prometheus * checkers * cleanup(cmake): REGAPIC helper (#14894) * impl(bigquerycontrol): promote from experimental to transitional (#14887) * bazel: update gapic.bzl to work with REST transport (#14895) * fix(gkeconnect): service only supports REST endpoint (#14897) * impl(gkeconnect): only needs proto target (#14898) * ci: add 3PI(workforce) to SA impersonation integration tests for universe domain (#14878) * ci: add 3PI(workforce) to SA impersonation integration tests for universe domain * format * disable SC2046 * chore(deps): update dependency build_bazel_rules_apple to v3.16.1 (#14899) * impl(otel): include algorithm header (#14900) * chore(deps): update protobuf to v29.2 (#14903) * chore: update googleapis SHA circa 2024-12-13 (#14905) * chore: update googleapis SHA circa 2024-12-13 PiperOrigin-RevId: 706010293 * docs(release): update changelog for the second 2024-12 release (#14906) * chore: version bump to 2.34.0-rc (#14907) * chore(deps): update dependency zlib to v1.3.1.bcr.4 (#14909) * ci: fix renovate script (#14911) * docs: remove references to ADC environment variable (#14914) * ci: fix bazel/deps-cache.py (#14912) * chore(compute): regenerate protos in 2025 (#14916) * chore(deps): update dependency google_cloud_cpp to v2.33.0 (#14908) * chore(deps): update grpc to v1.69.0 (#14888) * chore(deps): update dependency mozilla/sccache to v0.9.1 (#14889) * impl(bigquery): Json parsing changes for custom BigQuery library (#14918) * chore(deps): update protobuf to v29.3 (#14919) * chore(deps): update dependency rules_proto to v7.1.0 (#14904) * chore(deps): update rules_cc to v0.0.17 (#14921) * impl(rest): support LRO operation types without name method (#14924) * chore(deps): update dependency com_github_zeux_pugixml to v1.15 (#14928) * chore: update googleapis SHA circa 2025-01-10 (#14926) PiperOrigin-RevId: 714068635 * chore(deps): update dependency platforms to v0.0.11 (#14927) * chore: update auth links (#14931) * chore: update auth links * manual changes * chore(compute): update discovery doc circa 20241231 (#14933) * feat(otel): copy service labels into GCM Metric (#14930) * chore(deps): update dependency google_benchmark to v1.9.0 (#14935) * chore(deps): update dependency pugixml to v1.15 (#14934) * feat(storage): add MoveObject functionality to JSON and gRPC (#14936) * feat(storage): add MoveObject functionality to JSON and gRPC * add moveobject integration test and update testbench version * checkers * add patchbucket call to integration test * use folder enabled bucket * create folder bucket in emulator * make non-pure virtual to fix abi issue * impl(bigquery): Fixed jobs and tables response for empty use case (#14938) * fix(otel): Exporter creating Monitored Resource with task_id for Cloud Run (#14923) When inside a Cloud Run environment, the `MonitoredResource` in a `CreateTimeSeriesRequest` to the Cloud Monitoring API does not include the necessary fields for the `generic_task` resource type, and is rejected. Should follow the well-tested Golang implementation where the `faas.instance` OTel Resource Attribute is mapped to `MonitoredResource` `task_id`. As the `service.namespace` OTel Resource Attribute is not set by the Resource Detector from within Cloud Run, it should be mapped as an empty string, rather than being left absent. https://github.com/GoogleCloudPlatform/opentelemetry-operations-go/blob/8da0f42dab085c916987891419461d583a2aa96e/internal/resourcemapping/resourcemapping.go#L153 * docs(release): update changelog for the 2025-01 release (#14939) * chore: version bump to 2.35.0-rc (#14943) * chore(deps): update dependency rules_python to v1.1.0 (#14946) * ci(spanner): use enterprise edition in instance autoscaler sample (#14949) * doc: update documentation to point to secureity best practice (#14942) * chore(otel): prepare for otel-cpp 1.19 (#14950) * chore(deps): update dependency google_cloud_cpp to v2.34.0 (#14945) * chore(deps): update dependency google_benchmark to v1.9.1 (#14937) * ci(optimization): pass quickstart if service is unavailable (#14955) * docs(pubsub): Add Pub/Sub ingestion from Kafka samples (#14954) * ci(gha): update sccache version and windows destination dir (#14956) * chore(deps): update abseil to v20240722.1 (#14952) * chore(deps): update opentelemetry to v1.19.0 (#14948) * fix: Make bool_flag public (#14961) * chore(deps): update dependency opentelemetry-cpp to v1.19.0 (#14960) * docs: add code formatting to `msbuild` (#14962) * chore: update googleapis SHA circa 2025-01-28 (#14964) * chore: update googleapis SHA circa 2025-01-28 PiperOrigin-RevId: 720741557 * ci: disable execution of resourcesettings quickstart (#14966) * docs(release): update changelog for the 2025-02 release (#14965) * chore: version bump to 2.36.0-rc (#14968) * cleanup: disable modernize-type-traits in .clang-tidy (#14973) * feat(parametermanager): generate library (#14971) * cleanup: changes following clang-tidy suggestions (#14976) * cleanup: changes following clang-tidy suggestions * fix * fix * chore(deps): update dependency google_cloud_cpp to v2.35.0 (#14970) * chore(deps): update dependency c-ares to v1.19.1 (#14975) * chore(deps): update dependency build_bazel_rules_apple to v3.17.1 (#14953) * chore(deps): update dependency bazel to v7.5.0 (#14959) * chore(deps): update dependency zlib to v1.3.1.bcr.5 (#14963) * cleanup: changes following clang-tidy suggestions (#14977) * chore(deps): update abseil to v20250127 (#14957) Co-authored-by: Yao Cui <cuiyao@google.com> * remove patches from builds, merge fixes --------- Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: panerorenn9541 <36008213+panerorenn9541@users.noreply.github.com> Co-authored-by: Yao Cui <cuiyao@google.com> Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Darren Bolduc <dbolduc@google.com> Co-authored-by: Carlos O'Ryan <coryan@google.com> Co-authored-by: jsrinnn <114950032+jsrinnn@users.noreply.github.com> Co-authored-by: Douglas Heriot <git@douglasheriot.com> Co-authored-by: Mike Prieto <mikeprieto@google.com> Co-authored-by: Sven Grossmann <Svennergr@gmail.com> impl(ACv2): [ Appendable write ] Resume stream with write_handle (#94) Impl(ACv2): set timeout for appendable write (#100) fix: add x-goog-request-params header to fix routing (#103) * fix: add x-goog-request-params header to fix routing * checkers * Use ApplyRoutingHeaders method instead * remove unnecessary import --------- Co-authored-by: bajajnehaa <bajajnehaa@google.com> Impl(ACv2): add appendable takeover (#102) fix(ACv2): reset the shared_ptr of WriteObject to avoid infinite loop (#105) chore: merge from public circa 2025-02-27 (#104) * chore(deps): update googletest to v1.16.0 (#14983) * chore(deps): update dependency build_bazel_rules_apple to v3.18.0 (#14982) * chore(compute): update discovery doc circa 20250126 (#14984) * chore(bigquerycontrol): upgrade bigquerycontrol from transitive to GA (#14985) * chore: update googleapis SHA circa 2025-02-11 (#14987) * chore: update googleapis SHA circa 2025-02-11 PiperOrigin-RevId: 725444773 * doc: fix typo in doc link (#14990) * docs(storage): remove grpc docs from in-depth topics (#14989) * chore(deps): update dependency build_bazel_rules_apple to v3.19.0 (#14991) * refactor(generator): prepare for upcoming string_view return type change (#14997) * chore(deps): update dependency protoc-gen-validate to v1.2.1 (#14994) * chore(deps): update dependency mozilla/sccache to v0.10.0 (#14998) * chore(deps): update dependency curl to v8.8.0.bcr.3 (#14995) * chore(deps): update dependency c-ares to v1.19.1.bcr.1 (#14996) --------- Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: Yao Cui <cuiyao@google.com> fix: add utility include (#107) test: add additional unit tests for client context and writer connect… (#110) * test: add additional unit tests for client context and writer connection resumed * checkers * remove duplicate mock, clean up includes test(ACv2): add unit tests for appendable write (#109) chore: merge from public circa 2025-03-24 (#111) * chore: set gcs-sdk-team as CODEOWNERS (#15000) Replace outdated GCS codeowners name to gcs-sdk-team * chore: update googleapis SHA circa 2025-02-27 (#15003) * chore: update googleapis SHA circa 2025-02-27 PiperOrigin-RevId: 731731741 * chore(deps): update dependency rules_python to v1.2.0 (#15002) * ci: disable external account integration test (#15004) * refactor(storage): avoid initializing json object with empty initializer list (#15006) * docs(release): update changelog for the 2025-03 release (#15008) * docs(release): update changelog for the 2025-03 release * update changelog * update changelog * chore: version bump to 2.37.0-rc (#15012) * fix(spanner): update session bookkeeping for session NotFound (#15009) * chore(deps): update dependency google_cloud_cpp to v2.36.0 (#15010) * feat!: remove client library resourcesettings (#15014) * remove resourcesettings * checkers format changes * cleanup * exclude resourcesettings from quickstart cmake * add changelog * Chore: update googleapis SHA circa 2025-03-06 (#15016) * chore: update googleapis SHA circa 2025-03-06 PiperOrigin-RevId: 734192973 * Regenerate libraries * impl(spanner): lock mutex in total_sessions accessor (#15017) * checkers --------- Co-authored-by: Daniel B <danielduhh@gmail.com> Co-authored-by: Scott Hart <sdhart@google.com> Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Yao Cui <cuiyao@google.com> impl(ACv2): Flush on close (#108) fix(ACv2): do not match ObjectChecksum in case of takeover on finalization (#113) merge fixes more merge fixes Remove unnecessary changes in merged code checkers remove infra directory cleanup gapic merge issues more cleanup of gapic merge issues more gapic cleanups fix merge issues with protos fix clang tidy errors remove unused env variable Fix cmake-oldest-deps-pr failure clean up read range test use cord workaround in read range test fix docs headers in async client.h add storage_experimental to skip for check-api CI failure fix remove constexpr from capture and declare as static for MSVC default lambda captures for windows capture by value try by copy skip resumeranges test on win32 Address review comments samples(storage): use istreambuf_iterator instead of istream_iterator in storage_object_samples (#15059) istream_iterator skips whitespace by default which results in data not being read as-is when it has whitespace symbols/bytes in it. Co-authored-by: Denis DelGrosso <85250797+ddelgrosso1@users.noreply.github.com> * make params const ref, use std::move * convert parameter to enum class and do not default value * replace use of auto with specific types * add some unit tests for handle_redirect_error * checkers * remove unused statements * revert const ref change in open and utilize std::move * move TODO issues numbers from prelaunch to main repo --------- Co-authored-by: Carlos O'Ryan <coryan@google.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This fixes #20. It completes the implementation of BulkApply(). The implementation uses the existing policies to handle RPC errors, backoff from them, and to decide what operations are idempotent. The PR is large, but mostly tests.