Skip to content

Remove invalid chunk validation #2992

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Feb 14, 2017
Merged

Remove invalid chunk validation #2992

merged 4 commits into from
Feb 14, 2017

Conversation

garye
Copy link
Contributor

@garye garye commented Feb 9, 2017

Fixes #2980

@googlebot googlebot added the cla: yes This human has signed the Contributor License Agreement. label Feb 9, 2017
@lukesneeringer
Copy link
Contributor

@dhermes Can you glance at this one? I lack sufficient context.

@dhermes
Copy link
Contributor

dhermes commented Feb 9, 2017

@garye I'd guess there would be broken tests, but it's hard to say right now because there is a bad release of protobuf that is breaking everything

@@ -366,8 +366,6 @@ def _validate_chunk_row_in_progress(self, chunk):
"""Helper for :meth:`_validate_chunk`"""
assert self.state == self.ROW_IN_PROGRESS
self._validate_chunk_status(chunk)
if not chunk.HasField('commit_row') and not chunk.reset_row:
_raise_if(not chunk.timestamp_micros or not chunk.value)

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

@lukesneeringer
Copy link
Contributor

@dhermes Update on the protobuf thing: googleapis/api-client-staging#159

@dhermes
Copy link
Contributor

dhermes commented Feb 9, 2017

I restarted the CircleCI build: https://circleci.com/gh/GoogleCloudPlatform/google-cloud-python/865

Hopefully it works?

@garye
Copy link
Contributor Author

garye commented Feb 13, 2017

Can someone help me finish this off? The test failures don't really seem related. Getting this fix pushed and released will unblock a number of users.

@dhermes
Copy link
Contributor

dhermes commented Feb 13, 2017

@garye Just re-started the CircleCI build: https://circleci.com/gh/GoogleCloudPlatform/google-cloud-python/876

Hopefully the bad imports are fully resolved. I'll follow-up if not.

@dhermes
Copy link
Contributor

dhermes commented Feb 13, 2017

@garye So we've got a unit test failing that is actually caused by this PR:


    def test_invalid_empty_second_chunk(self):
        from google.cloud.bigtable.row_data import InvalidChunk
    
        chunks = _generate_cell_chunks(['', ''])
        first = chunks[0]
        first.row_key = b'RK'
        first.family_name.value = 'A'
        first.qualifier.value = b'C'
        response = _ReadRowsResponseV2(chunks)
        iterator = _MockCancellableIterator(response)
        prd = self._make_one(iterator)
        with self.assertRaises(InvalidChunk):
>           prd.consume_next()
E           AssertionError: InvalidChunk not raised

ISTM the right move is just to delete this test case since it no longer applies?

@garye
Copy link
Contributor Author

garye commented Feb 13, 2017

yeah, i think that's an invalid test case

@dhermes
Copy link
Contributor

dhermes commented Feb 13, 2017

Ping me when this PR is updated with the test case removed? You can ignore the speech test failures. It looks like the underlying protobuf libraries are (now correctly) using single instead of double precision for message fields with single precision

@garye
Copy link
Contributor Author

garye commented Feb 13, 2017 via email

with self.assertRaises(InvalidChunk):
prd.consume_next()


This comment was marked as spam.

This comment was marked as spam.

@garye
Copy link
Contributor Author

garye commented Feb 13, 2017

Yeah, thanks I noticed that and (hopefully) pushed a fix.

@tseaver
Copy link
Contributor

tseaver commented Feb 13, 2017

@garye Ugh, you need to be able to run tox -e lint locally:

bigtable/unit_tests/test_row_data.py:419:1: W293 blank line contains whitespace

@garye
Copy link
Contributor Author

garye commented Feb 13, 2017 via email

@tseaver
Copy link
Contributor

tseaver commented Feb 13, 2017

I think it should just work to run tox -e pylint. You probably want to export the following environment variables so it only runs on your changes:

$ export GOOGLE_CLOUD_TESTING_REMOTE=upstream  # the name of the remote for this repo
$ export GOOGLE_CLOUD_TESTING_BRANCH=master
$ tox -e lint

@dhermes
Copy link
Contributor

dhermes commented Feb 14, 2017

LGTM

@tseaver tseaver merged commit 951f5d0 into googleapis:master Feb 14, 2017
@garye garye deleted the patch-1 branch February 14, 2017 23:01
@garye
Copy link
Contributor Author

garye commented Feb 15, 2017

Thanks guys!

@dhermes
Copy link
Contributor

dhermes commented Feb 15, 2017

@garye We hope to cut a release in the next 2-3 days as well

richkadel pushed a commit to richkadel/google-cloud-python that referenced this pull request May 6, 2017
Remove invalid chunk validation
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy