Content-Length: 337465 | pFad | https://github.com/googleapis/python-storage/issues/13

80 Storage: No timeouts cause indefinite hanging · Issue #13 · googleapis/python-storage · GitHub
Skip to content

Storage: No timeouts cause indefinite hanging #13

Closed
@mcsimps2

Description

@mcsimps2

Library: Google Cloud Storage
Environment: Win7 and OSX
Python Version: 3.7.3
Google Cloud Storage Version: 1.25.0

I don't believe all methods of the storage client are using timeouts. I've come across several situations where an upload or download has completely hung because of this. Unfortunately, there's no stack trace because the thread is just hanging waiting for a response. Just from a brief code inspection, I can identify an example area where a timeout is not being honored:

Bucket.get_blob calls blob.reload(), which then calls the following without specifying a timeout:

api_response = client._connection.api_request(
            method="GET",
            path=self.path,
            query_params=query_params,
            headers=self._encryption_headers(),
            _target_object=self,
        )

This then calls JSONConnection.api_request (defaults timeout to None) -> JSONConnection._make_request (defaults timeout to None) -> JSONConnection._do_request (defaults timeout to None) -> AuthorizedSession.request (defaults timeout to None), which makes the ultimate call to the requests.Session object with a None timeout. The end result is that a request is made without a timeout, which can very easily cause a thread to hang indefinitely waiting for a response.

I realize that it would be a huge pain to try and find all possible None timeout paths and patch them, but I at least wanted to bring this to attention. I'm currently wrapping every call to the Google Cloud Python library with a custom timeout function that forcefully stops execution after my own specified timeout, since I have no way to pass that in to the library. A fix that either allows developers to pass in their custom timeout, either to each function called (e.g. get_blob(...)) or to the client object so that it's passed with every single request in the underlying http instance, would be amazing. (In this sense, I suppose this issue is a mix of a bug and a feature request, so my apologies if I chose the incorrect category.)

Storage code for resumable uploads that makes the call to resumable_media/requests/_helpers.py, more specifically the http_request function, seems to do much better since that function sets a default timeout of (60, 61) as opposed to None.

Metadata

Metadata

Labels

🚨This issue needs some love.api: storageIssues related to the googleapis/python-storage API.priority: p1Important issue which blocks shipping the next release. Will be fixed prior to next release.type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions









    ApplySandwichStrip

    pFad - (p)hone/(F)rame/(a)nonymizer/(d)eclutterfier!      Saves Data!


    --- a PPN by Garber Painting Akron. With Image Size Reduction included!

    Fetched URL: https://github.com/googleapis/python-storage/issues/13

    Alternative Proxies:

    Alternative Proxy

    pFad Proxy

    pFad v3 Proxy

    pFad v4 Proxy