standard-tests: migrate to pytest-recording #31425
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 1 Skipped Deployment
|
CodSpeed Walltime Performance ReportMerging #31425 will not alter performanceComparing
|
CodSpeed Instrumentation Performance ReportMerging #31425 will improve performances by 59.7%Comparing Summary
Benchmarks breakdown
|
* Create safe default serializer / deserializer. * need to update the conftest code
| deser = serializer.deserialize(data) | ||
| return deser["requests"], deser["responses"] |
There was a problem hiding this comment.
@eyurtsev I updated the return type of load_cassette following https://github.com/kevin1024/vcrpy/blob/19bd4e012c8fd6970fd7f2af3cc60aed1e5f1ab5/vcr/cassette.py#L361
dosubot
bot
added
the
size:L
|
@eyurtsev FYI the new serializer is noticeably slower, I don't think it's a problem so will ignore the regressions flagged by codspeed. |
|
How is that possible?? |
| config = _base_vcr_config.copy() | ||
| config.setdefault("filter_headers", []).extend(_EXTRA_HEADERS) | ||
| config["before_record_request"] = remove_request_headers | ||
| config["before_record_response"] = remove_response_headers | ||
| config["serializer"] = "yaml.gz" | ||
| config["path_transformer"] = VCR.ensure_suffix(".yaml.gz") | ||
|
|
||
| return config | ||
|
|
||
|
|
||
| @pytest.fixture | ||
| def vcr(vcr_config: dict) -> VCR: |
There was a problem hiding this comment.
@eyurtsev blocking issue:
if we don't define the vcr fixture like this, it gets value None for some reason in langchain-tests:
cd libs/partners/anthropic
rm tests/cassettes/*
ANTHROPIC_API_KEY=... uv run python -m pytest --capture=no tests/integration_tests/test_standard.py::TestAnthropicStandard::test_stream_time
however, implementing the vcr fixture here might overwrite pytest-recording's fixture, so if you decorate a test with @pytest.mark.vcr no cassette is generated.
I suspect we should remove the vcr fixture implementation here and somehow help the test in langchain-tests pick up the fixture.
There was a problem hiding this comment.
Believe this is fixed in 3aa729d
No description provided.