Content-Length: 360426 | pFad | https://github.com/googleapis/python-aiplatform/commit/fd38b49231bd1b35af57056b9e69a5427ddb114c

39 feat: GenAI Evaluation: Release GenAI Evaluation SDK GA features to `… · googleapis/python-aiplatform@fd38b49 · GitHub
Skip to content

Commit fd38b49

Browse files
jsondaicopybara-github
jsondai
authored and
copybara-github
committed
feat: GenAI Evaluation: Release GenAI Evaluation SDK GA features to vertexai.preview module.
feat: GenAI Evaluation: Add generic model-based `PointwiseMetric` and `PairwiseMetric` classes that allow customizing metric prompt templates. Add `PointwiseMetricPromptTemplate`, `PairwiseMetricPromptTemplate` classes to help formulate and customize metric prompt templates. Add `metric_column_mapping` parameter to `EvalTask` for metric prompt template input variable name mapping. feat: GenAI Evaluation: Open source model-based metric prompt template examples for Gemini 1.5 Pro autorater. Add `MetricPromptTemplateExamples` class to help retrieve model-based metric prompt templates. BREAKING_CHANGE: GenAI Evaluation: Rename "rapid_evaluation" to "evaluation" subpackage. Deprecate EvalTask support for metric bundle names and string model-based metric names. Deprecate individual model-based metric classes. Remove EvalTask individual column name mapping. Rename "content" column name to "prompt" in evaluation dataset. Remove "confidence" from model-based metric results PiperOrigin-RevId: 665004260
1 parent 2c93fc1 commit fd38b49

31 files changed

+2719
-2010
lines changed

‎setup.py

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -149,7 +149,7 @@
149149
"pydantic >= 2.6.3, < 3",
150150
]
151151

152-
rapid_evaluation_extra_require = [
152+
evaluation_extra_require = [
153153
"pandas >= 1.0.0, < 2.2.0",
154154
"tqdm>=4.23.0",
155155
]
@@ -190,7 +190,7 @@
190190
+ autologging_extra_require
191191
+ preview_extra_require
192192
+ ray_extra_require
193-
+ rapid_evaluation_extra_require
193+
+ evaluation_extra_require
194194
)
195195
)
196196
testing_extra_require = (
@@ -276,7 +276,7 @@
276276
"ray": ray_extra_require,
277277
"ray_testing": ray_testing_extra_require,
278278
"reasoningengine": reasoning_engine_extra_require,
279-
"rapid_evaluation": rapid_evaluation_extra_require,
279+
"evaluation": evaluation_extra_require,
280280
"langchain": langchain_extra_require,
281281
"langchain_testing": langchain_testing_extra_require,
282282
"tokenization": tokenization_extra_require,

0 commit comments

Comments
 (0)








ApplySandwichStrip

pFad - (p)hone/(F)rame/(a)nonymizer/(d)eclutterfier!      Saves Data!


--- a PPN by Garber Painting Akron. With Image Size Reduction included!

Fetched URL: https://github.com/googleapis/python-aiplatform/commit/fd38b49231bd1b35af57056b9e69a5427ddb114c

Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy