A framework for few-shot evaluation of language models.
-
Updated
Apr 18, 2025 - Python
#
evaluation-framework
Here are 190 public repositories matching this topic...
Test your prompts, agents, and RAGs. Red teaming, pentesting, and vulnerability scanning for LLMs. Compare performance of GPT, Claude, Gemini, Llama, and more. Simple declarative configs with command line and CI/CD integration.
testing ci evaluation ci-cd pentesting cicd vulnerability-scanners prompts evaluation-framework red-teaming rag llm prompt-engineering llmops prompt-testing llm-eval llm-evaluation llm-evaluation-framework
-
Updated
Apr 22, 2025 - TypeScript
The LLM Evaluation Framework
evaluation-metrics evaluation-framework llm-evaluation llm-evaluation-framework llm-evaluation-metrics
-
Updated
Apr 22, 2025 - Python
Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends
-
Updated
Apr 22, 2025 - Python
This is the repository of our article published in RecSys 2019 "Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches" and of several follow-up studies.
deep-learning neural-network reproducible-research collaborative-filtering matrix-factorization hyperparameters bpr recommendation-system recommender-system reproducibility recommendation-algorithms knn matrix-completion evaluation-framework content-based-recommendation hybrid-recommender-system funksvd bprmf bprslim slimelasticnet
-
Updated
May 25, 2023 - Python
Data-Driven Evaluation for LLM-Powered Applications
information-retrieval evaluation-metrics evaluation-framework rag llmops retrieval-augmented-generation llm-evaluation
-
Updated
Jan 22, 2025 - Python
AgentLab: An open-source framework for developing, testing, and benchmarking web agents on diverse tasks, designed for scalability and reproducibility.
-
Updated
Apr 22, 2025 - Python
Metrics to evaluate the quality of responses of your Retrieval Augmented Generation (RAG) applications.
evaluation-metrics evaluation-framework rag large-language-models llm llms llmops retrieval-augmented-generation
-
Updated
Nov 14, 2024 - Python
Python SDK for running evaluations on LLM generated responses
evaluation evaluation-metrics evaluation-framework llmops llm-eval llm-ops llm-evaluation llm-evaluation-toolkit
-
Updated
Apr 15, 2025 - Python
The official evaluation suite and dynamic data release for MixEval.
benchmark evaluation benchmarking-suite evaluation-framework benchmarking-framework foundation-models large-language-models large-language-model llm-inference llm-evaluation large-multimodal-models llm-evaluation-framework benchmark-mixture mixeval
-
Updated
Nov 10, 2024 - Python
Moonshot - A simple and modular tool to evaluate and red-team any LLM application.
-
Updated
Apr 21, 2025 - Python
A research library for automating experiments on Deep Graph Networks
-
Updated
Sep 9, 2024 - Python
AI Data Management & Evaluation Platform
-
Updated
Oct 5, 2023 - Svelte
PySODEvalToolkit: A Python-based Evaluation Toolbox for Salient Object Detection and Camouflaged Object Detection
latex metrics evaluation python3 evaluator evaluation-metrics evaluation-framework metrics-visualization mae saliency salient-object-detection saliency-detection co-saliency f-measure pr-curve e-measure s-measure camouflaged-object-detection fm-curve co-salient-object-detection
-
Updated
Sep 27, 2024 - Python
Expressive is a cross-platform expression parsing and evaluation framework. The cross-platform nature is achieved through compiling for .NET Standard so it will run on practically any platform.
xamarin parsing cross-platform evaluation netstandard expression-parser expression-evaluator hacktoberfest evaluation-framework
-
Updated
Oct 1, 2024 - C#
DevQualityEval: An evaluation benchmark 📈 and framework to compare and evolve the quality of code generation of LLMs.
-
Updated
Apr 22, 2025 - Go
Test and evaluate LLMs and model configurations, across all the scenarios that matter for your application
-
Updated
Aug 20, 2024 - TypeScript
Evaluation suite for large-scale language models.
-
Updated
Aug 15, 2021 - Python
A framework for standardizing evaluations of large foundation models, beyond single-score reporting and rankings.
-
Updated
Apr 22, 2025 - Python
Multilingual Large Language Models Evaluation Benchmark
multilingual nlp natural-language-processing evaluation evaluation-datasets datasets language-model evaluation-framework large-language-models
-
Updated
Aug 21, 2024 - Python
Improve this page
Add a description, image, and links to the evaluation-framework topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the evaluation-framework topic, visit your repo's landing page and select "manage topics."