skip to content

#evaluation (8)

Concepts (1)

Retrieval-Augmented Generation (RAG)

Grounding LLM responses in chunks retrieved from an external corpus so the model reasons over real, citable sources instead of parametric memory alone.
llm vector-search ai grounding embeddings hallucination evaluation web-researched

Cheat sheets (7)

langsmith — LLM Observability SDK

Package-level reference for the langsmith SDK on PyPI — install, versioning, env-var setup, and observability alternatives.
pip package llm ai observability evaluation tracing

ragas — RAG Evaluation Framework

Package-level reference for ragas on PyPI — install variants, LLM-as-judge dependencies, metric churn, and alternative evaluators.
pip package ai rag evaluation

trulens-eval — LLM App Evaluation and Tracing

Package-level reference for trulens-eval on PyPI — install variants, the trulens umbrella rename, framework extras, and alternative evaluators.
pip package ai rag evaluation observability

LLM Evaluations

Build production evaluation pipelines for LLM applications — golden datasets, LLM-as-judge, rubrics, statistical significance, regression detection, and evals vs tests.
evals evaluation llm-as-judge golden-dataset regression statistics prompting

LangSmith — LLM Observability & Evaluation

Trace, debug, evaluate, and monitor LLM applications with LangSmith. Covers tracing setup, datasets, evaluators, prompt hub, comparing runs, and CI integration.
python langsmith llm observability tracing evaluation langchain ai

ragas — RAG Evaluation Framework

Measure and improve RAG pipeline quality with ragas. Covers faithfulness, answer relevancy, context precision, context recall, dataset format, LLM judges, and CI integration.
python ragas rag evaluation llm ai retrieval metrics

TruLens — LLM App Evaluation

Evaluate and monitor LLM applications with TruLens. Covers the RAG triad, feedback functions, TruChain, TruLlama, custom evaluators, the dashboard, and CI integration.
python trulens rag evaluation llm ai observability feedback