Interface RunOnDatasetParams

interface RunOnDatasetParams {
    client?: Client;
    evaluationConfig?: RunEvalConfig;
    evaluators?: RunEvalType<"criteria" | "labeled_criteria" | "embedding_distance", RunEvaluator | ((props, options?) => Promise<EvaluationResult>) | ((props, options?) => EvaluationResult)>[];
    formatEvaluatorInputs?: EvaluatorInputFormatter;
    maxConcurrency?: number;
    projectMetadata?: Record<string, unknown>;
    projectName?: string;
}

Hierarchy

Properties

client? evaluationConfig? evaluators? formatEvaluatorInputs? maxConcurrency? projectMetadata? projectName?

Properties

Optional client

client?: Client

Client instance for LangSmith service interaction.

Optional evaluationConfig

evaluationConfig?: RunEvalConfig

⚠️ Deprecated ⚠️

Pass keys directly to the RunOnDatasetParams instead

This feature is deprecated and will be removed in the future.

It is not recommended for use.

Optional evaluators

evaluators?: RunEvalType<"criteria" | "labeled_criteria" | "embedding_distance", RunEvaluator | ((props, options?) => Promise<EvaluationResult>) | ((props, options?) => EvaluationResult)>[]

Evaluators to apply to a dataset run. You can optionally specify these by name, or by configuring them with an EvalConfig object.

Optional formatEvaluatorInputs

formatEvaluatorInputs?: EvaluatorInputFormatter

Convert the evaluation data into formats that can be used by the evaluator. This should most commonly be a string. Parameters are the raw input from the run, the raw output, raw reference output, and the raw run.

Example

// Chain input: { input: "some string" }
// Chain output: { output: "some output" }
// Reference example output format: { output: "some reference output" }
const formatEvaluatorInputs = ({
  rawInput,
  rawPrediction,
  rawReferenceOutput,
}) => {
  return {
    input: rawInput.input,
    prediction: rawPrediction.output,
    reference: rawReferenceOutput.output,
  };
};

Returns

The prepared data.

Optional maxConcurrency

maxConcurrency?: number

Maximum concurrency level for dataset processing.

Optional projectMetadata

projectMetadata?: Record<string, unknown>

Additional metadata for the project.

Optional projectName

projectName?: string

Name of the project for logging and tracking.

Settings

Member Visibility

Theme

On This Page

Generated using TypeDoc