Fine‑Tuning & Model Economics
Compare fine‑tuning, retrieval, and prompt engineering to choose the most cost‑effective customization path for your use case.
KPIs
Task Success Rate
Share of evaluation cases that meet the quality criterion.
Not availableEstimatedConfidence 78%Higher is better
Hallucination Rate
Fraction of responses flagged as unsupported or factually incorrect.
StaleUnknownConfidence 58%Higher is worse
Cost per Successful Request
Variable cost allocated per request that meets the quality threshold.
Not availableQualitativeConfidence 52%Higher is worse
Effective Cost per 1K Tokens
Blended cost per 1,000 tokens including retrieval and orchestration overheads.
StaleComputedConfidence 81%Higher is worse
Latency P95
95th percentile end-to-end latency (prompting + retrieval + inference).
CurrentUnknownConfidence 52%Higher is worse
Compliance Conformance Rate
Fraction of requests processed within allowed regions and with policy-compliant handling of sensitive data.
Not availableInferredConfidence 51%Higher is better
Net Savings (USD)
Baseline cost minus current cost for the same volume and task mix.
Insufficient sampleEstimatedConfidence 57%Higher is better
Payback Period (days)
Days to recoup fine-tuning program costs via net savings.
Insufficient sampleInferredConfidence 79%Higher is worse
Customization Fit Index
Composite 0–1 index combining quality, cost, latency, and compliance for the chosen approach.
StaleQualitativeConfidence 69%Higher is better
Internal Factors
Tokens per Request
Average total tokens (prompt + completion) used per request in the window.
Insufficient sampleProxyConfidence 89%Higher is worse
Context Utilization Ratio
Share of available context window actually used.
Insufficient sampleQualitativeConfidence 90%Higher is worse
Token Inflation Factor
Multiplier of tokens introduced by prompting/retrieval over raw input size.
CurrentInferredConfidence 89%Higher is worse
Retrieval Recall@K
Share of eval queries for which at least one relevant document appears in the top‑K.
StaleProxyConfidence 77%Higher is better
Retrieval Usage Share
Fraction of requests that invoked retrieval as part of the response.
Insufficient sampleProxyConfidence 61%Higher is worse
Embedding Index Age (days)
Days since the embedding index or corpus was last refreshed.
Insufficient sampleUnknownConfidence 74%Higher is worse
Labeled Dataset Size
Number of labeled examples available for fine‑tuning/evaluation.
Not availableComputedConfidence 78%Higher is better
Label Quality Score
Normalized 0–1 score capturing label accuracy/consistency.
Insufficient sampleInferredConfidence 59%Higher is better
Fine‑Tune Checkpoint Age (days)
Days since the fine‑tuned model checkpoint was produced.
CurrentUnknownConfidence 62%Higher is worse
Fine‑Tuning Total Cost (USD)
Cumulative spend on fine‑tuning runs, data prep, and evaluation.
CurrentUnknownConfidence 62%Higher is worse
Domain Drift Index
0–1 index of distribution shift between production queries and the fine‑tune/eval corpus.
StaleInferredConfidence 57%Higher is worse
PII Flag Rate
Share of requests flagged by data‑loss‑prevention (DLP) or privacy rules.
Insufficient sampleEstimatedConfidence 54%Higher is worse
In‑Region Processing Share
Share of requests executed in the region dictated by policy/jurisdiction.
Not availableQualitativeConfidence 58%Higher is better
Levers
Customization Strategy
Chosen approach to customization.
Insufficient sampleUnknownConfidence 59%
Context Window (tokens)
Maximum prompt tokens allowed per request.
Insufficient sampleQualitativeConfidence 83%
Max Output Tokens
Upper bound on generated tokens per request.
StaleUnknownConfidence 76%
Retrieval Top‑K
Number of documents retrieved per query.
Not availableDirectConfidence 79%
Reranker Policy
Reranking model/policy applied to retrieved candidates.
Insufficient sampleDirectConfidence 87%
LoRA Rank
Rank parameter used for Low‑Rank Adaptation during fine‑tuning.
Insufficient sampleProxyConfidence 86%
Fine‑Tune Epochs
Number of passes over the training set.
StaleQualitativeConfidence 67%
Quantization Level
Numeric precision for serving.
Not availableEstimatedConfidence 88%
Data Residency Policy
Placement rules for where data is processed.
Insufficient sampleComputedConfidence 75%