Regional task definition
Name the country, workflow, intended user and consequence of a wrong answer. The benchmark then targets regional knowledge and decisions rather than treating translated fluency as competence.
AI data services · Latin America
Regional benchmarks built from verifiable sources
Measure whether a system understands a country-specific workflow, code, institution or operational decision. Savia designs benchmark scopes around authorized evidence and separates questions that test public knowledge from questions that require rows supplied at evaluation time. Answer keys can be computed from source records; any model-assisted wording remains labeled as model-inferred until a qualified person confirms it.
Scope the work
Evidence and answer construction
Agree the public or otherwise authorized sources, relevant dates, transformations and answer rules. Where the truth can be calculated, the answer key is derived deterministically and ships with the derivation.
Knowledge versus grounded reasoning
Keep closed-book questions separate from questions that require a supplied table or document. Without the required rows, abstention may be the correct behavior; it must not be averaged into a flattering knowledge score.
What you receive
- Task files with prompts, track, item type, scoring rule and regional assumptions.
- Answer file with source references, derivations, tolerances or accepted options as applicable.
- Manifest recording versions, provenance states, source licences, counts and known limitations.
- Baseline report that separates competence, grounding and calibration metrics when the design calls for them.
Quality and controls
- Every source must be publicly verifiable or covered by documented authorization before it enters the scope.
- Computed answers remain labeled DERIVED; expert decisions become HUMAN_CONFIRMED only after named review.
- Model-assisted prompt wording remains MODEL_INFERRED until confirmed and is never presented as an expert-authored fact.
- Rejected items, leakage checks, ambiguity rules and benchmark limitations are documented for audit and critique.
Savia Carga · Colombia
Inspect the method in a published release
Savia Carga publishes its tasks, answer key, derivations, rejected item, sources and limitations. It shows how Savia separates source evidence, deterministic computation, human confirmation and model inference.
Open Savia Carga ↗AI data services · Latin America
Questions about regional benchmarks
Can a model write the answer key?
Savia’s published method does not use a model to produce answers. In Savia Carga, answers are computed deterministically from public rows and ship with their derivations. A new scope defines the appropriate source-backed answer process.
What is the difference between DERIVED and MODEL_INFERRED?
DERIVED means a documented computation produced the value from source evidence. MODEL_INFERRED means a model produced the content. In Savia Carga, answers are DERIVED while the Spanish prompt wording is MODEL_INFERRED and not yet expert-confirmed.
Do you build private benchmarks?
A benchmark may use material the client is authorized to provide, with rights, access, retention and deletion defined in writing.
What proof is available today?
Savia Carga v0.1.0 is a downloadable 43-item Colombian road-freight preprint using public MinTransporte sources. It publishes the answer key and limitations; its questions have not yet been confirmed by a domain expert.
Discuss a pilot
Send the task, market, source material, rubric and expected volume. We will define what can be staffed and verified, then price the scope in writing before work begins.
Discuss a pilotAI data services
Services for AI and data teams
Spanish AI evaluation for Latin American markets
Regional Spanish AI evaluation for es-CO, es-MX, es-AR and other scoped variants: rubric scoring, preference ranking and red-teaming.
Explore the serviceExpert annotation for healthcare, law and finance
Healthcare, legal and finance annotation in Latin America with project-specific practitioner verification, source rights and item-level provenance.
Explore the serviceEgocentric video collection in Latin America
Project-based egocentric video collection in Latin America: recruitment, local-language onboarding, household consent and daily quality review.
Explore the service