Reports to: Human Data Evals Lead
Owns: end-to-end project delivery, staffing, production, quality, and on-time shipping
Location: Remote / Latam / US
You run our data projects from scoped brief to delivered package. Once an engagement is won, it’s yours: you assemble the right experts, drive production, enforce the quality bar on every artifact, and ship on time. You are the person who makes sure nothing leaves the building below frontier standard, and nothing comes back rejected without us already knowing why. On a small team running multiple concurrent projects, you are the delivery engine of the Human Data Division.
Project execution. Take scoped engagements through staffing, production, QC, and delivery. Own timeline, throughput, and the delivered package across several concurrent projects at once.
Expert orchestration. Staff each project from the expert pool, brief and calibrate contributors, manage their output and utilization, and keep production moving without quality drift.
Quality control. Run the QC structure on every deliverable — calibration layers, severity-weighted rubrics, deterministic verifiers, evidence maps — so each package survives buyer-side scrutiny before it ships.
Throughput & instrumentation. Track output-per-contributor-week, flag bottlenecks early, and protect margin by managing expert hours against scope.
Handoff. Keep the Strategic Project Lead and the Human Data Evals Lead informed on status, risk, and what shipped; surface anything that needs escalation before it becomes a delivery miss.
Delivered technical or data projects end to end under deadline pressure, owning quality and throughput across multiple concurrent workstreams.
Managed distributed contributors or expert pools — briefing, calibrating, and holding output to a standard.
Built or run QC processes that caught defects before a client did.
3+ years in program/delivery management, technical project management, or data operations, ideally in AI/ML data, evaluation, or annotation.
Comfortable with how frontier models are evaluated — rubrics, pass rates, what makes a task discriminate a model — enough to QC it, even if you don’t author it.
Proven ability to run several projects in parallel without dropping quality or timelines.
Fluent English. Spanish is a nice to have.