English Prompt Engineer: LLM Migration & Optimization

United States

Mandatory
English
Oops! This job isn't available anymore. Check out other jobs with English
DESCRIPTION
Are you an expert at navigating the complex logic of Large Language Models? Welo Data is seeking a technical Prompt Engineer to lead the end-to-end migration of template workflows into high-performance LLM autoraters.

This isn't just about writing prompts—it’s about engineering a technical bridge. You will use advanced APG/APO tools and manual refinement to ensure our automated systems meet (and exceed) human accuracy baselines.

The Mission: Architecting the Future of Rating

  • Technical Migration: Take ownership of the workflow for transitioning templates to LLM autoraters.
  • Optimization Leadership: Run and supervise Automated Prompt Optimization (APO) tools, identifying where logic plateaus and providing the manual "spark" to overcome deadlocks.
  • Metrics-Driven Accuracy: Continuously measure quality against gold data, calculating critical performance metrics like precision, recall, and F1 scores.
  • Edge-Case Engineering: Solve complex scenarios by designing manual prompts that handle anti-patterns and broken logic in legacy architectures.

Project Details

  • Schedule: Part-Time (Flexible hours within project milestones)
  • Location: 100% Remote (Must be based in the United States)
  • Employment Type: Freelance / Independent Contractor

 Who We Are Looking For

  • Linguistic & AI Mastery: 2+ years of experience as a Prompt Engineer. You must be comfortable tuning LLMs for structured outputs and complex classification tasks.
  • Academic Background: BS, MS, or PhD in Computer Science, Data Science, Computational Linguistics, or a related analytical field.
  • Technical Agility: Fast learner capable of mastering proprietary internal tools and interfaces (like the Goose API) with minimal supervision.
  • Data Fluency: Strong ability to identify error patterns and use SQL or data analytics tools to analyze model performance.

Preferred Skills

  • Familiarity with shadowbot disagreement tracking between humans and LLMs.
  • Hands-on experience with Chain-of-Thought (CoT) and few-shot learning.
  • Proven ability to draft high-level Launch Certification Documentation.
SKILLS
Prompt
Prompt Delivery
Computer Science
Data Science
Computational Linguistics
LLMs
Launch Certification Documentation
Check out similar jobs