LLM-GenAI Model Evaluator

Immediate need for a talented LLM-GenAI Model Evaluator. This is a 12+months contract opportunity with long-term potential and is located in Austin, TX OR Sunnyvale, CA (Hybrid). Please review the job description below and contact me ASAP if you are interested. Job Diva ID: 26-00670 Pay Range: $50-$55/hr. Employee benefits include, but are not limited to, health insurance (medical, dental, vision), 401(k) plan, and paid sick leave (depending on work location). Key Requirements and Technology Experience: • Key Skills: - Artificial Intelligence, Machine Learning, AI/ML frameworks (PyTorch, TensorFlow, HuggingFace, LangChain) • Looking for GC and US Citizens. • Strong understanding of LLMs, generative AI, and transformer-based architectures. • Experience with Python, data analysis, and model evaluation frameworks. • Familiarity with prompt engineering, embeddings, RLHF/RLAIF, and LLM-based scoring methods. • Experience building evaluation datasets and working with annotation platforms. • Understanding of safety alignment, bias detection, and adversarial testing. • ML/AI frameworks: PyTorch, TensorFlow, HuggingFace, LangChain. • Evaluation/annotation tools: Scale AI, GroundTruth, Labelbox, Prodigy. • Prompt testing tools: Weights & Biases, MLflow, OpenAI evals, LLM-as-a-judge pipelines Our client is a leading IT Industry, and we are currently interviewing to fill this and other similar contract positions. If you are interested in this position, please apply online for immediate consideration Pyramid Consulting, Inc. provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state, or local laws. By applying to our jobs, you agree to receive calls, AI-generated calls, text messages, or emails from Pyramid Consulting, Inc. and its affiliates, and contracted partners. Frequency varies for text messages. Message and data rates may apply. Carriers are not liable for delayed or undelivered messages. You can reply STOP to cancel and HELP for help. You can access our privacy policy here.