The primary goal of
Content Reviewers (Raters) is to evaluate various health related prompts, make individual judgments and decisions based on predefined options from task guidelines to support machine learning project objectives. Your feedback will directly inform positive and negative changes to the underlying AI models and contribute to the refinement of evaluation guidelines.
Key Responsibilities:
- Thoroughly read and understand a new set of evaluation guidelines, including specific criteria and examples for assessing model output.
- Evaluate the overall quality of responses based on provided criteria, focusing on areas such as:
- Ability to answer user questions (including identifying explicit refusals/punts).
- Helpfulness of responses in meeting user needs and providing actionable information.
- Understanding and interpretation of user queries.
- Factuality of information, including identifying hallucinations or fabricated knowledge.
- Provision of recommendations for health and wellness improvement or general health knowledge.
- Comprehensiveness (clarity and directness) in addressing all aspects of a query.
- Utilization of personal user data ( metrics, demographics, memories) in responses.
- Extraction and correct use of information from personal data.
- Overall sentiment and appropriateness of tone in messages.
- Overall quality and room for improvement of the response/model.
- Appropriateness of response length to the query.
- Cultural and regional appropriateness of the response (terminology, units, references, tone).
- Document any questions, concerns, or issues encountered while applying the guidelines to the new model output.
- Provide clear, concise, and constructive comments directly on the guidelines to facilitate fast-follow updates.
- Participate in calibration sessions with team members, other locations, or clients as instructed.
- Collaborate with team members to share best practices.
- Work effectively both independently and as part of a team.
- Synthesize evaluation findings into actionable recommendations for modeling and product teams.
- Communicate clearly with internal and external contacts as required.
- Manage multiple assignments concurrently.
- Adhere to Tech Mahindra and client procedures, policies, and guidelines.
- Follow and adhere to all KPI-related tasks to prevent business impact.
- Ensure breaks are pre-approved by the SME, Team Leader, or Managers.
- Comply with all instructions from the Team Leader & superiors.
- Adhere to all process-specific policies.
- Maintain confidentiality for all process-related work and critical details.
- Undertake any additional ad hoc tasks that contribute to improved client service or KPI enhancement.
- Identify opportunities that would help enhance the existing workflow.