NLP Annotation Judge

Our Customer is a tech real-estate marketplace company with a portfolio of the largest and most vibrant real estate and home-related brands on the web and mobile. The company’s brands focus on all stages of the home lifecycle: renting, buying, selling, financing and home improvement. They are committed to empowering consumers with unparalleled data, inspiration and knowledge around homes, and connecting them with the right local professionals to help.

We are seeking an NLP Annotation Judge on a contract basis to join the Human In The Loop Operations team and contribute to the growing set of Natural Language Processing (NLP) projects. The Annotation Judge will play a crucial role in fine-tuning and calibrating AI/ML processes that handle verbal interactions, between home buyers and real estate agents for example. This role is 100% remote.

The Annotation Judge will play a crucial role in fine-tuning and calibrating AI/ML processes that handle verbal interactions, between home buyers and real estate agents for example.

This role will work directly with experienced Operations Managers and Natural Language experts. You will be taught everything you need to know!

Once ramped up, you will be responsible for ensuring the quality of a number of possible elements related to projects, from the accuracy and humanlike traits of automated chatbots answers, to the classification of elements of a conversation annotated automatically or the review of search results based on understanding of complex users queries.

Will play an essential role in defining clear guidelines which may be used by non-native English speakers. Attention to detail and natural curiosity for language subtleties will be essential for the long term scaling of solutions. Ideal candidate is a responsible and reliable self-starter who can be comfortable with ambiguity and is willing to learn new things.

RESPONSIBILITIES:

• Review and/or evaluate annotated data to ensure it meets quality standards and project requirements.
• Collaborate with internal teams to optimize model performance based on annotated data.
• Help develop and maintain annotation guidelines and best practices.
• Annotate training data to establish Ground Truth data.
• Identify gaps in training content and suggest improvements.

MINIMUM QUALIFICATIONS:

• Excellent attention to detail and analytical skills
• Enthusiasm for the AI space and willingness to learn
• Ability to work in ambiguous situations
• Ability to work effectively in a collaborative team environment.

PREFERRED QUALIFICATIONS:

• Experience or affinity for linguistics
• Experience with copywriting, UX authoring, content development around voice and brand guidelines.
• Experience following complex processes
• Ability to work both independently and with a team to achieve a goal.

BONUS QUALIFICATIONS:

• Bachelor's degree in English, Linguistics or a related field.
• Experience with Machine-generated Speech evaluation, Prompt Engineering, Search Results Evaluation, Search Query Categorization
• Experience with tools and software used for NLP annotation is a plus.

Perks are available through our 3rd Party Employer of Record (Available upon completion of waiting period for eligible engagements)

Health Benefits: Medical, Dental, Vision, Life (including spouse & child), 401k, STD/LTD, AD&D, and Commuter Benefits program.

An Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, or protected veteran status and will not be discriminated against on the basis of disability.