AI/ML Quality & Prompt Analyst - English Language

Location: Remote (Serbia/Montenegro)

Position Type: Full time

About the Company

SETVI is a Software startup (SaaS) based in Philadelphia, United States, and Belgrade, Serbia. Our product is a revenue acceleration platform for the manufacturing and distribution industry. This platform organizes the company’s sales operations to upsell, cross-sell, and increase profitability. Focusing on Manufacturing and Wholesale/Distribution verticals allows SETVI to build deeply impactful tools and workflows.

Learn more at setvi.com.

Job Overview

We are a cutting-edge technology company at the forefront of the Artificial Intelligence (AI) revolution. Our mission is to create intelligent solutions to transform the Manufacturing and Distribution industry and drive innovation. As we continue to expand, we seek a talented and motivated LLM Prompt Analyst (English Language) to join our dynamic team and play a pivotal role in shaping the future of AI.

In this role, you will not only craft, optimize, and refine prompts for Large Language Models (LLMs) but also rigorously validate and test the results within our AI-driven platform. Your work will be pivotal in ensuring that AI outputs—from product matching to attribute comparisons—meet our high standards for accuracy, relevance, and quality.

Responsibilities

Prompt Development & Optimization: 

• Design, refine, and maintain clear, concise, and creative prompts to improve LLM performance.
• Experiment with various LLM prompt-engineering techniques to enhance model responses and tailor them to specific business needs.
• Analyze model outputs to identify areas of improvement and suggest prompts to our Data Scientist to optimize AI model performance.  You will be trained on LLM prompts.
• Experiment with various LLM prompt engineering techniques to enhance model responses.

Rigorous Output Validation & Testing:

• Conduct thorough testing of AI outputs—such as product matches and product attribute comparisons—to ensure accuracy and consistency.
• Develop and follow test plans to identify, document, and track issues or areas of improvement in model responses.
• Self-learning and conducting research to stay updated with the latest advancements in AI, NLP (Natural Language Processing), and prompt engineering practices.

Cross-Functional Collaboration:

• Work closely with product managers, developers, and data scientists to align prompt strategies with overall product goals and customer requirements.
• Participate in client-facing meetings to gather feedback, provide insights, and ensure prompt strategies meet end-user expectations.

Qualifications

• University degree in English Language, Linguistics, Machine Learning, Data Science or other related fields
• Strong proficiency in English with an exceptional command of grammar, syntax, and language nuances.
• Familiarity with AI-related tools, platforms, and models such as OpenAI, GPT, or other LLMs.
• Experience in client-facing roles.
• A creative mindset with the ability to design engaging, clear, and effective prompts to elicit the desired AI responses.
• Solid problem-solving and analytical skills, with a keen eye for detail.
• Ability to conduct research and experiment with different prompt techniques to refine and improve model outputs.
• Excellent communication skills, both written and verbal, with the ability to clearly explain prompt strategies and adjustments.
• Passion for artificial intelligence, natural language processing, and the future of AI-driven technologies.

Nice-to-have

• Knowledge of LLM, data science, machine learning principles, or NLP concepts.

Perks of working at SETVI

• Compensation above the industry average - because we want you to feel appreciated
• Work from anywhere - as long as you get the job done
• Opportunity to work on exciting projects at a growing startup company
• Monthly get-togethers with the local team
• Expand your knowledge and learn more about design, technology, and all things that spark your interest (Udemy courses)
• Referral bonuses

The selection process

1. Submit your CV
2. HR interview
3. Interview with the hiring manager
4. Assignment
5. Final interview