JOB DESCRIPTION Key Skills: GCP Test Automation, CI/CD, API & UI testing automation, experience with Playwright framework and exposure to GenAI for test automation use case What you will do: API and Integration Testing:

• Design, develop, and execute comprehensive API test plans and test cases for AI models and services, covering functional, performance, security, and contract testing.
• Conduct rigorous testing of new language models (commercial and open source) via their APIs, focusing on accuracy, performance, scalability, and cost effectiveness.
• Validate data integrity and consistency across various API endpoints and integrations.
• Implement and maintain API test automation frameworks and tools.

Automation Development and Execution:

• Develop and maintain automated test scripts using Python and relevant testing frameworks to maximize test coverage and efficiency.
• Automate the testing of containerized microservices running in Google Cloud Platform (GCP) using appropriate CI/CD pipelines.
• Champion automation best practices and drive continuous improvement in automation coverage.
• Develop and maintain performance testing scripts for API endpoints and backend services.

AI Model Evaluation Framework Validation:

• Ensure the evaluation framework provides a robust and reliable environment for testing the latest AI offerings via automated processes.
• Develop automated tests to validate the functionality and performance of the evaluation framework itself.
• Implement automated reporting and analysis of test results.
• Run automated tests against one or multiple language models simultaneously through API interfaces.

Collaboration and Communication:

• Work closely with software engineers, data scientists, and other stakeholders to understand requirements and ensure the delivery of high quality AI solutions.
• Clearly communicate technical concepts and testing results to both technical and non technical stakeholders.
• Continuous Improvement:
• Stay abreast of the latest advancements in quality engineering, API testing, and AI technologies, incorporating best practices into the AI Hubs testing processes.
• Continuously evaluate and improve