Machine Learning Specialist

Role: Machine Learning Model Validation & Statistical Assurance Specialist

Location: Bay Area, CA. (Onsite)

W2 role

Key Responsibilities

Statistical Viability & Hypothesis Testing

• Designing and Executing Rigorous Statistical Tests: This responsibility entails the foundational work of ensuring that any observed relationship between variables is not merely due to chance. The specialist will design experiments and tests to establish the likelihood that an outcome is genuinely caused by the factors under study, rather than random variability. The proactive involvement in experimental design is essential, moving beyond mere post-hoc analysis.
• Performing Hypothesis Testing and Statistical Inference: The specialist will be responsible for defining and testing null and alternative hypotheses to determine statistical significance.
• Applying Advanced Statistical Methods for Data and Model Analysis: This encompasses a broad range of techniques vital for deeply analyzing datasets and model outputs. These include probability theory, which quantifies uncertainty and aids in prediction. Various sampling methods (e.g., random, stratified, cluster) are crucial for selecting representative subsets of data, allowing for reliable inferences about larger populations.

Machine Learning Algorithm Assessment & Selection

• Evaluating and Recommending Appropriate ML Algorithms: The specialist will assess the suitability of various ML algorithms, including supervised learning tasks like classification, regression, and forecasting, as well as unsupervised learning methods such as clustering and dimensionality reduction, and reinforcement learning.
• Conducting In-depth Analysis of Algorithmic Soundness and Applicability: This involves scrutinizing the theoretical underpinnings of selected algorithms, their assumptions, and their appropriateness for the specific problem and dataset.
• Assessing Model Performance Metrics: The specialist will evaluate models based on a variety of metrics relevant to the task. For classification tasks, this includes accuracy, precision, recall, and F1-score; for regression, metrics such as Root Mean Squared Error (RMSE) and Mean Absolute Error (MAE) are used.

Model Validation & Performance Assurance

• Developing and Implementing Comprehensive Validation Protocols: This involves designing and executing rigorous validation protocols and test cases to evaluate model robustness, reliability, and compliance under diverse conditions. This includes quantitative approaches such as back testing, which uses historical data to assess accuracy; developing challenger models to provide independent verification against a "champion" model; performing sensitivity analysis to understand how changes in variables affect outcomes; and conducting stress testing using speculative scenarios to evaluate model resilience.
• Diagnosing and Troubleshooting Model Performance Issues: The specialist will identify root causes of performance degradation, accuracy issues, or runtime problems, and propose effective solutions. This often involves detailed model debugging to understand why a model made a particular mistake and how it can be improved. The ability to diagnose and resolve issues extends beyond merely identifying a problem; it requires a deep understanding of model mechanics and data pipelines to pinpoint the root cause.
• Conducting Various Performance Analyses: This responsibility includes analyzing validation results and delivering detailed reports that provide insights into model strengths, weaknesses, and opportunities for improvement. It also involves continuously monitoring models for drift and ensuring they continue to perform as expected in production environments.
• Ensuring Compliance with Standards and Regulations: This role ensures that ML models adhere to specified internal standards, ethical guidelines, and external regulatory requirements, especially in sensitive domains. The consistent emphasis on compliance and ethical guidelines underscores that this role is not solely about technical performance but also about governance and risk management.

ML Interpretability & Fairness

• Applying Techniques for Model Explainability (Global and Local): The specialist will utilize various interpretability techniques to understand how ML models arrive at their predictions. This includes analyzing intrinsically explainable models like linear regression and decision trees by examining their inherent structures and coefficients.
• Identifying and Mitigating Algorithmic Biases: The role involves systematically analyzing ML models to detect potential biases that could lead to unfair or discriminatory outcomes. This includes assessing fairness, transparency, and accountability in AI decision-making processes.

Data Governance & Quality Assurance

• Ensuring Data Quality and Integrity for Model Development: The specialist will play a crucial role in ensuring that data used for ML applications is accurate, complete, and consistent. This includes contributing to data collection, cleaning, preprocessing, and validation efforts, which are critical steps that directly impact the quality of insights derived from analysis.
• Contributing to Data Lineage and Compliance: The specialist will help track the origin and transformations of data throughout the ML pipeline to ensure traceability, understand its impact on model performance, and support compliance with data protection regulations.

Reporting & Communication

• Producing Detailed Validation Documentation and Analytical Reports: The specialist will create comprehensive records of validation processes, test cases, results, and detailed reports with insights on model strengths, weaknesses, and improvement opportunities. These reports must be standardized and suitable for diverse audiences, including senior management, executive committee members, and regulatory bodies. Data visualization software will be utilized to present findings clearly and compellingly. The emphasis on transparency and communicating to a wide range of audiences highlights that the role requires exceptional communication skills. The output of validation is not just a technical report but a tool for informed decision-making across the organization.
• Communicating Complex Findings to Diverse Audiences: The specialist will effectively convey the results of analysis, including statistical significance, model interpretability, and ethical considerations, to both highly technical and non-technical stakeholders. This involves translating complex technical concepts and data-driven insights into actionable business recommendations. The repeated emphasis on communicating complex technical concepts to non-technical stakeholders goes beyond basic communication.

Required Qualifications

Education

• Master's in a quantitative field such as Computer Science, Data Science, Engineering Statistics, Mathematics, or Machine Learning.
• A strong academic foundation in probability, linear algebra, calculus, and statistical inference is essential.

Experience

• Minimum of 5+ years of progressive experience in machine learning validation, model risk management, data science, or a closely related role with a strong focus on statistical rigor and algorithmic assessment.
• Proven track record in successfully validating and deploying AI/ML-based solutions to solve complex business problems.
• Experience designing and conducting experiments to validate model performance.

Technical Skills

• Programming Proficiency: Expert-level proficiency in Python and/or R for data manipulation, statistical analysis, and machine learning.
• Experience with SQL for querying and managing databases.
• Familiarity with C++ and/or Java for performance-critical applications or integration. The ability to work across different parts of the ML stack, from data querying to model deployment and potentially low-level optimization.
• Machine Learning Frameworks & Tools: Experience with common ML frameworks such as TensorFlow and PyTorch.
• Data Analysis & Statistical Tools: Proficiency in data analysis and statistical tools, including Python/R libraries like NumPy, SciPy, Pandas, Matplotlib, and Seaborn.
• Data Visualization Tools: Experience with data visualization software like Looker, Looker Studio to present findings effectively.

Statistical & ML Expertise

• Deep understanding of machine learning algorithms, their underlying principles, and performance metrics.
• Strong knowledge of statistical analysis, modeling techniques, and hypothesis testing.
• Familiarity with advanced statistical modeling techniques such as Bayesian Inference and Variational Inference, and their integration with AI models.

Tech Mahindra is an Equal Employment Opportunity employer. We promote and support a diverse workforce at all levels of the company. All qualified applicants will receive consideration for employment without regard to race, religion, color, sex, age, national origin or disability. All applicants will be evaluated solely on the basis of their ability, competence, and performance of the essential functions of their positions with or without reasonable accommodations. Reasonable accommodations also are available in the hiring process for applicants with disabilities. Candidates can request a reasonable accommodation by contacting the company ADA Coordinator at [email protected].