New

Principal Group Applied Science Manager, Online Experimentation - CoreAI

Microsoft
$163,000.00 - $296,400.00 / yr
United States, Washington, Redmond
Feb 14, 2026
Overview CoreAI is at the forefront of Microsoft's mission to redefine how software is built and experienced. Weare responsible forbuildingthe foundationalplatforms, services, programming models, and developer experiences that power the next generation of applications using Generative AI. Our work enables developers and enterprises to harness the full potential of AI to create intelligent, adaptive, and transformative software. As part of CoreAI, theExP teamis focused onfurtheringMicrosoft'sindustry-leadingExperimentation platformtosupport the rapidlychanginggenerativeAI landscapeandmeet thedemandforagile,trustworthy experimentationatscale.AsanAppliedScience managerwithin ExPyou willlead ourExperimentationscienceteam,pushingplatforminnovation andworkingcloselywith teams across the company togrowtheir experimentationthroughputandproductlearnings. In this role, you will: Push the envelope ononline experimentationmethodologyandagent evaluations, advancing thestate-of-the-art by connectingproduct developmentneeds, latest sciencetechniquesandplatformopportunities Contribute to one of the highest-scale experimentation platforms on the planet Collaborate closely with backend engineers, data scientists, site reliability engineers (SREs), and product managers to gather requirements, iterate on features, and deliver seamless, end-to-end user experiences. Collaborate closely with designers,UI-experts, scienceleadersand engineers to ensure clear, actionable dashboards and visualizations that empower users to make trustworthy science-informed decisions. Microsoft's mission is to empower every person and every organization on the planet to achieve more. Asemployeeswe come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond. Responsibilities Shapes and propels Microsoft's industry-leading online experimentation platform, driving breakthrough advancements in product-development tooling and publishing impactful articles and research toestablishintellectual property and attract andretaintalent Serves as a technical visionary and thought leader, positioning the ExP Platform to streamline product learning through innovative, research-backed approaches to enable trustworthy, agile, and data-driven decision making on a global scale. Provides expert technical guidance and deep industry insight, advising cross-functional teams on the adoption ofcutting-edgetools, technologies, and methodologies toensure successful experimentationprograms anduphold the highest standards of quality and scientific rigor. Ensures teams develop and apply advanced subject matterexpertiseacross a diverse set of specialized tools and methods, driving measurable business outcomes andelevating the scope and impact of Microsoft's experimentation initiatives. Leverages an extensive professional network and personalexpertiseto attract exceptional science and engineering talent to Microsoft,identifyemerging technology opportunities, and build new areas of application through strategic recruitment and talent development. Qualifications Required: Bachelor's Degree in Statistics, Econometrics, Computer Science, Electrical or Computer Engineering, or related field AND 8+ years related experience (e.g., statistics, predictive analytics, research) OR Master's Degree in Statistics, Econometrics, Computer Science, Electrical or Computer Engineering, or related field and AND 6+ years related experience (e.g., statistics, predictive analytics, research) OR Doctorate in Statistics, Econometrics, Computer Science, Electrical or Computer Engineering, or related field AND 5+ years related experience (e.g., statistics, predictive analytics, research). OR equivalent experience. 5+ yearsof peoplemanagement experience. Preferred: Master's Degree in Statistics, Econometrics, Computer Science, Electrical or Computer Engineering, or a related field and 12+ years of related experience (e.g., statistics, predictive analytics, research), or Doctorate in Statistics, Econometrics, Computer Science, or a related field and 8+ years of related experience, or equivalent experience. 5+ years of experience presenting research or technical work at conferences or industry events as an invited speaker. 7+ years of experience designing, developing, and deploying products or systems across multiple stages of the product lifecycle, from ideation and experimentation through production deployment and iteration. Experience designing, building, and scaling online experimentation platforms that support highvelocity A/B and multivariate testing across complex, multiteam product ecosystems, including governance models, experimentation guardrails, and causal inference best practices. Experience leading product measurement strategy, including defining northstar metrics, architecting endtoend measurement frameworks, and influencing senior product and engineering leaders on interpreting ambiguous or noisy data at scale. Experience evaluating and benchmarking large language models and autonomous agents, spanning offline and online evaluation methodologies, safety and reliability measurement, and translating model and agentquality signals into productlevel decisions. Applied Sciences M6 - The typical base pay range for this role across the U.S. is USD $163,000 - $296,400 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $220,800 - $331,200 per year. Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here: https://careers.microsoft.com/us/en/us-corporate-pay This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled. Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.