Research Intern - STAC, NYC (Sociotechnical Alignment Center)

Overview

Research Internships at Microsoft provide a dynamic environment for research careers with a network of world-class research labs led by globally-recognized scientists and engineers, who pursue innovation in a range of scientific and technical disciplines to help solve complex challenges in diverse fields, including computing, healthcare, economics, and the environment.

The Sociotechnical Alignment Center ( (STAC) is a team of researchers, applied scientists, and linguists in Microsoft Research (MSR) who represent part of the research pillar of Microsoft's Responsible AI efforts. Founded within Microsoft Research in 2022, STAC's mission is to guide the sociotechnical alignment of AI systems, with a particular emphasis on measuring risks, capabilities, performance, and other properties of these systems. STAC collaborates extensively with researchers from the Fairness, Accountability, Transparency, and Ethics in AI ( (FATE) group and other groups across Microsoft.

STAC is looking to hire several Research Interns in NYC to conduct research on measuring risks, capabilities, performance, and other properties of AI systems, with a focus on "generative" or "general purpose" systems. STAC's approach to evaluation is grounded in measurement theory from the social sciences and in statistics. All STAC Research Internships will involve collaborations and/or co-mentorship with one or more FATE researchers ( .

STAC is especially interested in candidates with expertise in theoretical and methodological approaches that can be applied to help advance and mature the field of AI system evaluation. Prior experience with AI system evaluation specifically is preferred. We especially encourage applications from candidates interested in topics including but not limited to: measurement theory from the social sciences; reliability of generative AI systems and generative AI system evaluation; validity of generative AI system evaluation; automated methods for constructing conceptual definitions for evaluation; improving human and automated annotation methods for use in evaluation, including LLM-as-a-judge; generative AI user simulations and synthetic data generation; linguistic models of conversational organization (including conversational norms, turn-taking, politeness, and other aspects of sociopragmatic competence); and applications of methodologies from psychometrics and educational testing to the design and validation of evaluations (including methods such as item response theory, factor analysis, and other latent variable models).

Applicants should have a demonstrated interest and established expertise in methods applicable to AI system evaluation, and a desire to work in a highly interdisciplinary environment. We welcome applicants with backgrounds in technical fields (e.g., machine learning, artificial intelligence, natural language processing, computer vision, statistics) and applicants with backgrounds in sociotechnical fields (e.g., linguistics, economics, human-computer interaction, information science, educational testing, psychometrics).

Applicants to the STAC internship may also be interested in applying for the separate FATE NYC internship. Applicants interested in multiple internship opportunities should apply to each posting separately.

You can upload documents by going to your profile on the career site and clicking on the Resume Manager tab in the top right of the page, and from there selecting Other Documents.

If you are having trouble submitting your application, please go to the bottom of the page and click "Support" and fill out the requested information.

Responsibilities

Research Interns put inquiry and theory into practice. Alongside fellow doctoral candidates and some of the world's best researchers, Research Interns learn, collaborate, and network for life. Research Interns not only advance their own careers, but they also contribute to exciting research and development strides. During the 12-week internship, Research Interns are paired with mentors and expected to collaborate with other Research Interns and researchers, present findings, and contribute to the vibrant life of the community. Research internships are available in all areas of research, and are offered year-round, though they typically begin in the summer.

Qualifications

Required Qualifications

• Currently enrolled in a relevant PhD program.

Other Requirements

• Research Interns are expected to be physically located in their manager's Microsoft worksite location for the duration of their internship.

• In addition to the qualifications below, you'll need to submit a minimum of two reference letters for this position as well as a cover letter and any relevant work or research samples. After you submit your application, a request for letters may be sent to your list of references on your behalf. Note that reference letters cannot be requested until after you have submitted your application, and furthermore, that they might not be automatically requested for all candidates. You may wish to alert your letter writers in advance, so they will be ready to submit your letter.

Preferred Qualifications

• Demonstrated ability to conduct original research.

• Able to collaborate effectively with other researchers and product development teams.

• Interpersonal skills, cross-group, and cross-culture collaboration.

• Ability to think unconventionally to derive creative and innovative solutions.

Applications (including reference letters) should be submitted by December 18 for full consideration, but will be accepted until all positions are filled.

Applicants must submit (1) a cover letter that follows the guidelines below; (2) their CV; and (3) 1 or 2 writing samples, published or unpublished, which may be research papers, dissertation chapters, or other research-related writing. Applicants who do not submit a cover letter may not be considered.

Your cover letter should be no more than 2 pages long. (Shorter is fine!) Please address the following points:

• Include your name, email address, and website URL.

• Briefly describe your current research, and how it connects with the focus of STAC (see Overview for a list of some of the group's current research interests).

• What kinds of methodologies have you applied in your research/are you most familiar with? (E.g., coding, statistics, large scale data analysis, theoretical/mathematical analysis, user studies, experimental design, ethnography, etc.)

• We want to learn about the kinds of research that interest you. Are there particular projects you would like to do during your internship? Recent (MSR or non-MSR) research or ideas you've come across that you are particularly interested in? What, to you, makes for a good project/what are you looking for in a project?

• All researchers in the group review every application. However, if you want to indicate which MSR NYC applied scientists or researchers you would be especially interested in working with, you may optionally do so.

You can upload documents by going to your profile on the career site and clicking on the Resume Manager tab in the top right of the page, and from there selecting Other Documents.

If you are having trouble submitting your application, please go to the bottom of the page and click "Support" and fill out the requested information.

The base pay range for this internship is USD $6,710 - $13,270 per month. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $8,760 - $14,360 per month.

Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here:

Benefits/perks listed below may vary depending on the nature of your employment with Microsoft and the country where you work.

This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.

Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations. (