Description
WHAT YOU DO AT AMD CHANGES EVERYTHING
We care deeply about transforming lives with AMD technology to enrich our industry, our communities, and the world. Our mission is to build great products that accelerate next-generation computing experiences – the building blocks for the data center, artificial intelligence, PCs, gaming and embedded. Underpinning our mission is the AMD culture. We push the limits of innovation to solve the world's most important challenges. We strive for execution excellence while being direct, humble, collaborative, and inclusive of diverse perspectives.
AMD together we advance_
THE ROLE
- To lead the team's pretraining efforts for our open source, multilingual language model pretraining, supporting the expansion of open-source AI support for long tail and lower resource languages.
- Plan and execute efficient language model pretraining while managing architecture risk; specific need for experience and interest in efficient training techniques like MoE and fine-grained MoE .
- Provide technical leadership and drive team's approach to model architecture, stay up to date on current relevant research, experimentally evaluate new techniques, and drive or implement feature work to support training desired architecture.
- This team is crucial in building the role of Silo AI as a thought leader and foreground actor in developing and sharing multilingual open-source language models.
Collaboration with others
- This role will provide technical leadership for our model pretraining efforts, coordinating and directing work related to model architecture
- Provide input and collaborate with data teams on data pretraining mixes, synthetic data, mid-training data, etc.
- Provide feedback to and coordinate efforts with OpenEuroLLM model architecture work package.
Main goals for first 6 months
- Establish a research and development roadmap which involves experimental work applied to the current model building activities in the Base Models team, with an emphasis on efficient training techniques and advanced model architectures.
- Coordinate efforts with OpenEuroLLM architecture research workstreams.
- Publish position papers and white papers, release first new experimental models.
THE CANDIDATE
- Experience training large language models, especially MoE models.
- Comfort working as a “full stack” researcher, familiarity with the full model training stack.
- A record of achieving engineering successes and of research or research adjacent activities, such as. publications, patents, or open source releases; academic or industrial collaborations.
- A clear and documented commitment to open source excellence and software craftmanship.
- A willingness to take technical risks and meet technical challenges head-on.
- Experience from industrial research labs and of software engineering processes.
- Demonstrate communication and leadership skills.
We would like to see
- Mathematical and statistical competence beyond the expected for computer science engineers.
- Understanding of linguistic behavior and linguistic data.
- Working knowledge of more than one language.
- Research experience, e.g. a postgraduate degree in a relevant field.
#LI-DB1
#LI-HYBRID
Benefits offered are described: AMD benefits at a glance.
AMD does not accept unsolicited resumes from headhunters, recruitment agencies, or fee-based recruitment services. AMD and its subsidiaries are equal opportunity, inclusive employers and will consider all applicants without regard to age, ancestry, color, marital status, medical condition, mental or physical disability, national origin, race, religion, political and/or third-party affiliation, sex, pregnancy, sexual orientation, gender identity, military or veteran status, or any other characteristic protected by law. We encourage applications from all qualified candidates and will accommodate applicants' needs under the respective laws throughout all stages of the recruitment and selection process.
Apply on company website
Find Connections via Linkedin
