Data Engineer - Microsoft

Do you love turning complex, messy information into clean, connected data that others can build on? Are you energized by creating systems that help teams discover what matters and move faster? If you enjoy working across engineering, research, and product teams to shape reliable, scalable data foundationsand youre excited by the opportunity to build something new from the ground upthis role offers the chance to make a meaningful impact. Role Overview As a Data Engineer, youll build and refine the pipelines, data models, and services that make Microsofts research content discoverable and usable across modern applications and emerging AI scenarios. Youll define the architecture and core data systems largely from scratch, creating the foundation for both web and AI-powered experiences. Youll use existing AI models and out-of-the-box Microsoft tools to turn unstructured content into structured, high-quality datanot heavy ML research, but practical, model-assisted enrichment. Youll partner with a PM and a full stack engineer while independently driving backend and data direction. The work includes an initial infrastructure lift to establish the environment, followed by iterative development as the MVP grows. The contract and working with 2A This role is an Embedded Consultant position, which means youll sit directly with Microsofts internal team to offer hands-on support. This remote, full-time contract runs for 618 months with the potential for renewal. While youll spend most of your days working directly with your Microsoft team, 2A offers multiple points of connection to support you in your role and career. Well meet with you and your manager regularly to help ensure a successful engagement, and youll have opportunities to join 2A working groups, attend community-building events, and engage in professional development. Activities Build and maintain core data pipelines • Build and maintain end-to-end ingestion pipelines for documents, datasets, code repositories, videos, transcripts, and internal knowledge sources. • Clean, normalize, structure, and store data in formats that support both web applications and AI-driven use cases. • Use out of the box Microsoft toolssuch as Fabric, Azure services, Cosmos DB, or Copilot Studioto create reliable, maintainable systems. Enrich and model research data • Use AI models to transform unstructured content into structured metadata and durable knowledge assets. • Design the architecture and foundational data systems, establishing the patterns and infrastructure for a new, scalable environment. • Develop and refine embeddings, vector indexes, and retrieval components to support semantic search and grounding scenarios. Build backend and data services • Build data services, APIs, and backend components that power internal applications and agent-supported workflows. • Iterate on systems after the initial MVP, improving reliability, performance, and scalability over time. Collaborate and translate requirements • Collaborate with a PM and full stack engineer to understand requirements and translate them into actionable data solutions. • Work cross-functionally to define data needs and align systems with downstream consumers and discovery workflows. Skills And Qualifications - Need To Have • Proven ability to design and build end-to-end data systems, from ingestion through cleaning, structuring, storage, and serving. • Experience building and shipping data products that deliver practical value. • Demonstrated impact using AI models in data workflows (applied use, not ML research). • 5+ years of software or data engineering experience, including at least 2 years of hands-on work with data pipelines. • Comfortable defining architecture and starting systems from scratch, working independently in a small cross-functional team. • Proficiency in Python, SQL, or similar languages used in data engineering workflows. Skills And Qualifications - Nice To Have • Experience with Microsoft Fabric, Cosmos DB, Azure data services, or Copilot Studio. • Background building data that supports embeddings, semantic search, or retrieval use cases. • Familiarity with metadata frameworks, taxonomies, or knowledge modeling. • Experience shaping ambiguous information into structured datasets and iterating quickly after an MVP. $155,000 - $175,000 a year This role is offered on a W2 basis, and benefits include a flexible time-off plan; 100% employer-paid medical, dental, and vision insurance; employer-paid life insurance for those enrolled in medical coverage; a 401(k) plan with company match; fertility, surrogacy, and adoption benefits; fitness and caregiver benefits; an Employee Assistance Program; and 100% employer-paid short- and long-term disability coverage. Next steps See yourself in the job description? Apply! If your skills and experience match the role, a recruiter will reach out to schedule a phone screen. About 2A Recruiting & Staffing We help tech companies build future-ready teams by connecting them with forward-thinking marketing, creative, and technical professionals. With deep client networks across the tech industry we give candidates access to roles they wont find on public job boards and the guidance to stand out. We take a people-first approachshowing up with care, responsiveness, and support that continues well past placement, backed by benefits designed to help our contract team members thrive. Were committed to anti-racism and building a diverse team by hiring individuals who bring different perspectives, and we know we still have work to do. If theres anything 2A can do to create a more comfortable or accessible application process for you, please let us know. 2A is proud to be an equal opportunity employer. Candidates from diverse backgrounds are strongly encouraged to apply. All qualified applicants will be considered without regard to race, color, religion, sex, gender identity, national origin, disability status, protected veteran status, or any other characteristic protected by law. We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.