Senior Data Engineer

Responsibilities Job Description: • Work closely with Subject Matter Experts (SMEs) to design and develop the ontology (data model) and data pipelines. • Design, implement, and maintain robust data pipelines and ETL processes. • Develop and optimize cloud-based data storage and processing solutions. • Collaborate with data analysts and data scientists to understand data requirements and deliver high-quality data solutions. • Ensure data integrity and security across all data workflows and storage solutions. • Monitor and troubleshoot data pipelines, addressing any issues promptly to ensure the smooth flow of data. • End to End data pipeline development – responsible for using Python and Pyspark to perform ETL and derive new datasets necessary for business applications, Minimum Qualifications • Bachelor's Degree in Computer Science, Engineering, or related field or related experience. • 5+ years of experience as a data engineer. • Hands-on experience with building data pipelines • Strong SQL skills and experience working with large datasets and complex data structures. • Proficient in Python, and Pyspark • Experience in Typescript (preferred) or Javascript • Excellent problem-solving and analytical skills with a strong attention to detail. Preferred Qualifications • Experience with Palantir Foundry application development and data visualization tools • Knowledge with commercial visualization tools such as Tableau or Power BI. • Experience in Typescript (preferred) or Javascript • Databases – familiarity with common relational database models and proprietary instantiations, such as SAP, Salesforce etc. • Git – knowledge of version control / collaboration workflows and best practices Agile – familiarity with agile and iterative working methodology and rapid user feedback gathering concepts • UX design – knowledge of best practices and applications Skills: design,pipelines,python,data,etl,cloud