Data Engineer

As a Data Engineer, you will play a critical role in designing, building, and maintaining data pipelines and infrastructure to support our organization's analytics and reporting needs by converting raw data into usable information. You will work closely with business data lead, bi engineers and business stakeholders to ensure data quality, consistency, security, accessibility and solution performance. ESSENTIAL RESPONSIBILITIES Data Pipeline Development: Outstanding analytical and programming skills. Design and implement scalable complex data pipelines to extract, transform, and load (ETL) data from various sources (e.g., databases, APIs, files). Optimize data pipelines for performance and efficiency. Develop automated data ingredients, mock data generation and validation processes. Maintain source control hygiene (branch protection, git-flow) CI/CD that includes build, test and automated deployment (like Jenkins, Travis, DevOps) Database Design and Tuning: Ensure referential integrity and query optimization. Design and implement scalable complex data pipelines to extract, transform, and load (ETL) data from various sources (e.g., databases, APIs, files). Optimize data pipelines for performance and efficiency. Data Infrastructure Management: Manage and maintain data warehouses, data lakes, and other data storage solutions. Ensure data security, privacy, and compliance with relevant regulations. Monitor system performance and troubleshoot issues. Data Modeling: Create and maintain data models that accurately represent the organization's business processes and data entities. Implement data governance policies and standards. Data Quality Assurance: Develop and implement data quality checks and validation procedures. Identify and resolve data quality issues. T Tool Selection and Implementation: Evaluate and select appropriate data engineering tools and technologies. Implement and manage data engineering platforms. Collaboration: Collaborate with business data lead, project managers and business stakeholders to understand their data requirements. Provide technical guidance and support for data-driven initiatives. Participating in agile software planning and development activities including daily standups, user story and task organization and grooming activities, and effort estimation Analyze business and technical requirements to develop documentation, designs, code and tests Willingness to jump where needed, learn new tech, and work effectively across full stack QUALIFICATIONS Preferred Azure Fabric experience and Azure cloud expertise. Significant experience working with relational database technologies such as Oracle & SQL Server and No-SQL databases. Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases Experience in performance tuning of data pipelines and queries. Experience working with common languages and frameworks such as python and pyspark. In-depth understanding of modern application design principles, DevOps & microservices Strong analytical, quantitative and problem-solving abilities Strong written and verbal communication skills including ability to convey proposed solutions through natural language, diagrams, and design patterns High school diploma or GED required. Bachelor's degree preferred. Valid Driver’s License.