Senior Data Engineer

Job Number: 25-04370 Want to be part of the Energy Industry? ECLARO is looking for a Senior Data Engineer for our client in White Plains, NY. ECLARO's Client is America's largest state power organization and is a national leader in energy efficiency and clean energy technology. If you’re up to the challenge, then take a chance at this rewarding opportunity! Project Overview: • Responsible for managing critical Business-As-Usual (BAU) services that support enterprise-wide data operations. • These services include the development, maintenance, and monitoring of data pipelines, integrations, and reporting infrastructure that are essential for ongoing business functions. Pay Rate: $60.00 - $70.00/Hour Responsibilities: • Maintaining and troubleshooting ETL workflows (Pentaho, Databricks, ADF). • Supporting daily data loads and ensuring data availability for business reporting. • Responding to ad-hoc requests from business users. • Coordinating with DBAs and application teams for incident resolution. • Performing enhancements to support evolving business data needs These BAU services are essential for keeping business operations running smoothly and delivering timely insights across multiple departments. • ETL & Data Integration: • Design, develop, and optimize ETL pipelines using Azure Databricks, ADF, and Pentaho to support enterprise data workflows. • Implement and maintain data movement, transformation, and integration across multiple systems. • Ensure seamless data exchange between cloud, on-prem, and hybrid environments. • Work with Globalscape FTP for secure file transfers and automation. • API Development and Integration: • Develop, consume, and integrate RESTful and SOAP APIs to facilitate data. • Work with API gateways and authentication methods such Oauth, JWT, certificate, and API keys. • Implement and optimize API-based data extractions and real-time data integrations. • Data Quality & Governance: • Implement data validation, cleansing, and enrichment techniques. • Develop and execute data reconciliation processes to ensure accuracy and completeness. • Adhere to data governance policies and security compliance standards. • BAU Support & Performance Optimization: • Troubleshoot and resolve ETL failures, data load issues, and performance bottlenecks. • Optimize SQL stored procedures and complex queries for better performance. • Support ongoing enhancements and provide operational support for existing data pipelines. • Collaboration & Documentation: • Work closely with Data Analysts, Business Analysts, and stakeholders to understand data needs. • Document ETL processes, data mappings, and workflows for maintainability and knowledge sharing. • Provide guidance and best practices to ensure scalability and efficiency of data solutions. Required Qualifications: • 7+ years of experience in ETL development, data integration, and SQL scripting. • Strong expertise in Azure Databricks, ADF (Azure Data Factory), and Pentaho. • Experience handling secure file transfers using Globalscape FTP. • Hands-on experience in developing and consuming APIs (REST/SOAP). • Experience working with API security protocols (OAuth, JWT, API Keys, etc.,). • Proficiency in SQL, stored procedures, performance tuning, and query optimization. • Understanding of data modeling, data warehousing, and data governance best practices. • Hands-on experience with cloud-based data platforms (Azure/AWS) is a plus. • Strong problem-solving skills, troubleshooting abilities, and ability to work independently. • Excellent communication skills and ability to work in a fast-paced environment. Preferred Qualifications: • Experience working in large-scale enterprise data integration projects. • Knowledge of Python, PySpark for big data processing. • Familiarity with CI/CD for data pipelines (Azure DevOps, GitHub Actions, etc.). If hired, you will enjoy the following ECLARO Benefits: • 401k Retirement Savings Plan administered by Merrill Lynch • Commuter Check Pretax Commuter Benefits • Eligibility to purchase Medical, Dental & Vision Insurance through ECLARO If interested, you may contact: Eileen Sares esares@eclaro.com 646-755-9301 Eileen Sares | LinkedIn Equal Opportunity Employer: ECLARO values diversity and does not discriminate based on Race, Color, Religion, Sex, Sexual Orientation, National Origin, Age, Genetic Information, Disability, Protected Veteran Status, or any other legally protected group status, in compliance with all applicable laws.