Data Engineer

Pay Rate: $77.00 - $82.00 Negotiable based upon years of experience Global Financial Firm located in IRVING, TX has an immediate contract opportunity for an experienced Data Engineer Responsibilities • Partner with project managers, business stakeholders, and senior managers to translate complex business requirements into technical solutions. • Drive cross-functional collaboration to ensure seamless integration of functions. • Identify and implement necessary system enhancements for new products and process improvements. • Lead the resolution of high-impact problems through the evaluation of business processes and system architectures. • Serve as a subject matter expert in application programming and ensure adherence to architectural blueprints. • Provide mentorship and technical guidance to mid-level developers and analysts. • Design, develop, and maintain data pipelines using PySpark. • Collaborate with data scientists and analysts to understand data requirements. • Optimize Spark jobs for performance and efficiency. • Implement data quality checks and ensure data integrity. Requirements • 6-10 years of experience in systems analysis and programming of software applications. • Strong proficiency in Java application technologies, including TDD, Spring framework, and Microservices architecture. • Extensive hands-on experience with PySpark and Python programming skills. • Experience with Big Data ecosystems, including Cloudera and/or Data Bricks. • Proficiency in SQL and experience with relational and non-relational databases. • Experience with distributed query engines like Starburst (Trino/Presto). • Proficient in designing and managing workflows using scheduling tools like Apache Airflow. • Experience in RESTful API development and integration. • Experience with job scheduling and Linux environments. Preferred Skills • Experience with AI development tools. • Interest in the financial services industry. • Experience in real-time data streaming and processing using PySpark Structured Streaming. • Knowledge of machine learning concepts and MLOps practices. • Familiarity with data visualization tools like Tableau or Power BI.