Senior AWS Data Engineer

• • NO C2C • • Purpose: • Establish data feeds to the digital platform and manage ETL processes. Design raw file data structures to be used in data science applications. Associated commercial data includes data required for forecasting, analysis, optimization modeling and data visualization. • Work with Data Analysts to ensure enterprise data is cataloged in accordance with data governance standards. • Develop system test and integration plans, execute test procedures, support user community, and obtain approvals for change management. • Provide hands-on technical configuration, application development, integration and testing. Implement cloud data integration strategies between cloud providers. • Provide technical expertise to the business and internal IT with respect to digital solutions, including data visualization and facilitate business expansion with solutions to scale, managing the critical input and output data. • Design, develop, and maintain data solutions that leverage the power and flexibility of AWS Requirements • Knowledge of energy market data. • In-depth knowledge of digital platforms such as AWS, Dataiku and Databricks. • In-depth proven software engineering experience using Python and machine learning knowledge. • Designing and building scalable data architectures on AWS, with strong experience using AWS data storage and processing services such as S3, EMR, Data Lake Formation, AWS Glue, and Amazon Athena. • Experience in working with DevOps tools and methodologies such as Git, Jenkins, Docker, or Kubernetes. • Proficiency in programming languages such as Python, Java, Scala, SQL, Spark, and Unix shell scripting, including AWS CLI. • Automating data processing workflows using AWS services, AWS CLI, APIs, and scripting languages like Python, Java, and/or Scala. • Experience with SOAP/RESTful and GraphQL APIs using AWS services and frameworks and familiarity with other cloud services such as Snowflake. • Knowledge of ETL tools and frameworks such as Apache Spark, Airflow, AWS IoT Analytics, IoT Device Management, and AWS IoT Events. • Familiarity with database technologies such as SQL, NoSQL, and data warehousing concepts. Working knowledge of Amazon Aurora, DynamoDB, and Amazon RDS. • Understanding of data security, privacy, and compliance requirements. • Experience with data modeling, metadata management, and data lineage tracking. • Strong analytical, problem-solving, and communication skills. • Current data visualization tool experience such as Power BI Service and Desktop. • Prior design and implementation experience with data warehouses. • Expert knowledge of SQL databases, SQL queries, Amazon Athena, SQL Integration Services, SQL Reporting Services and data formats such as XML and JSON. • Knowledge of ISO’s and major market activities preferred. Education/Experience • Bachelor’s degree in a technical field or 15+ software development experience in lieu of degree. • 10-15years of information technology, systems, and application development experience. • Minimum 5 years’ experience supporting the needs of an Energy Trading or Asset Management is business preferred • Proficient with Microsoft Office products (Excel, Word, Visio, PowerPoint) • Proficient with Windows and Linux operating systems. • ISO Market systems knowledge preferred. • Modern cloud technologies used for managing data (AWS is highly preferred).