Senior Data Engineer

Title : Senior Data Engineer Location : Charlotte NC, Dallas TX or Malvern PA Note: This role requires candidates to attend an onsite interview in Charlotte, NC; Dallas, TX; or Malvern, PA. Job Description Key Skills: Python SQL AWS – Lambdas, Glue/Pyspark, ECS Qualifications: • Proficiency in Python programming • Strong expertise in SQL, Presto, HIVE, and Spark • Knowledge of trading and investment data • Experience in big data technologies such as Spark and developing distributed computing applications using PySpark • Experience with libraries for data manipulation and analysis, such as Pandas, Polars and NumPy • Understanding of data pipelines, ETL processes, and data warehousing concepts • Strong experience in building and orchestrating data pipelines • Experience in building APIs Write, maintain, and execute automated unit tests using Python • Follow Test-Driven Development (TDD) practices in all stages of software development • Extensive experience with key AWS services/components including EMR, Lambda, Glue ETL, Step Functions, S3, ECS, Kinesis, IAM, RDS PostgreSQL, Dynamodb, Timeseries database, CloudWatch Events/Event Bridge, Athena, SNS, SQS, and VPC • Proficiency in developing serverless architectures using AWS services • Experience with both relational and NoSQL databases • Skills in designing and implementing data models, including normalization, denormalization, and schema design • Knowledge of data warehousing solutions like Amazon Redshift • Strong analytical skills with the ability to troubleshoot data issues • Good understanding of source control, unit testing, test-driven development, and CI/CD • Ability to write clean, maintainable code and comprehend code written by others • Strong communication skills • Proficiency in data visualization tools and ability to create visual representations of data, particularly using Tableau Note:This role requires candidates to attend an onsite interview in Charlotte, NC; Dallas, TX; or Malvern, PA.