Azure Databricks Engineer with Python & Spark - 2 Positions (Senior & Lead) - Only W2

Role: Azure Databricks Engineer with Python & Spark (Senior / Lead) Location: Weehawken, NJ (Onsite from Day 1) Job Type: W2 Contract Note: 2 positions are available - Senior & Lead Experience Level: Senior OR Lead Must Have Skills (Mandatory) • Apache Spark • Python • Azure Data Factory • Azure SQL Microsoft • Azure Synapse Analytics • ETL/ELT Solutions Nice To Have Skills • Apache Kafka • CI/CD Experience & Qualifications • Expert level understanding of Azure Data Factory, Azure Synapse, Azure SQL, Azure Data Lake, and Azure App Service • Working knowledge of Python • Expert level skills writing and optimizing complex SQL • Experience with complex data modeling, ETL design, and using large databases in a business environment • Experience with building data pipelines and applications to stream and process datasets at low latencies • Fluent with Big Data technologies like Spark, Kafka and Hive • Designing and building data pipelines using API ingestion and Streaming ingestion methods • Knowledge of Dev-Ops processes (including CI/CD) and Infrastructure as code • Experience in developing NoSQL solutions using Azure Cosmos DB • Thorough understanding of Azure and AWS Cloud Infrastructure offerings Key Responsibilities • Design and implement scalable and secure data processing pipelines using Azure Data Factory, Azure Databricks, and other Azure services • Manage and optimize data storage using Azure Data Lake Storage, Azure SQL Data Warehouse, and Azure Cosmos DB • Monitor and troubleshoot data-related issues within the Azure environment to maintain high availability and performance • Automate data pipelines and workflows to streamline data ingestion, processing, and distribution tasks • Utilize Azure's analytics services, such as Azure Synapse Analytics, to provide insights and support data-driven decision-making • Provide guidance and support for data governance, including metadata management, data lineage, and data cataloging Note: Visa Independent candidates are highly preferred