Senior PySpark Engineer

Senior PySpark/Databricks Data Engineer Location: Jersey City, NJ (Hybrid – 3 Days Onsite) Required Skills • 8+ years of Data Engineering experience. • Strong hands-on experience with Python, PySpark, Spark SQL, and Azure Databricks. • Experience building batch and real-time data pipelines on Azure. • Strong expertise with Azure Data Factory (ADF) and Azure Data Lake Storage (ADLS). • Experience with Azure SQL Database, SQL, T-SQL, and Data Warehouse concepts. • Hands-on experience with Kafka and Azure Event Hub for streaming data. • Experience with Azure services including Data Factory, Data Lake, Logic Apps, Azure Functions, and Data Analytics. • Strong experience in ETL development, data integration, and data migration. • Expertise in Databricks performance tuning and Spark optimization. • Experience with Git/GitHub for source code management. • Familiarity with Agile/Scrum methodology and Jira. • Strong analytical, troubleshooting, and problem-solving skills. Nice to Have • Experience developing reusable data ingestion frameworks. • Experience building materialized views, streaming pipelines, and API integrations. • Strong understanding of enterprise data architecture and cloud-native data platforms. • Excellent communication and stakeholder collaboration skills.