Big Data Developer

Job Tittle: Big Data Architect Duration: 06-month contract with possible extension Location: Charlotte, NC 28202 (100% Onsite Role) This role is only open for US Citizens or Green Card (Permanent Residents), Kindly do not apply if you are OPT, EAD, H1B or C2C candidates. Job Description: • As a Big Data Architect Contractor, you will support the project team by designing and implementing large-scale data solutions to meet business needs. • Design and develop scalable big data architectures to handle large volumes of data. Collaborate with stakeholders to understand data requirements and translate them into technical solutions. • Implement data integration, data processing, and data storage solutions using big data technologies. Ensure data security, data quality, and data governance standards are met. • Optimize data architectures for performance, scalability, and cost-efficiency. Responsibilities: • Design, develop and optimize ETL/ELT pipelines using Azure Data Factory (ADF) and Databricks • Write and tune PySpark / Spark SQL notebooks for large-scale data transformation • Architect end-to-end data solutions across dev → UAT → prod environments using Unity Catalog • Lead and drive design discussions with client architects and other counterparts • Collaborate with different teams on data contracts and schema agreements • Lead design and optimization of high-volume data pipeline. • Define and enforce data engineering standards — naming conventions, partitioning strategies, cluster configurations, Spark tuning • Drive performance optimization — AQE tuning, liquid clustering, broadcast joins, shuffle partition management • Design Databricks cluster policies, autoscaling configurations, and cost optimization strategies • Conduct root cause analysis on production incidents and implement permanent fixes • Mentor junior and mid-level engineers through code reviews and pair programming • Evaluate new technologies and recommend adoption (e.g., DABs, DLT, Auto Loader, Serverless Compute, event hubs) Mandatory Skills: • Python, PySpark, Spark SQL, SQL Server • Azure (ADF, ADLS Gen2, Key Vault, Azure Monitor) • Databricks (Delta Lake, Unity Catalog, Workflows) • Apache Airflow • Git / Azure DevOps • Deep Spark internals (DAG optimization, spill analysis, skew handling) • Delta Lake advanced features (time travel, deletion vectors, predictive I/O) • Unity Catalog governance (row/column security, external locations, system tables) • IaC — Terraform, Azure ARM templates EEO: “Mindlance is an Equal Opportunity Employer and does not discriminate in employment on the basis of – Minority/Gender/Disability/Religion/LGBTQI/Age/Veterans.”