Senior Python Engineer / PySpark Data Engineer

Dice is the leading career destination for tech experts at every stage of their careers. Our client, BCforward, is seeking the following. Apply via Dice today! Job Title: Python Data Engineer Location: Jersey City, NJ Duration: Contract - 12 months Pay Range: $73.67/hr (W2) Job ID: 374052 Start: ASAP Work Arrangement: 3 days onsite required Glider: Candidate Identification Only Notes: Must include current location and intended onsite location on resume. About BCforward BCforward is a leading global IT consulting and workforce solutions firm providing services and support to Fortune 500 and government clients. Founded in 1998, BCforward has grown with our customers needs into a full-service business solutions provider. With delivery centers and offices across North America and India, we take pride in building long-term relationships and delivering excellence through innovation, collaboration, and integrity. Job Description We are seeking a Python Engineer to join our dynamic team. The ideal candidate will have strong experience in Python, PySpark/Spark, and data engineering on Cloudera/Hadoop and Databricks and a proven ability to design, implement, and optimize secure, observable, and testable data ingestion and transformation pipelines. Interview Process: • Three rounds. Some virtual and some onsite in New Jersey. Responsibilities: • Design and implement batch and streaming data ingestion and transformation jobs using Python and PySpark. • Build reusable frameworks for data quality checks, schema management, and error handling with retry logic. • Integrate pipelines with CI/CD processes and Git workflows with artifact versioning. • Apply secure coding practices, protect secrets and PII, and ensure compliance. • Tune performance using partitioning, caching, broadcast joins, and memory configuration. • Implement observability with structured logging, metrics, and tracing for distributed debugging. • Collaborate with architects and Data-Ops to deliver robust and compliant solutions with clear documentation. Required Skills & Qualifications: • Strong Python, including packaging and virtual environments. • PySpark/Spark with demonstrated performance tuning expertise. • Data ingestion for batch and streaming, schema management, error handling, and retry logic. • Test discipline across unit and integration tests, data quality assertions, and reproducible pipelines. • CI/CD using Azure DevOps or Jenkins, Git workflows, artifact versioning, and release readiness. • Experience on Cloudera/Hadoop (HDFS, Spark, Hive/Impala) and Databricks (clusters, jobs, notebooks, Delta). • Observability with structured logging, metrics, tracing, and debugging in distributed contexts. • Secure coding, secret management, PII protection, and compliance awareness. • Strong communication and collaborative work style with documentation of frameworks and patterns. • Minimum 5 years of relevant experience. Preferred Skills: • Experience with Agile ceremonies and iterative delivery. • Experience building reusable data quality and pipeline frameworks. Why BCforward? At BCforward, we believe in advancing lives and careers. When you join our team, you gain access to: • Competitive compensation and benefits. • Opportunities for growth with global clients. • A supportive, inclusive culture that values innovation and people. • Exposure to cutting-edge technologies and projects. About Our Commitment BCforward is an equal opportunity employer. We value diversity and are committed to creating an inclusive environment for all employees. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, sexual orientation, gender identity, national origin, age, disability, or veteran status. Interested? Apply Now! If this sounds like the right opportunity for you, please apply with your most recent resume. Include your current location and intended onsite location in Jersey City, NJ.