Freelance Data Engineer – Remote

Join a dynamic team as a data engineering expert, responsible for designing and building robust, scalable ETL/ELT pipelines to process both structured and unstructured data from diverse sources in real-time and batch environments. You will architect and optimize data lakes and warehouses on leading cloud platforms such as AWS, GCP, or Azure, ensuring efficiency, reliability, and cost-effectiveness. This role involves close collaboration with cross-functional teams to deliver data-driven products, machine learning feature stores, and analytical datasets, while leading initiatives in data governance, quality assurance, and observability. You will also mentor junior engineers and contribute to the evolution of technical best practices and platform enhancements. Deliverables • Design, implement, and maintain scalable ETL/ELT pipelines for real-time and batch data processing • Architect and optimize cloud-based data lakes and warehouses for performance and cost efficiency • Develop and manage data models, transformation workflows, and orchestration systems (e.g., Airflow, Prefect, Dagster) • Build and maintain real-time streaming systems and internal/external APIs • Lead data governance, quality assurance, lineage tracking, and observability initiatives • Develop monitoring and alerting solutions using tools such as Datadog and Prometheus • Mentor junior engineers and contribute to technical design and best practices • Research and integrate new technologies to enhance the data platform Requirements • 6+ years of experience in data engineering or analytics infrastructure roles • Advanced proficiency in Python or Scala for data engineering tasks • Deep experience with cloud-native environments, preferably AWS • Expertise in data modeling, warehousing, orchestration, distributed computing, and CI/CD practices • Familiarity with containerization, NoSQL systems, streaming data platforms, and analytics tools • Experience with orchestration tools such as Airflow, Prefect, or Dagster • Strong understanding of data governance, quality assurance, and observability best practices • Excellent communication skills in English, with an accent similar to native speakers • Ability to work fully remotely and collaborate across time zones • Skills or interest in MLOps, infrastructure-as-code, and GenAI/LLM-based systems are a plus About Twine Twine is a leading freelance marketplace connecting top freelancers, consultants, and contractors with companies needing creative and tech expertise. Trusted by Fortune 500 companies and innovative startups alike, Twine enables companies to scale their teams globally. Our Mission Twine's mission is to empower creators and businesses to thrive in an AI-driven, freelance-first world.