Sr Data Engineer

Title: Sr Data Engineer Location: NYC, NY(Hybrid) Duration: 12-24+ Months This is a highly hands-on backend engineering role within a lean engineering team, where senior engineers are expected to take ownership of problems, design practical solutions, and implement them end-to-end. Bloomberg will deliver new market data directly into client's Snowflake environment through Snowflake Direct Share; however, the shared data is overwritten daily and does not retain historical records. This engineer will be responsible for designing and building a new historical Bloomberg data platform in Snowflake that will ultimately replace portions of the existing legacy Sybase environment, which is currently populated primarily through SFTP-based ETL processes. The role requires a highly motivated, organized, and experienced engineer with strong ownership mentality, disciplined software development practices, and a focus on clean design, reliability, and operational stability. This individual will: • Use Snowflake native warehousing capabilities to design and build a historical Bloomberg data archive and expose it securely to downstream consumers using Snowflake RBAC. • Work on Per Security Data License request files and convert them to SQL scripts that utilize the Direct Share data directly. • Work closely with various Ops and Tech stakeholders to rationalize existing usage of Bloomberg data for various use cases to find redundancy and duplication. • Create and evolve Snowflake schemas to replace legacy Sybase structures while ensuring a smooth transition for users and dependent applications. • Use DBT (Data Build Tool) when transformations are required between raw ingestion layers and curated downstream datasets. • Use Python when scripting or automation is needed to support the platform. • Respond to end-user requests and troubleshoot issues related to data access, latency, performance, and operational support. Key Responsibilities • Design and build a historical data archive in Snowflake sourced from Bloomberg Direct Share feeds. • Organize incoming Bloomberg data into scalable and well-structured schemas with appropriate date and time versioning to support historical analysis and downstream consumption by Trading, Operations, and Middle Office teams. • Work with other teams to understand their current Bloomberg data consumption patterns and optimize them for performance and cost. • Re-architect existing pipelines and develop new inbound and outbound data integration processes for partner and vendor data consumption. • Coordinate infrastructure design, configuration, and deployment activities in collaboration with impacted technology teams. • Develop synchronization and reconciliation pipelines to support Snowflake data warehousing and ensure accessibility from downstream platforms and databases. • Participate in a collaborative agile engineering environment and contribute to continuous improvement of engineering and operational practices. Desired Candidate Profile • Experience working within teams responsible for acquiring, ingesting, and managing Bloomberg market data. • Strong understanding of relational database technologies within financial institutions. • Strong backend database development experience using SQL in the context of financial products and market data platforms. • Ability to work effectively both independently and as part of a team. • Strong verbal and written communication skills. • 10+ years of development experience with the ability to operate in fast-paced environments and deliver solutions with quick turnaround. Required Skills • Hands-on backend engineering experience working with Bloomberg market data within a financial institution. • Experience using Bloomberg Request and Response files used in Bloomberg Per Security Data License. • Advanced SQL expertise across multiple relational databases and warehouses including Snowflake, Sybase, SQL Server, and Oracle, particularly for complex joins, reconciliations, and historical data analysis. • Hands-on experience with ETL techniques and processes, including identifying and resolving gaps, overlaps, and inconsistencies in market data feeds. • Prior experience partnering with cross-functional Operations and Technology teams to evaluate current market data usage, rationalize overlapping datasets and workflows, and improve efficiency, performance, and cost management across the platform. • Knowledge of best practices for code quality, testing, and performance optimization. • Broad understanding of the systems and business processes supporting Middle Office and Back Office functions within investment banks or asset managers. Desired Skills • Experience with Snowflake, including building historical data archives and designing schemas for time-series financial market data. • Experience with Azure cloud architecture (nice to have; AWS or GCP background acceptable). • Experience with DevOps practices including GitLab CI/CD pipelines (both YAML and UI-based configurations). • Experience with DBT (Data Build Tool). • Strong understanding of AI / ML concepts and implementations, including exposure to AI coding tools such as Claude Code or GitHub Copilot. • Experience with Jira and agile frameworks. • Experience with Linux and shell scripting.