Sr. Data Engineer - Databricks (Toronto)

Working with one of our top financial clients, this role calls for a Sr. Data Engineer - Databricksspecializing in Databricks and Data Platform Migration. This position offers a compelling chance to work on a large-scale initiative to transition trading and market data from legacy systems into modern, scalable data frameworks, supporting strategic modernization efforts within Capital Markets.

Responsibilities

• Design and develop data ingestion pipelines, both batch and real-time, leveraging Databricks platform.
• Build and optimize Spark/Scala-based data processing workflows for high performance and reliability.
• Implement streaming ingestion solutions using Kafka, APIs, or similar technologies.
• Develop medallion architecture models (bronze, silver, gold layers) consistent with best practices.
• Create and maintain materialized views and other data structures to facilitate productive data consumption.
• Ensure data quality...