Technical Requirements :
- Skilled in designing and building complex distributed systems.
- Proficient in modern batch and streaming data ecosystems.
- Expertise in Java, Scala, or Python, along with SQL and Bash.
- Capable of leading, organizing, and managing team projects.
- Experienced in project management and reporting.
- Mentorship abilities for junior team members on engineering best practices and code quality.
Responsibilities :
- Develop and oversee data platform and assist its users.
- Collaborate with Traders and Researchers to identify and integrate new data sources for analysis to enhance trading opportunities.
- Design and create data solutions for discovering, purchasing, organizing, tracking, and managing data rights and quality to meet the strategic needs of trading teams.
- Continuously monitor data ingestion pipelines and data quality to ensure system stability, reliability, and data integrity.
Also, contribute to the development and enhancement of monitoring and quality control systems and processes.
Qualifications :
- Over 7 years of experience in modern data technologies and building data-heavy distributed systems.
- Advanced proficiency in Java, Scala, or Python with a track record of producing high-quality, maintainable code.
- In-depth knowledge of SQL and Bash.
- Experience in utilizing and developing cloud-native technologies for scalable data processing.
- Previous work with both batch and streaming data systems, with a clear understanding of their limitations.
- Familiarity with various data processing technologies like Flink, Spark, Polars, Dask, etc.
- Knowledge of diverse data storage technologies such as S3, RDBMS, NoSQL, Delta / Iceberg, Cassandra, Clickhouse, Kafka, etc.
- Proficiency with various data formats and serialization systems including Arrow, Parquet, Protobuf / gRPC, Avro, Thrift, JSON, etc.
- Experienced in managing intricate data ETL pipelines using tools like Kubernetes, Argo Workflows, Airflow, Prefect, Dagster, etc.
- Prior experience in schema governance and managing schema evolution.
- Experience in developing quality control processes to identify and rectify data discrepancies.
- Motivation to mentor junior team members and uphold high standards of engineering practices and code quality.
- Strong problem-solving skills in technical contexts.
- Demonstrated ability to thrive in an agile, fast-paced environment, managing multiple tasks and projects efficiently in a trading context.
Senior Data Analytics Engineer
Key Responsibilities:Develop and maintain data models in dbt/Snowflake and create dashboards in TableauSupport and enhance existing data ingestion pipelines for the Snowflake data warehouseAnalyze and present business data to stakeholders for informed decision-makingRequired Qualifications:5+ years ...
Seen - Senior Data Engineer
In this role, you will work within a team consisting of product managers, designers, software engineers, data scientists and other data engineers to deliver new products and features. Seen is looking for a Senior Data Engineer to join one of our various cross-functional teams. Ingest data from SFTP,...
Senior Data Engineer
As a Senior Data Engineer, you will play a crucial role in designing, building, and maintaining our data infrastructure, ensuring the availability, reliability, and performance of our data systems. Are you a seasoned data engineering professional with a passion for driving innovation in data solutio...
Senior Data Engineer
We are looking for a Senior Data Engineer to help us continue to build the next evolution of our data platform in a scalable, performant, and customer-centric architecture. EquipmentShare is Hiring a Senior Data Engineer. Primary responsibilities for a Senior Data Engineer. Collaborate with Product ...
Senior Data Engineer
You will be responsible for leading a team of data engineers, designing and implementing robust data solutions, and ensuring the scalability and reliability of our data system. You bring 7 – 12+ years of experience in data engineering designing, implementing and maintaining efficient data platforms....
Senior Data Engineer
Are you passionate about leveraging cutting-edge technology to data-driven innovation? A leading trading firm in Chicago, is seeking a Senior Data Engineer to join their Data Engineering team. At this leading trading firm, you’ll be at the forefront of data engineering, working on high-impact projec...
Senior Lead Software Engineer - Pricing Reference Data
As a Senior Lead Software Engineer at JPMorgan Chase within the Corporate Technology's Reference Data Engineering division, your role is crucial to an agile team dedicated to improving, developing, and delivering top-tier technology products in a secure, stable, and scalable manner. Provide primary ...
Senior Staff Data Engineer
Educate and mentor the junior data engineers on data engineering best practices. We are looking for a Senior Staff Data Engineer on our Customer Information Management (CIM) team. You will work with SMEs, architects, analysts, data scientists, stake holders and others to build solutions that integra...
Senior Data Engineer
Center 1 (19052), United States of America, McLean, VirginiaSenior Data Engineer. We are seeking Data Engineers who are passionate about marrying data with emerging technologies. As a Capital One Data Engineer, you’ll have the opportunity to be on the forefront of driving a major transformation with...
Senior Data Engineer (Team Lead)
We are looking for a driven and dynamic Senior Data Engineer who will have the responsibility to expand and maintain our data warehouse, develop scalable data products, and help orchestrate terabytes of data flowing through the Lasso platform. This individual will work directly with the Data Archite...