Job Descriptions : -
As a Data Engineer, you will be responsible for the development of complex data sources and pipelines into our data platform (i.
e. Snowflake) along with other data applications (i.e. Azure, Terraform, etc.), automation and innovation.
Primary Responsibilities :
Create & maintain data pipelines using Azure & Snowflake as primary tools
Create SQL Stored procs and Functions to perform complex transformations
Understand data requirements and design optimal pipelines to fulfil the use-cases
Creating logical & physical data models to ensure data integrity is maintained
Code management, CI / CD pipeline creation & automation using GitHub & GIT Actions
Tuning and optimizing data processes
Design and build best in class processes to clean and standardize data
Code Deployments to production environment, troubleshoot production data issues
Modelling of big volume datasets to maximize performance for our Business Intelligence & Data Science Team
Qualifications
Required Qualifications :
Computer Science bachelor's degree or similar
Min 1-4 years of industry experience as a Hands-on Data engineer
Excellent communication skills Verbal and Written
Excellent knowledge of SQL
Excellent knowledge of Azure Services such as Blobs, Functions, Azure Data Factory, Service Principal, Containers, Key Vault, etc.
Excellent knowledge of Snowflake - Architecture, Features, Best practices
Excellent knowledge of Data warehousing & BI Solutions
Excellent Knowledge of change data capture (CDC), ETL, ELT, SCD etc.
Hands on experience on the following technologies :
o Developing data pipelines in Azure & Snowflake
o Writing complex SQL queries
o Building ETL / ELT / data pipelines using SCD logic
o Query analysis and optimization
Analytical and problem-solving experience applied to a Big Data datasets
Data warehousing principles, architecture and its implementation in large environments
Experience working in projects with agile / scrum methodologies and high performing team(s)
Knowledge of different data modelling techniques such as Star Schema, Dimensional models, Data vault is an Advantage
Experience in code lifecycle management and repositories such as GIT & GitHub
Exposure to DevOps methodology
Good understanding of Access control and Data masking
Preferred Qualifications :
Knowledge and experience on Terraform, CI CD Pipelines and automation is an advantage
Automation and orchestration using ADF
Create real-time analytics pipelines using Snowpipe Streaming
Experience developing optimized data models for Viz tool E.g. Tableau, PowerBI is an Advantage
Exposure and experience in other programming language such as Python, Spark etc. is an advantage
Hands-on experience of CI CD Pipelines using GIT & GIT Actions
Understanding of United States Healthcare data and applicable regulations
L!-CEIPAL
Data Engineer II
JobPosting","title":"Data Engineer II","datePosted":"2024-04-15T00:00:00","validThrough":null,"description":"Data Engineer II (Multiple Openings), Project Management Institute, Inc. Data Engineer II (Multiple Openings), Project Management Institute, Inc. The position requires a minimum of a Bachelor...
Actuarial Specialist, Actuarial Data Engineer (Remote Consideration)
Lincoln Financial Group Actuarial Specialist, Actuarial Data Engineer (Remote Consideration). Work with actuarial and data science software including MG-ALFA, Axis, Prophet, Dataiku, R, and Tableau. Undergraduate degree in Actuarial Science, Applied Mathematics, Statistics, Economics, Computer Scien...
ANR Data Engineer - Juniper
Message queuing, stream processing, and highly scalable ‘big data’ data stores; manipulating, processing, and extracting value from large, disconnected datasets; manipulating structured and unstructured data for analysis; constructing complex queries to analyze results using databases or in a data p...
Data Protection / Backup (CommVault) Systems Engineer
CommVault Backup Systems Engineer. The Enterprise End User Technologies (EEUT) division of the EOUSA utilizes the General Integration (GI) technical team to provide information technology technical solutions support with systems and network engineering, project coordination, scheduling, backup/recov...
Data Scientist / Engineer
We’ve assembled an outstanding team of software and data engineers, data analysts, project managers, and data coordinators to tackle difficult problems and provide excellent support to our partners at businesses, non-profits, federal and state agencies, and universities. Work with data librarians to...
Lead GenAI LLM Chatbot Engineer (Software/Data Eng. Space)
Job Title: Lead GenAI LLM Chatbot Engineer (Software/Data Engineering Space)<br /> Location: Fremont, CA / Remote</b><br /> <b>Duration: Contract<br /> Interview: Webex </b></p> <p><b>Job Description:</b></p> <p><b>Positi...
Decision Support Analyst (Data Engineer)
Performs modeling of data sources and flows, works with data management staff to define an operational framework that will define, develop and implement policies and security frameworks to ensure precise and secure delivery of knowledge and information. Perform modeling of data sources and flows, wo...
Senior Data Engineer
Build the infrastructure required for optimal extraction, transformation, and loading (ETL) of data from a wide variety of data sources using SQL, Azure, and AWS technologies Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational ef...
Analytic Data Engineer – Databricks
This role within the 'Data Engineering & Self-Service Products' team involves architecting and engineering analytic data solutions, including designing and developing data marts in Databricks using PySpark or Spark SQL, building interactive Power BI dashboards to visualize KPIs and trends, and creat...
100% Remote Data Engineer With Strong PostgreSQL and Tableau server
Data Management: Extract, transform, and load (ETL) data from multiple sources, primarily using PostgreSQL databases, ensuring data accuracy, performance, and scalability. Ideally this person will have experience working with data center infrastructure data (racks, devices, GPUs, power utilization) ...