Sr Data Engineer

Job Summary :

As a Sr. Data Engineer, you will create and maintain optimal data pipeline architecture, Infrastucture automation(IAC) and also will assemble large, complex data sets that meet functional / non-functional business requirements across Data Solutions.

You will also identify, design, and implement internal process improvements : automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability etc.

Implement foundational and advanced analytics methodologies to address key metrics to improve customer experience, network performance, quality of service, profitability of products, and personalized offerings

Delivering technical functionality developed against the Global Lockton Cloud Data Platform's with a Product Mindset.

Use data, analysis, and automation to generate value within the company through advanced data modeling and other advanced analytics principles

Building the infrastructure required for optimal ETL / ELT of data from a wide variety of data sources using Cloud Technologies and OnPrem.

Responsible for helping an organization deliver high data availability and quality throughout the entire data life cycle from ingestion to end products : dashboards, machine learning models, and production datasets.

The Successful Applicant

Technical Knowledge, Key skills and experience required for the successful candidate :

Hands-on experience with Azure services such as Azure Data Factory, Azure Databricks, ADLS-Delta lake, Azure functions, Logic aps, Synapse, Azure Logic Apps, Azure Storage, Azure Search, Purview, Azure SQL server, Scala / Spark / SQL / PYTHON etc.

Azure Database development experience with Infra as a Code, Azure CI / CD, Azure Classic Pipelines, Azure YAML, ARM / Terraforms / Ansible, AKS Maintenance, Azure ML, Pipeline integrations, Azure PaaS and IaaS, Azure Infra Experience (Spinning VM's, Databases and Cosmos)

Expertise in Data warehousing - data modelling, relational and Dimension modelling, Data quality, data catalog and Data Governance

Design and build highly scalable data pipelines using new generation tools and technologies like Spark, Kafka to induct data from various systems

Experience with data security and data access controls and design across digital transformation / cloud migration programs.

Deep understanding of relational as well as NoSQL data stores, methods and approaches (star and snowflake, dimensional modelling)

Ensure proper requirement traceability and Quality delivery during development and integration testing

A strong background in the Microsoft SQL Server stack (including SSIS, SSRS, Power BI, and SSAS).

Ability to build processes supporting data transformation, data structures, metadata, dependency, and workload management.

Strong analytic skills related to working with structured and unstructured datasets

Should support the Production live application by maintaining 100% Data accuracy and data availability in all 24

• 7.

Requirements :

Soft Skills Required

Academics

Career progression :