Python Engineer

Job Title : Python Engineer

Location : Irving , TX / Tampa,FL / Jersey City , NJ(onsite)

Job Type : Full time

Job Description

Must Have Technical / Functional Skills

Design , build and maintain scalable pipelines using Python / Databricks

Leverage Spark and SQL to process and transform large scale datasets

Develop and optimize ELT / ETL processes for high volume of data workflows

ETL : Hands on experience of building data pipelines. Proficiency in data integration platforms such as Apache Spark or Talend

Big Data : Experience of 'big data' platforms such as Hadoop, Hive or Snowflake for data storage and processing

Data Warehousing & Database Management : Understanding of Data Warehousing concepts, Relational Oracle database design

Data Modeling & Design : Good exposure to data modeling techniques; design, optimization and maintenance of data models and data structures

Languages : Proficient in any programming languages commonly used in data engineering such as Python or Scala

DevOps : Exposure to concepts and enablers - CI / CD platforms, version control systems (e.g. GIT), automated quality control management

Data Quality & Controls : Exposure to data validation, cleansing, enrichment and data controls

Roles & Responsibilities

Design , build and maintain scalable pipelines using Python / Databricks

Leverage Spark and SQL to process and transform large scale datasets

Develop and optimize ELT / ETL processes for high volume of data workflows

ETL : Hands on experience of building data pipelines. Proficiency in data integration platforms such as Apache Spark or Talend

Design, develop, and maintain ETL processes to extract, transform, and load data from various sources into our data warehouse.

Write complex SQL queries and PL / SQL scripts to perform data manipulation, validation, and transformation.

Develop and maintain data pipelines using Python and related libraries.

Optimize ETL processes and data pipelines for performance and scalability.

Collaborate with data analysts and other stakeholders to understand data requirements and develop solutions to meet their needs.

Implement data quality checks and monitoring to ensure data accuracy and consistency.

Troubleshoot and resolve data-related issues.

Create and maintain technical documentation for ETL processes, data pipelines, and database solutions.

Stay up-to-date with the latest trends and technologies in data management and analytics.