Senior Data Engineer

Hello

We are from MavinsysTalent Acquisition team based on One World Trade Centre New York.We are specializing in IT services and staffing majorly in lateralhiring / contract.

Below is one of our requirement to fillimmediately if youre interested please share your candidature to

Job Title : SeniorData Engineer

Location : NewYork(Remote)

Duration : 12months

• JobDescription;
• 59 years of relevant industryexperience with a BS / Masters or 2 years with aPhD
• Experience withdistributed processing technologies and frameworks such as HadoopSpark Kafka and distributed storage systems (e.g.

HDFSS3)

Demonstrated ability to analyze large datasets to identify gaps and inconsistencies provide data insights andadvance effective product

solutions.

• Expertise with ETL schedulers such as ApacheAirflow Luigi Oozie AWS Glue or similarframeworks
• Solidunderstanding of data warehousing concepts and handson experiencewith relational databases (e.g. PostgreSQL MySQL) and columnar

databases(e.g. Redshift BigQuery HBase ClickHouse)

Design build and maintain robustand efficient data pipelines that collect process and store datafrom various sources including userintercation

financial details and external datafeeds.

Develop data models that enable the efficientanalysis and manipulation of data for merchandising optimization.Ensure data quality consistency

andaccuracy.

• Build scalable data pipelines (SparkSQL &Scala) leveraging Airflow scheduler / executorframework
• Collaboratewith crossfunctional teams including Data Scientists ProductManagers and Software Engineers to define data requirements and

deliverdata solutions that drive merchandising and salesimprovements.

Contribute to the broader Data Engineeringcommunity at Airbnb to influence tooling and standards to improveculture and productivity Improve

code and data quality byleveraging and contributing to internal tools to automaticallydetect and mitigate issues.

Effective at building partnerships withbusiness stakeholders engineers and product to understand use casesfrom intended data consumers Able

to create & maintaindocumentation to support users in understanding how to usetables / columns

• Experience creating and evolving dimensionaldata models & schema designs to structure data forbusinessrelevant analytics.
• Strong experience using ETL framework (ex : Airflow) to build and deploy productionquality ETLpipelines.
• Experienceingesting and transforming structured and unstructured data frominternal and thirdparty sources into dimensionalmodels.
• Experiencewith dispersal of data to OLTP (ex : MySQL Cassandra HBase etc) andfast analytics solutions.

Data Systems Design

• Strong understanding ofdistributed storage and compute (S3 HiveSpark)
• Knowledge indistributed system design such as how mapreduce and distributeddata processing work at scale
• Basic understanding of OLTP systems likeCassandra HBase Mussel Vitess etc.

Coding

• Experience building batch data pipelines inSpark
• Expertise inSQL
• General SoftwareEngineering (e.g. proficiency coding in Python JavaScala)
• Experiencewriting data quality unit and functionaltests.
• Proficiency inSalesforce and understanding of its data structure. (Optional)
• Knowledge onSalesforce Bulk Operators. (Optional)

Hadoop,Spark,Airflow,Python,SQL