Data Engineer (5)

Data engineers build and maintain systems that collect, manage, and transform data into usable information. They work with large amounts of data from various sources, and ensure that data is accessible and flows smoothly to its destination.

Must have 4-6 years of experience with :

Databricks

Apache Spark

Python

PySpark

Establishes database management systems, standards, guidelines and quality assurance for database deliverables, such as conceptual design, logical database, capacity planning, external data interface specification, data loading plan, data maintenance plan and security policy. Documents and communicates database design. Evaluates and installs database management systems. Codes complex programs and derives logical processes on technical platforms. Builds windows, screens and reports. Assists in the design of user interface and business application prototypes. Participates in quality assurance and develops test application code in client server environment. Provides expertise in devising, negotiating and defending the tables and fields provided in the database. Adapts business requirements, developed by modeling / development staff and systems engineers, and develops the data, database specifications, and table and element attributes for an application. At more experienced levels, helps to develop an understanding of client's original data and storage mechanisms. Determines appropriateness of data for storage and optimum storage organization. Determines how tables relate to each other and how fields interact within the tables for a relational model.