Senior Data Engineer - Remote

In the role of Senior Data Automation Engineer, you are at the heart of our efforts to streamline data-centric processes.

Your responsibilities include crafting and overseeing databases, as well as devising comprehensive solutions that enhance the efficiency of our business operations team.

Collaborating with various departments, you will assess data requirements, devise automation scripts, and roll out scalable systems tailored to our workforce's needs.

This position demands the integration of multiple external systems via APIs, linked servers, and standardized industry practices to develop new platforms that elevate our staff's productivity in their day-to-day activities.

Furthermore, you will lead the deployment of AI Ops-style solutions, incorporating advanced technologies such as Generative AI, Large Language Models (LLMs), Retrieval-Augmented Generation (RAG) models, and various Machine Learning techniques to automate the transfer and interpretation of data across different systems or endpoints.

Responsibilities :

• Architect, build, and maintain diverse database systems, ensuring optimal performance and quick response times for front-end queries.
• Craft and refine code in Python, PowerShell, and additional programming languages to streamline data processing and unify various systems through automation.
• Spearhead the creation and upkeep of scripts dedicated to the extraction, transformation, and loading (ETL) of data.
• Assist other teammates in web design elements to enhance user engagement when using our data automation tools.
• Employ RESTful APIs for seamless integration with a range of third-party services.
• Manage and analyze large data sets using sophisticated querying methods to draw actionable insights and aid in decision-making.
• Implement Machine Learning or Generative AI techniques to boost data analysis and predictive modeling capabilities.
• Keep abreast of the latest technological advancements and trends in data to refine and advance our automation strategies continuously.

Required Knowledge and Skills :

• Ability to convey complex technical / operational information to non-technical colleagues and stakeholders.
• Adjusts processes to enhance effectiveness, solve recurring problems or improve efficiency.
• Collaborates with functional and cross-organizational teams to generate solutions to complex and challenging issues.
• Expertise in MS-SQL, Databricks, Python, PowerShell, and other scripting languages, demonstrating strong coding proficiency.
• A thorough understanding of web technologies, including HTML, CSS, and JavaScript, alongside fundamental web design concepts.
• A proven track record in full-stack development, showing a capability to handle both backend and frontend tasks.
• Experience handling large datasets and conducting complex data analysis through advanced querying.
• A solid grasp of Machine Learning or Generative AI methodologies, with familiarity in using platforms such as Microsoft’s Semantic Kernel, Google’s Vertex AI, Open AI API, or comparable technologies.

Basic Qualifications :

• BA / BS degree and 4-6 years’ relevant experience OR equivalent combination of education and experience.
• 4+ years’ of experience in database management, including designing, developing, and maintaining various database systems.
• Proficiency in programming languages such as Python and PowerShell for automating tasks and integrating systems.
• Experience with RESTful API integration and managing large datasets.
• Basic knowledge of web technologies (HTML, CSS, JavaScript) for developing user interfaces.
• Understanding of data extraction, transformation, and loading (ETL) processes

Preferred Qualifications :

• Master’s degree and 6+ years’ relevant experience OR equivalent combination of education and experience.
• 4+ years of experience in a data automation or data engineering role, with a proven track record of implementing scalable solutions.
• 3+ year of experience with software solutions based in Healthcare or pharmaceutical environments (PHI / PII)
• 3+ year of experience with Kafka and other similar services
• 3+ year of experience managing and developing in large data lakes
• 1+ year of public and local cloud architectures, such as AWS, Docker, Nutanix
• 1+ year of experience of inventory / warehouse management

Work Conditions

• Remote environment
• Ability to travel less than 10% of the time
6 days ago