Responsibilities
include but are not limited to :
- Work with Data architects to understand current data models, to build pipelines for data ingestion and transformation.
- Design, build, and maintain a framework for pipeline observation and monitoring, focusing on reliability and performance of jobs.
- Surface data integration errors to the proper teams focusing on :
- Ensuring timely processing of new data
- Performance of data pipelines
- Integrity and quality of source data
- Hands-on experience building data-lake style infrastructures using streaming data set technologies (particularly with Apache Kafka)
Qualifications
- Bachelor’s degree in computer science, data science or related technical field, or equivalent practical experience
- Experience building and maintaining AWS based data pipelines : currently utilizing AWS Lambda, Docker / ECS, MSK, Airflow, Databricks, Unity Catalog
- Development experience utilizing two or more of the following :
- Python : (Pandas / Numpy, Boto3, SimpleSalesforce)
- Databricks (pySpark, pySQL, DLT)
- Apache Spark
- Kafka and the Kafka Connect ecosystem (schema registry and Avro)
- Terraform (or other infrastructure as code platform)
- Enthusiasm for working directly with customer teams (Business units and internal IT)
Preferred Qualifications
- Proven experience with relational and NoSQL databases (. Postgres, Redshift, MongoDB)
- Experience with version control (git) and peer code reviews
- Familiarity with data visualization techniques using tools such as Grafana, PowerBI, AWS Quick Sight, and Excel
Benefits Package At Shamrock we hire bright, ambitious people and give them the tools they need to be successful. By investing in training and development, we hope to become a long-term career for employees, where there are always opportunities for advancement.
Shamrock also offers a premier set of benefits for employees and their families :
- Medical : Fully paid healthcare, dental and vision premiums for employees and eligible dependents
- Work-Life Balance : Competitive PTO and paid leave policies
- Financial : Generous company 401(k) contributions and employee stock ownership after one year
- Wellness : Onsite gym and discounted membership to select fitness centers. Jogging trails available at Overland Park offices
Field Engineer- Data Centers
Scope of work- Relays, PLC & Controls Engineering. Location: Data Centers throughout the U. With over 90 years' combined experience, NES Fircroft (NES) is proud to be the world's leading engineering staffing provider spanning the Oil & Gas, Power & Renewables, Chemicals, Construction & Infrastructur...
Senior Big Data Engineer
A company is looking for a Senior Big Data Engineer. ...
Data Engineer/DBA
Description:TEKsystems is looking for a highly experienced Data engineer/DBA to work directly with product managers, EHR engineers, data engineers, DBAs, and support professionals to facilitate retrieving data from a variety of sources, and transforming it into required formats, and creating data pr...
Data Infrastructure Engineer
A company is looking for an Experienced Data Infrastructure Engineer. ...
Data Scientist / ML Engineer
Proficient in machine learning data workflows, data collection methodologies, and data analysis. Expertise with SQL, noSQL, Python, R, Javascript programming languages and big data environments (such as Splunk, Hadoop, Spark, Flink, Stream Analytics, Kafka, Docker, Kubernetes etc. Experience develop...
Senior ML Systems Engineer ML Data Platform
A company is looking for a Senior ML Systems Engineer, ML Data Platform. Proficiency in TypeScript, React, Redux, Golang/Python, and relational databasesStrong understanding of object-oriented programming, data structures, and algorithm designExperience with A/B testing frameworks, telemetry systems...
Vulnerability Data Analytics Engineer
We are seeking a highly skilled and motivated individual to join our team as a Big Data Cloud-Based Vulnerability Management Data Analytics Developer. This is an exciting opportunity to work on cutting-edge technology and contribute to our mission of safeguarding critical data and infrastructure. Th...
Distinguished Engineer – Finance Data - Remote
Advanced experience developing new and enhancing existing data processing (data ingestion, data transformation, data storage, data management, data quality) components. Data processing/data transformation using ETL/ELT tools such as DBT (Data Build Tool) or Databricks. We are seeking experienced eng...
Senior Civil Engineer - Data Center
As a Senior Civil Engineer on our Data Center Civil Team, you will be a part of the firm’s largest and most complex projects. Olsson currently has one opportunity for a Senior Engineer to support our Data Center Site Design group. You will serve as a project manager on some projects and lead design ...
Senior Engineer- Data Analytics
Our Senior Engineer is a key member of the engineering staff working across the organization to provide a friction-less experience to our customers and maintain the highest standards of protection and availability. Utilize your passion for data exploration to produce high quality, accurate data and/...