Responsibilities
include but are not limited to :
- Work with Data architects to understand current data models, to build pipelines for data ingestion and transformation.
- Design, build, and maintain a framework for pipeline observation and monitoring, focusing on reliability and performance of jobs.
- Surface data integration errors to the proper teams focusing on :
- Ensuring timely processing of new data
- Performance of data pipelines
- Integrity and quality of source data
- Hands-on experience building data-lake style infrastructures using streaming data set technologies (particularly with Apache Kafka)
Qualifications
- Bachelor’s degree in computer science, data science or related technical field, or equivalent practical experience
- Experience building and maintaining AWS based data pipelines : currently utilizing AWS Lambda, Docker / ECS, MSK, Airflow, Databricks, Unity Catalog
- Development experience utilizing two or more of the following :
- Python : (Pandas / Numpy, Boto3, SimpleSalesforce)
- Databricks (pySpark, pySQL, DLT)
- Apache Spark
- Kafka and the Kafka Connect ecosystem (schema registry and Avro)
- Terraform (or other infrastructure as code platform)
- Enthusiasm for working directly with customer teams (Business units and internal IT)
Preferred Qualifications
- Proven experience with relational and NoSQL databases (. Postgres, Redshift, MongoDB)
- Experience with version control (git) and peer code reviews
- Familiarity with data visualization techniques using tools such as Grafana, PowerBI, AWS Quick Sight, and Excel
Benefits Package At Shamrock we hire bright, ambitious people and give them the tools they need to be successful. By investing in training and development, we hope to become a long-term career for employees, where there are always opportunities for advancement.
Shamrock also offers a premier set of benefits for employees and their families :
- Medical : Fully paid healthcare, dental and vision premiums for employees and eligible dependents
- Work-Life Balance : Competitive PTO and paid leave policies
- Financial : Generous company 401(k) contributions and employee stock ownership after one year
- Wellness : Onsite gym and discounted membership to select fitness centers. Jogging trails available at Overland Park offices
Senior Data Engineer
You'll primarily be working with our operations and engineering teams to create and enhance data pipelines, conform and enrich data, and deliver information to business users. Hands-on experience with database design and data modeling, preferable experience with Data Vault methodology. Contribute to...
Data Engineer II
We are looking for an accomplished and forward-thinking Data Engineer II to help propel our data engineering team. As a Data Engineer II, you will be responsible for designing, developing, and assisting with the architecture, infrastructure, and tools required for efficient and scalable data storage...
Software Engineer, ML Ops and Data Engineering - Regulatory
Software Engineer, ML Ops and Data Engineering – Regulatory. As a Machine Learning Operations and Data Engineer - Regulatory, you’ll support the prototyping, training and productionization of ML models and applications for the surveillance of financial markets, which generate terabytes of new data e...
Field Engineer- Data Centers
Scope of work- Relays, PLC & Controls Engineering. Location: Data Centers throughout the U. With over 90 years' combined experience, NES Fircroft (NES) is proud to be the world's leading engineering staffing provider spanning the Oil & Gas, Power & Renewables, Chemicals, Construction & Infrastructur...
Field Engineer- Data Centers
Scope of work-Â Relays, PLC & Controls Engineering. Location: Data Centers throughout the U. With over 90 years' combined experience, NES Fircroft (NES) is proud to be the world's leading engineering staffing provider spanning the Oil & Gas, Power & Renewables, Chemicals, Construction & ...
Data Engineer
Associate or bachelor’s degree in computer science, software engineering or 5+ years equivalent related IT experience. Extensive experience with relational databases. Knowledge of applicable data privacy practices and laws. ...
Software Engineer - Aviation Cloud Data Content
Essential Functions Creates and executes designs for small sets of new functionalities as part of a software project using C, C++, C#, Java, assembly language, or other selected languages Utilizes software engineering tools such as configuration management systems, build processes, and debuggers in ...
Field Engineer- Data Centers
Scope of work- Relays, PLC & Controls Engineering. Location: Data Centers throughout the U. With over 90 years' combined experience, NES Fircroft (NES) is proud to be the world's leading engineering staffing provider spanning the Oil & Gas, Power & Renewables, Chemicals, Construction & Infrastructur...
Sr. Data Engineer
Implementing and operating data pipelines for collecting data from applications and environments. Leading and/or creating data and UI designs. Planning and performing all data-related development and optimization job functions. Streamlining and managing data schema or stored procedures changes throu...
Senior Engineer- Data Analytics
Our Senior Engineer is a key member of the engineering staff working across the organization to provide a friction-less experience to our customers and maintain the highest standards of protection and availability. Utilize your passion for data exploration to produce high quality, accurate data and/...