Job Title : Data Engineer / Data Architect
Location : Corvallis, OR 5 Days Onsite Role Job Description :
- At least 8 to 10+ years’ experience in data engineering , AI and ML and Aws / Azure
- Architect, Design and implement distributed data processing pipelines using Spark, Python, SQL and other tools and languages prevalent in the Big Data / Lakehouse ecosystem.
- Experience in AI and ML on AWS / AZURE
- Analyzes design and determines coding, programming, and integration activities required based on general objectives.
- Play the technical lead role representing deliverables from vendor team resources at onsite and offshore locations.
- Lead the technical co-ordination and Business Knowledge transition activities to offshore team.
- Reviews and evaluates designs and project activities for compliance with architecture, security and quality guidelines and standards.
- Writes and executes complete testing plans, protocols, and documentation for assigned portion of data system or component;
identifies defects and creates solutions for issues with code and integration into data system architecture.
- Collaborates and communicates with project team regarding project progress and issue resolution.
- Works with the data engineering team for all phases of larger and more-complex development projects and engages with external users on business and technical requirements.
- Collaborates with peers, engineers, data scientists and project team.
What you bring :
- Bachelor's or Master's degree in Computer Science, Information Systems, Engineering or equivalent.
- 6+ years of relevant experience with detailed knowledge of data technical architectures, infrastructure components, ETL / ELT and reporting / analytic tools.
- 2+ years of experience with Cloud based DW such as Redshift, Snowflake etc.
- 1+ years’ experience in Big Data Distributed ecosystems (Hadoop, SPARK, Unity Catalog & Delta Lake)
- 1+ years’ experience in Big Data Distributed systems such as Databricks, AWS EMR, AWS Glue etc.
- Leverage monitoring tools / frameworks, like Splunk, Grafana, CloudWatch etc.
- Experience with container management frameworks such as Docker, Kubernetes, ECR etc.
- Experience working on CI / CD processes such as Jenkins, Codeway etc. and source control tools such as GitHub, etc.
- Strong experience in coding languages like Python, Scala & Java
Knowledge and Skills :
- Fluent in relational based systems and writing complex SQL.
- Fluent in complex, distributed and massively parallel systems.
- Strong analytical and problem-solving skills with ability to represent complex algorithms in software.
- Strong understanding of database technologies and management systems.
- Strong understanding of data structures and algorithms
- Database architecture testing methodology, including execution of test plans, debugging, and testing scripts and tools.
- Strong analytical and problem-solving skills.
Nice to Have :
- Experience with visualization tools such as PowerBI, Tableau etc.
- Experience with transformation tools such as dbt.
- Have experience in building realtime streaming data pipelines
- Experience in pub / sub streaming technologies like Kafka, Kinesis, Spark Streaming etc.
Data Engineer/ Data Architect
Job Title: Data Engineer/ Data Architect. Architect, Design and implement distributed data processing pipelines using Spark, Python, SQL and other tools and languages prevalent in the Big Data/Lakehouse ecosystem. Writes and executes complete testing plans, protocols, and documentation for assigned ...
Senior Data Architect
As a Senior Data Architect on the Data team, you’ll establish consistent data standards, reference architectures, patterns, and practices across the organization for both OLTP and OLAP (Data warehouse, Data Lake house) MDM and AI/ML technologies. Additionally, you’ll create strategies and design sol...
Azure/Snowflake Data Warehouse Engineer
Proven experience (5+ years) in data engineering, with a focus on data warehousing, ETL, and data modeling. Collaborate with data architects and analysts to design and implement data models that meet the business requirements. We are seeking a skilled Senior Cloud-Based Data Engineer to join our dyn...
Senior Big Data Engineer
In partnership with other business, platform, technology, and analytic teams across the enterprise, design, build and maintain well-engineered data solutions in a variety of environments, including traditional data warehouses, Big Data solutions, and cloud-oriented platforms. Align with security, da...
Data Architect
Architect, Design and implement distributed data processing pipelines using Spark, Python, SQL and other tools and languages prevalent in the Big Data/Lakehouse ecosystem. Writes and executes complete testing plans, protocols, and documentation for assigned portion of data system or component; ident...
Senior Electrical Engineer- Mission Critical Data Center Program Lead
Senior Electrical Engineer- Mission Critical Data Center Program Lead. Our mission is to engineer and commission “green buildings and data center campuses that work. Strong background in electrical engineering and designing data centers including ancillary power [EM1] distribution systems including ...
Data Engineer
The Data Engineer will play a key role in developing a plan for a connected EPH data ecosystem, including connecting NPAIHB’s ArcGIS Online environment to tribes’ Esri Enterprise environments, connecting GIS data to environmental services data, and connecting EPH data to NPAIHB’s Data Hub, a newly c...
Analytic Data Engineer – Databricks
This role within the 'Data Engineering & Self-Service Products' team involves architecting and engineering analytic data solutions, including designing and developing data marts in Databricks using PySpark or Spark SQL, building interactive Power BI dashboards to visualize KPIs and trends, and creat...
Decision Support Analyst (Data Engineer)
Performs modeling of data sources and flows, works with data management staff to define an operational framework that will define, develop and implement policies and security frameworks to ensure precise and secure delivery of knowledge and information. Perform modeling of data sources and flows, wo...
Public Works GIS Data Steward Analyst
Collects, inputs, edits, and verifies spatial data from a variety of internal and external data inputs. Modifies and maintains CMMS data to support asset data analysis. Collaborates with Asset Management staff and Public Works supervisors to assist in program development by gathering information for...