Talent.com
Sr. Hadoop Data Engineer

Sr. Hadoop Data Engineer

Sonoma ConsultingSaint Louis, MO, US
job_description.job_card.variable_days_ago
serp_jobs.job_preview.job_type
  • serp_jobs.job_card.full_time
  • serp_jobs.job_card.permanent
job_description.job_card.job_description

Halo Group is a premier provider of IT talent. We place technology experts within

the teams of the world's leading companies to help them build innovative

businesses that keep them one step closer to their customers and one step

ahead of the competition. We offer a meaningful work environment for

employees, attractive and interesting engagements for consultants, and cutting-edge

digital innovation for our customers.

We delight in helping our customers execute their digital vision. Big projects or

small, Halo Group knows that by combining the highest quality talent with our

unwavering support, we will become an invaluable extension of the team. Halo

Group's experienced consultants in Detroit, Atlanta and Dallas specialize in all

areas of product / project governance, UX / UI, multi-platform applications, quality

assurance / testing, cloud computing, and data analytics.

Since its inception, Halo Group has been recognized for numerous awards, including :

  • INC 5000
  • Future 50
  • 101 Best and Brightest
  • Michigan 50 Companies to Watch
  • Goldline Research - "Most Dependable Companies"
  • Ernst & Young - "Entrepreneur of the Year" Finalist

Job Description

Preferred

  • Selecting and integrating Big Data tools and frameworks required to provide requested capabilities
  • Implementing Data ingestion and ETL processes on Hadoop
  • Monitoring performance and advising any necessary infrastructure changes
  • Defining data retention policies
  • Design and build data processing pipelines for structured and unstructured data using tools and frameworks in the Hadoop ecosystem
  • Develop applications that are scalable to handle millions of events / records
  • Design and launch scalable, reliable and efficient processes to move, transform and report on large amounts of data
  • Participate in meetings with business (account / product management, data scientists) to obtain new requirements
  • Follow our Agile software development process with daily scrums and monthly Sprints
  • Ability to work collaboratively on a cross-functional team with a wide range of experience levels

    • QUALIFICATIONS

  • Bachelor's degree and 8+ years relevant experience or Master's degree and 6+ years of relevant experience
  • 4+ years in industry implementing big data solutions on Hadoop
  • Proficient understanding of distributed computing principles
  • Proficiency with Hadoop v2, MapReduce, HDFS
  • Experience with building stream-processing systems, using solutions such as Storm or Kafka and Spark-Streaming
  • Good knowledge of Big Data querying tools, such as Pig, Hive, Phoenix
  • Experience with Spark
  • Experience with integration of data from multiple data sources
  • Experience with 1 or 2 NoSQL / Graph databases, such as HBase, Cassandra, MongoDB, Neo4j
  • Proficiency in programming languages like SCALA, Java, Python
  • Experience with Linux OS, shell scripting
  • Experience with relational databases (SQL)
  • Experience in working with real-time data feeds
  • Experience in working with unstructured data
  • Experience in implementing Scoop Jobs to Import / Export data from Hadoop
  • Knowledge of various ETL techniques and frameworks, such as Pig, Hive, or Flume
  • Experience with various messaging systems, such as Kafka or RabbitMQ
  • Experience with Big Data ML toolkits, such as Mahout, SparkML, or H2O
  • Good understanding of Lambda Architecture, along with its advantages and drawbacks
  • Experience with Hortonworks Hadoop Data Platform (HDP)
  • Experience with all or some of the following supporting Hadoop administration and security frameworks : HCatalog, Drill, NiFi, Oozie, Falcon, Ranger, Ambari, Zeplin.

    • Qualifications

  • Bachelor's degree or foreign equivalent required from an accredited institution. Will also consider three years of progressive experience in the specialty in lieu of every year of education.
  • At least 2 years of experience with Information Technology.

    • Additional Information

  • U.S. Citizens and those who are authorized to work independently in the United States are encouraged to apply. We are unable to sponsor at this time.

    • This is a Full-Time / Permanent job opportunity.

    Only for US Citizen and Green Card Holder

  • All your information will be kept confidential according to EEO guidelines.

    • J-18808-Ljbffr

    serp_jobs.job_alerts.create_a_job

    Sr Data Engineer • Saint Louis, MO, US

    Job_description.internal_linking.related_jobs
    • serp_jobs.job_card.promoted
    • serp_jobs.job_card.new
    Epic Certified Data Engineer II

    Epic Certified Data Engineer II

    VirtualVocationsSaint Louis, Missouri, United States
    serp_jobs.job_card.full_time
    A company is looking for a Data Engineer II.Key Responsibilities Design and develop data pipelines and integrate new business data into the Data Warehouse while adhering to best practices Collab...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_hours
    • serp_jobs.job_card.promoted
    Senior Data Analytics Engineer

    Senior Data Analytics Engineer

    VirtualVocationsSaint Charles, Missouri, United States
    serp_jobs.job_card.full_time
    A company is looking for a Senior Data Analytics Engineer.Key Responsibilities Lead the migration of dashboards, reports, and visualizations from ThoughtSpot to Power BI Build reusable data mode...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
    • serp_jobs.job_card.promoted
    • serp_jobs.job_card.new
    ETL Data Architect

    ETL Data Architect

    VirtualVocationsFlorissant, Missouri, United States
    serp_jobs.job_card.full_time
    A company is looking for an ETL Data Architect to design and develop enterprise-wide application systems.Key Responsibilities Collaborate with business partners and teams to identify technical an...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_hours
    • serp_jobs.job_card.promoted
    Information Delivery Engineer

    Information Delivery Engineer

    VirtualVocationsFlorissant, Missouri, United States
    serp_jobs.job_card.full_time
    A company is looking for an Information Delivery & Visualization Engineer.Key Responsibilities Design, develop, and implement end-to-end data integration workflows using Informatica IICS Develop...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
    • serp_jobs.job_card.promoted
    Data Science Engineer

    Data Science Engineer

    VirtualVocationsSaint Charles, Missouri, United States
    serp_jobs.job_card.full_time
    A company is looking for a Data Science Engineer to join a specialized project team focused on building and maintaining production-level data pipelines and enabling machine learning workflows.Key R...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
    • serp_jobs.job_card.promoted
    • serp_jobs.job_card.new
    Senior Release Engineer

    Senior Release Engineer

    VirtualVocationsSaint Louis, Missouri, United States
    serp_jobs.job_card.full_time
    A company is looking for a Senior Release Engineer to manage and enhance its release infrastructure and processes.Key Responsibilities Own and evolve CI / CD infrastructure and release processes wi...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_hours
    • serp_jobs.job_card.promoted
    Lead Data Engineer

    Lead Data Engineer

    VirtualVocationsFlorissant, Missouri, United States
    serp_jobs.job_card.full_time
    A company is looking for a Lead Data Engineer to join their data engineering team.Key Responsibilities Lead the technical design and execution of complex data projects while mentoring mid-level e...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
    • serp_jobs.job_card.promoted
    Data Engineer II

    Data Engineer II

    VirtualVocationsSaint Louis, Missouri, United States
    serp_jobs.job_card.full_time
    A company is looking for a Data Engineer II to enhance data solutions for their insurance data platform.Key Responsibilities Design, create, and maintain data solutions, including data pipelines ...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
    • serp_jobs.job_card.promoted
    Senior Data Architect

    Senior Data Architect

    VirtualVocationsFlorissant, Missouri, United States
    serp_jobs.job_card.full_time
    A company is looking for a Senior Data Architect - Product Security.Key Responsibilities Define and implement a secure, scalable data architecture for risk, vulnerability, and compliance data Bu...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
    • serp_jobs.job_card.promoted
    Senior Data Management Engineer

    Senior Data Management Engineer

    VirtualVocationsSaint Charles, Missouri, United States
    serp_jobs.job_card.full_time
    A company is looking for a Senior Engineer, Data Management (Remote).Key Responsibilities Build and automate data pipelines, ensuring data quality and operationalizing AI / ML solutions Collaborat...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
    • serp_jobs.job_card.promoted
    • serp_jobs.job_card.new
    Data Steward

    Data Steward

    VirtualVocationsSaint Charles, Missouri, United States
    serp_jobs.job_card.full_time
    A company is looking for a Staff Data Steward.Key Responsibilities Coordinate design sessions with various stakeholders to capture business metadata and design documentation Implement data solut...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_hours
    • serp_jobs.job_card.promoted
    Principal Data Architect

    Principal Data Architect

    VirtualVocationsFlorissant, Missouri, United States
    serp_jobs.job_card.full_time
    A company is looking for a Principal Data Architect - AI.Key Responsibilities Lead the architecture, design, and implementation of a cloud-native data platform for enterprise analytics and AI / ML ...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
    • serp_jobs.job_card.promoted
    Senior Data Engineer

    Senior Data Engineer

    VirtualVocationsSaint Louis, Missouri, United States
    serp_jobs.job_card.full_time
    A company is looking for a Senior Data Engineer to design and develop innovative data solutions that transform complex information into actionable insights. Key Responsibilities Design, develop, a...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
    • serp_jobs.job_card.promoted
    • serp_jobs.job_card.new
    Senior Data Warehouse Developer

    Senior Data Warehouse Developer

    VirtualVocationsFlorissant, Missouri, United States
    serp_jobs.job_card.full_time
    A company is looking for a Senior Data Warehouse Developer to design, develop, and maintain data integration processes.Key Responsibilities Design and maintain data warehouse architecture, includ...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_hours
    • serp_jobs.job_card.promoted
    Data Engineer

    Data Engineer

    VirtualVocationsSaint Charles, Missouri, United States
    serp_jobs.job_card.full_time
    A company is looking for a Data Engineer - REMOTE.Key Responsibilities Develop enterprise-grade data platforms, services, and pipelines for clients Lead and architect the migration of data envir...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
    • serp_jobs.job_card.promoted
    Staff Data Engineer

    Staff Data Engineer

    VirtualVocationsSaint Louis, Missouri, United States
    serp_jobs.job_card.full_time
    A company is looking for a Staff Data Warehouse Engineer.Key Responsibilities Design and deploy architectural branches that evolve over time Ensure data quality, security, and governance across ...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
    • serp_jobs.job_card.promoted
    Data Architect

    Data Architect

    VirtualVocationsFlorissant, Missouri, United States
    serp_jobs.job_card.full_time
    A company is looking for an IICS Snowflake Data Architect.Key Responsibilities Collaborate with business partners and architects to identify technical and functional system needs Lead the design...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
    • serp_jobs.job_card.promoted
    Data Warehouse Architect

    Data Warehouse Architect

    VirtualVocationsSaint Louis, Missouri, United States
    serp_jobs.job_card.full_time
    A company is looking for a Data Warehouse Architect & AI Analytics Lead.Key Responsibilities Lead the design and implementation of extensions to a Data Vault 2. Architect and pilot predictive anal...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
    • serp_jobs.job_card.promoted
    • serp_jobs.job_card.new
    Data Engineer

    Data Engineer

    System OneSt. Louis, MO, US
    serp_jobs.job_card.full_time
    System One is seeking an Azure Data Engineer (ERP) for a long-term opportunity in St.The Data Engineer will lead and implement data engineering projects that integrate data from multiple ERP system...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_hours
    • serp_jobs.job_card.promoted
    Data Warehouse Developer

    Data Warehouse Developer

    VirtualVocationsSaint Charles, Missouri, United States
    serp_jobs.job_card.full_time
    A company is looking for a Data Warehouse Developer to design, develop, and maintain integration processes for data storage and data mining. Key Responsibilities Develop, implement, and execute bu...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30