Interview : Coding test + Video
Visa : USC, GC, GC EAD, H4, L2
Remote or In-Office (St. Louis, MO)
Description :
- Data Engineer responsible for designing, developing, and maintaining data pipelines and warehousing solutions.
- Key tasks will include API integration, ETL development, data modeling (Star Schema or Snowflake Schema), and supporting Power BI reporting.
- Will primarily work on the OIL Analytics Report Migration project initially, then transition to the Tamarac Data Warehouse project.
- Collaborate with internal project teams to ensure data accuracy, integrity, and structured organization for business intelligence.
Tech stack
Azure Synapse Analytics
Two separate environments (e.g., Development and Production).Handles data warehousing and large-scale analytics workloads.Azure Data Lake
Centralized storage layer.Supports both structured and unstructured data.Scalable foundation for analytics and data integration.Azure Key Vault
Manages secrets, encryption keys, and certificates.Ensures secure access across both environments.Azure DevOps
CI / CD pipelines for automated builds and deployments.Manages data pipeline lifecycle and component delivery.Apache Spark Notebooks
Deployed in both environments.Used for interactive data exploration, transformation, and analytics.Azure Integration Runtime
Facilitates secure and scalable data movement.Enables transformations across network boundaries within Synapse or Data Factory.Metastore Data Warehouse
Centralized metadata repository.Maintains schema definitions, and table metadataARM Template (Azure Resource Manager)
Defines and automates infrastructure deployment.Enables consistent provisioning of Synapse, Data Lake, Key Vault, and other resources across environments.