Azure Big Data Engineer 887 views

EPAM is committed to providing its global team of 36,700+ EPAMers with inspiring careers from day one. EPAMers lead with passion and honesty and think creatively. Their people are the source of their success and they value collaboration, try to always understand their customers’ business, and strive for the highest standards of excellence. In today’s new market conditions, they continue to support operations for hundreds of clients around the world remotely, with the vast majority of their teams working from home. No matter where you are located, you’ll join a dedicated, diverse community that will help you discover your fullest potential.

Description

EPAM Big Data Practice is looking for an Azure Big Data Engineers

• As an Azure Data Engineer, you will be responsible for designing and implementing the management, monitoring, security, and privacy of data using the full stack of Azure data services to satisfy the business needs.

You are curious, persistent, logical, and clever – a true techie at heart. You enjoy living by the code of your craft and developing elegant solutions for complex problems. If this sounds like you, keep reading to learn more about this exciting role!

Come and join EPAM where Engineering is in their DNA.

Responsibilities

• Implement non -relational data stores:
  • Implement a solution that uses Cosmos DB, Data Lake Storage Gen2
  • Implement data distribution and partitions
  • Implement a consistency model in Cosmos DB
  • Provision a non-relational datastore
  • Provide access to data to meet security requirements
  • Implement for high availability, disaster recovery, and global distribution
• Implement relational data stores:
  • Provide access to data to meet security requirements
  • Implement for high availability, disaster recovery, and global distribution
  • Implement data distribution and partitions for Azure Synapse Analytics
• Manage data security:
  • Implement data masking
  • Encrypt data at rest and in motion
• Develop batch processing solutions:
  • Develop batch processing solutions by using Data Factory and Azure Databricks
  • Ingest data by using Stream Sets and/or ADF
  • Implement the integration runtime for Data Factory
  • Create linked services and datasets
  • Create pipelines and activities
  • Create and schedule triggers
  • Implement Azure Databricks clusters, notebooks, jobs, and autoscaling
  • Ingest data into Azure Databricks
• Develop streaming solutions:
  • Configure input and output
  • Select the appropriate windowing functions
  • Implement event processing by using Stream Analytics
  • Ingest and query streaming data with Azure Data Explorer
• Monitor data processing:
  • Monitor Data Factory pipelines
  • Monitor Stream Analytics
  • Monitor Stream Analytics
• • Troubleshoot data partitioning bottlenecks
  • Optimize Data Lake Storage
  • Optimize Stream Analytics
  • Optimize Azure Synapse Analytics
  • Optimize SQL Database
  • Manage the data lifecycle
    Optimize Azure data solutions

Requirements

• 5-10+ years in IT
• 2-3+ years in Azure
• Experience in Agile or PMI methodology managed projects
• Experience in enterprise applications, solutions and data infrastructures
• Experience in designing data management solutions
• Experience in designing robust CI/CD solutions
• Python/PySpark -highly desired
• ADLS* (gen 1, gen 2 – preview mode)
• ADF* v2 – two instances per project: ingestion and transformation
• Databricks
• SQLDB
• Synapse Analytics (former SQLDW)
• AAS
• Embedded Power BI
• Java/Scala – Spark +
• Git, GitLab, Artifactory +
• ADO
• Microsoft Certified: Azure Data Engineer Associate +
• SCRUM Developer +
• Azure DevOps +

What They Offer

• They offer a range of discretionary benefits from time to time, including:
  • Group personal pension plan, life assurance, and income protection
  • Private medical insurance, private dental care, and critical illness cover
  • Cycle scheme and season ticket loan
  • Employee assistance program
  • Gym discount, Friday lunch, on-site massage, and social events
  • 1 day off for your wedding and baby basket
  • Tech purchase scheme
  • Unlimited access to LinkedIn learning solutions
• Some of these benefits may be available only after you have passed your probationary period