Good Communication & presentation skills.
• Strong Knowledge of Azure Components Azure Data Lake, Azure Data Factory, Azure SQL, Azure Databricks
Data Pipeline Development:
Design, build, and maintain efficient, reusable, and reliable data pipelines.
Ensure data flow between different systems and databases is seamless and error-free.
ETL Processes:
Develop and maintain ETL (Extract, Transform, Load) processes to aggregate data from various sources.
Ensure data quality and integrity throughout the ETL process.
Performance Optimization:
Optimize data pipelines and queries for performance.
Identify and resolve bottlenecks in data processing workflows.
Experience - 2 years or above
Qualification - B.Tech, MTech
• Must have strong knowledge & hands on in Apache Spark and Python programming, working with Delta Tables, Experience in Databricks is must
• Strong SQL Skills: Data modelling, Developing SQL Store Procedures, Functions, Dynamic SQL queries, Joins
• Must be aware of development of components for data fetching from APIs
• Strong knowledge of Data Warehousing Concepts.
• Hands on experience in ingesting data from various data sources and data types & file types
• Should have good knowledge on development lifecycle , best practice, and coding standards. Should be able to help team members, review the code.
• Good knowledge in Azure DevOps, understanding of build and release pipelines

