Senior Data Engineer(Spark, SQL, Databrick, Python)
Roles and Responsibilities
- Skills- Spark, SQL, Databrick, Python.
Experience with at least one of the following languages Scala, Python, R and or Java
A minimum of 3-5 years of hands-on technical experience with:
big data implementation and technology offerings
AWS/cloud big data modeling & data management
analytics and ingestion architecture of big data
data lake management and data architecture
data lake design patterns & cloud best enterprise practices
IoT and streaming, real time processing
Big data related AWS technologies
Experience in AWS technologies such as Kinesis, Lambda, EC2, Redshift, RDS, Cloud formation, EMR, AWS S3, AWS Analytics, Spark, Databricks
Experience with at least one of the following languages Scala, Python, R and or Java
Experience with designing, developing, and implementing complex integration for end-to-end solutions at a middleware and app level with focus on performance optimization
Strong implementation skill in area of cloud development in AWS
Demonstrated ability in implementing cloud scalable, real time and high-performance data lake solutions (AWS)
Ability to quickly perform proof-of-concepts for validating new technology or approach
Ability to exercise independent judgment and creative problem-solving techniques in a highly complex environment using leading-edge technology and/or integrating with diverse application systems
Ability to lead and drive technology change in a fast-paced, dynamic environment and all phases of the entire software life cycle
Strong experience with data catalog, data governance, Collibra, MDM and/or Data Quality (IDQ) toolset
Strong experience with integration of diverse data sources (batch and real time) in the cloud
Lead the design and sustainment of data pipelines and data storage
Expertise in Structured, unstructured, SQL and No-SQL technologies
Expertise with identifying and understanding source data systems and mapping source system attributes to the target
Experience with design and automation of ETL\ELT processes
AWS and cloud performance tuning and optimization experience
Experience with effort estimation for new projects/proposals on an ongoing basis.
Excellent communication skills across all levels; ability to communicate with ease the complex and technical concepts.
Ability to work effectively in a fast-paced environment
Role:Data warehouse Developer
Salary: Not Disclosed by Recruiter
Industry:Analytics / KPO / Research
Functional AreaEngineering - Software & QA
Role CategoryDBA / Data warehousing
Employment Type:Full Time, Permanent
Education
UG:Graduation Not Required
Company Profile
CHANGE LEADERS CONSULTING
We are passionate career advisers and community builders supporting brands in Predictive analytics, Consulting and IOT.
Recruiter Name:Renu
Contact Company:CHANGE LEADERS CONSULTING
Telephone:9560638567
Website:http://careers.changeleaders.in