Job Description :
We are looking for a Data Engineer to design, develop, and support scalable data solutions in a cloud-based environment. The role focuses on AWS, data processing, cloud data platforms, and modern data engineering practices.
Responsibilities:
- Design, develop, and maintain scalable data pipelines and data solutions.
- Build and optimize data processing workflows using AWS services.
- Develop data transformations and processing solutions using Python, Java, SQL, and PySpark.
- Work with Databricks and Dataiku to support data engineering and analytics use cases.
- Develop and maintain cloud infrastructure using CloudFormation and Terraform.
- Implement CI/CD pipelines and follow software development lifecycle (SDLC) practices.
- Work with Spark/PySpark to process and transform large datasets.
- Ensure data solutions are reliable, scalable, and maintainable.
- Collaborate with data, technology, and business teams.
- Create and maintain technical and SDLC documentation.
Profile / Requirements :
- Experience with AWS, including: S3, IAM, Redshift, SageMaker, Glue, Lambda, Step Functions, CloudWatch.
- Proficiency in Python or Java and SQL, with strong experience in Redshift preferred.
- Hands-on experience with Spark/PySpark.
- Experience with Databricks and/or Dataiku.
- Experience with Jenkins, Git, Docker, CloudFormation, and Terraform.
- Good understanding of data engineering and SDLC practices.
- Strong communication and documentation skills.
Nice to Have:
- Knowledge of DSCS and DPTM.
- Experience with enterprise-scale data platforms and cloud environments.
- Experience working in Agile or cross-functional teams.
About us :
ALGOTEQUE is an IT consultancy firm that helps startups, mid-sized and large corporations to create and deliver innovative technologies. Our team has a successful track record in designing, developing, implementing, and integrating software solutions (AI, ML, BI, Web, Automation) for Telecom, Energy, Bank, Insurance, Pharma, Automotive, Industry, e-commerce. We deliver our services both in fixed-price and time-and-materials models, helping our customers achieve their business and IT strategies.