What You'll Do
Map the current environment by identifying jobs, pipelines, and queries in HQL on Cloudera, as well as orchestration routines in Shell Script. Convert HQL queries to Amazon Athena, AWS Glue, or Amazon EMR, selecting the most suitable service based on use case. Redesign orchestration routines in Shell Script for AWS managed services like Step Functions, Lambda, EventBridge, and MWAA/Airflow. Model the target data lake in Amazon S3 and, when applicable, in Amazon Redshift.
What We're Looking For
Experience with Cloudera Hadoop, HQL, Shell Script, Amazon Athena, AWS Glue, Amazon EMR, AWS Step Functions, AWS Lambda, Amazon EventBridge, Amazon MWAA/Airflow, Amazon S3, and Amazon Redshift. Experience with data migration and pipelines between different environments is a plus. Knowledge of other Big Data and Cloud technologies is advantageous.
What We Offer
The company offers a positive impact on both business and employee lives, with a focus on professional growth in an innovative environment. Benefits include food allowance, discounts on courses and institutions, online courses with certification, mentoring, medical and dental assistance, and a pet agreement.
Key Skills & Technologies
Additional Information
Experience Level
Mid-Level
Job Language
Portuguese
Employment Type
Full-time
Work Mode
Remote