Requirements

• In-depth knowledge of Big Data technologies - Spark, HDFS, Hive, Kudu, Impala
• Solid programming experience in Python, Java, Scala, or other statically typed programming language
  Production experience in core Hadoop technologies including HDFS, Hive and YARN
• Strong working knowledge of SQL and the ability to write, debug, and optimize distributed SQL queries
• Excellent communication skills; previous experience working with internal or external customers
• Strong analytical abilities; ability to translate business requirements and use cases into a Hadoop solution, including ingestion of many data sources, ETL processing, data access, and consumption, as well as custom analytics
• Experience working with workflow managers like Airflow, Prefect, Luigi, Oozie
• Experience working with Data Governance tools like Apache Sentry, Kerberos, Atlas, Ranger
• Experience working with streaming data with technologies like Kafka, Spark streaming
• Strong understanding of big data performance tuning
• Experience handling different kinds of structured and unstructured data formats (Parquet/Delta Lake/Avro/XML/JSON/YAML/CSV/Zip/Xlsx/Text etc.)
• Experience working with distributed NoSQL storage like ElasticSearch, Apache Solr
• Experience deploying big data pipelines in the cloud preferably using GCP and AWS
• Well versed with Software Development Life Cycle Methodologies and Practices
• Spark Certification is a huge plus
• Cloud experience is a must have preferably with GCP
• Contribution to open source community and Apache committer will be big plus