- As a Senior Data Engineer, you will build and design highly scalable data pipelines using new generation tools and technologies like Spark, Kafka, Storm and Big Query to induct data from various systems to provide efficient reporting and analytics capability.
- You will translate complex business requirements into scalable technical solutions, and design dashboards or visualization using BI tools to perform data analysis and to support business.
- You will collaborate with multiple cross functional teams including Product Management, Solution Architectures, Security, and Software Engineering.
Required Skills and Experience:
- Experience in designing and building dimensional data models to improve accessibility, efficiency, and quality of data
- Experience in designing and developing ETL data pipelines.
- In-depth understanding of data modelling, data structures and algorithms
- 1 year of experience with Google Cloud Platform (GCP) is required
- Stream processing experience - Storm, Spark-Streaming
- Experience with all aspects of software development life cycle (source control, continuous integration, deployments, etc.)
- Experience in building high quality software in Java, Python or Scala preferred
- The successful candidate will also have the experience, the soft skills and enthusiasm to play an active positive role in the interface between product management, solution architecture and software engineering
Preferred Skills and Experience:
- Experience with Enterprise Java; Hadoop; Hive; HBase; Kafka; Spark; Impala; Yarn; Drill; HTTP APIs
- Experience developing Big Data/Hadoop applications using java, Spark, Hive, Oozie, Kafka, and Map Reduce is a huge plus
- Experience with or advance courses on data science and machine learning is ideal
- Any experience with BI tools is a plus