For a tech company with an AI/big data solution for the retail industry we are looking for DATA engineer (located in Yokneam - Hybrid)
What will you be doing?
Be responsible for Data flow stages from definition to architecture, including Data acquisition, Data set acceptance criteria, Data storage technologies, Data Science integration
Performing strategic analysis of data, architecture designing and customizing technological solutions for large scale data processing
The Data Engineer will take responsibility to explore technologies to scale up Data ecosystem to handle rapidly big Data growth
Work closely with Data science team to embed ML / AI algorithms into the product
Work with cloud, data science, application, and client teams to make sure your solution is solving significant business problems in a robust, scalable manner.
Design, develop, and maintain ETL/ELT from various data streams
Use cutting edge data engineering technologies: Python, Airflow, Java, Kubernetes, Spark, Scala, Apache Parquet, Talend, Data lake…
2+ years of relevant data engineering work experience in production environments (Data environments, Big Data processing, Database techniques, Data analysis or similar)
· B.Sc / BA in Computer Science or equivalent
· Must have hands on programming experience in Python, Java (Scala – advantage)
· Must have hands on experience in SQL & databases
· Excellent English communication skills spoken and written
· Ability to learn new technologies and work in a dynamic fast paced environment
· Focused on personal development and eager to learn.
· Result-driven, pragmatic, and innovative.
· Working experience with Airflow and Spark is a big plus.
· Experience with Cloud technology is an advantage.