Data warehouse team is responsible for building the Shopee data warehouse to support the company decision making and also build application data for different kinds of data products. The data products are built for the management team, business operations team, product managers and also external sellers to make smarter decisions with the data.
- Participate in or be responsible for data model design, development and rollout of PB-level e-commerce data warehouse.
- Participate in or be responsible for the construction of data modeling methodology, data pipeline development specifications, data quality specifications, metrics system and data stability.
- Translate data product requirements into scalable technical data solutions with low latency and high concurrency.
- Design and build real-time/batch data pipelines on big data ecosystems such as Spark, Flink, Hbase, Druid, ElasticSearch and more.
- Build a high-quality team, help team members to grow their technical skills and improve their work efficiency.
- Bachelor's or higher degree in Computer Science or related technical field.
- 8+ years of experience in building batch or streaming production pipeline using Spark, MapReduce, Flink with big data sets.
- 8+ years of experience in building Data Warehouses in internet companies.
- Proficient in writing Advanced SQLs, expertise in performance tuning of SQLs.
- Experience with data science and machine learning tools and technologies is a plus.
- Knowledge & experience with Hadoop big data ecosystem(e.g. Hadoop, Spark, Airflow, Hive, Presto, Druid, Kafka and others)
- Team leading experience.
- Communication and leadership experience, with experience in initiating and driving projects.