About the Role
Being a Lead Data Engineer at Smarkets means leading our newly-formed Data Team. The Data Team takes ownership of multiple data pipelines and systems that enable us to quickly and effectively capture data and analyse trends, giving us control and insight relating to various aspects of the business. The data team also sits in the middle of the other engineering teams at Smarkets and works closely with them to ensure the data needs of their broad domains (fraud detection, trading analytics, ledger accounting, customer insights and marketing, etc) are met, and that any possible commonalities and synergies are exploited.
You’ll be responsible for: architecting future analytical databases and data pipelines, advancing the data tooling, and helping to spread domain knowledge. Aside from the longer term projects the data engineers work on continually improving the system in regards to collection, processing, storage and dissemination of data across the business. This is in conjunction with other engineering teams and a business intelligence function.
Our present core current tech stack consists of Linux, Docker, Kubernetes, Python, Postgres, Kafka and Redshift.
- You’ll be familiar with writing code in Python.
- You’ll enjoy collaborating with different teams to help better define the systems you’ll be creating
- You’ll have dealt with with large volumes of data and have a great track record of building automated, scalable and robust data processing systems
- You’ll have a good understanding of database technologies and you know the practical and theoretical difficulties of building distributed systems
- You’ll have worked with data warehouse systems like BigQuery or Redshift and both batch and semi-online building blocks like MapReduce, Spark, Kinesis, dataflow etc.
- Ideally you’ll have experience with managing other data engineers and data scientists and a history of building a data function and data-driven culture within a company
Smarkets brings a professional approach to the sports betting world which has allowed us to create one of the most sophisticated products in the industry. We embrace collaboration and our culture of trust, innovation, and scientific rigour has enabled us to grow to where we are today. Our people have a large impact on the product - be it changing the direction of our technical stack or pushing new features - you’ll be key in driving Smarkets forward.
Our culture is built on the principles of self-management and rewards people on merit and excellence, avoiding the bureaucracy of pyramid management. Ultimately we are a group of like-minded individuals who enjoy learning, playing, and eating the excellent meals provided by the chefs!