Staff Engineer - Data Platform
Someone at staff level and ideally with some experience of building / leading a team, or someone currently in a director level role but looking for a hands-on position.
We’re looking for someone who will onboard and start having an early impact by addressing some of our current tactical issues - so needs to be able to operate as an individual contributor as well as coordinate the work of others. And work from there to build out a strategic roadmap and grow out a data platform infrastructure team.
- Execute on tactical issues with our current data pipelines: address development velocity issues, consolidate and converge approaches e.g. move to managed airflow on AWS
- Develop and maintain a detailed execution roadmap as we tackle our legacy monolith and move data and processing into the cloud
- Build out our data architecture to meet low latency, batch and other emerging use cases
- Drive observability and cost optimization
Technical skills required
- Sufficient skills overlap as an IC with our current technical stack:
- SQL, S3-based data lakes, AWS Athena, AWS Glue
- AWS DMS, CDC and other replication approaches
- Hadoop; Spark/Scala; AWS EMR
- Kafka stream processing
- AWS; Terraform
- Oracle; AWS RDS
- Prior experience of building out data warehouse solutions to meet disparate use cases. In particular experience with approaches to fulfilling low latency (sub second) use cases.
- Awareness of the current big data technology landscape including hadoop and cloud datawarehouse solutions such as Snowflake or Databricks
Soft skills required
- Self starter and able to deal with a certain level of ambiguity
- Delivery focused and able to convey a sense of urgency to a team
- Ability to partner with senior/staff engineers to move our tech stack forward.
- Foster a healthy and collaborative environment aligned with Kobalt values.