Staff Engineer - Data Platform

Seniority

Someone at staff level and ideally with some experience of building / leading a team, or someone currently in a director level role but looking for a hands-on position.

We’re looking for someone who will onboard and start having an early impact by addressing some of our current tactical issues - so needs to be able to operate as an individual contributor as well as coordinate the work of others. And work from there to build out a strategic roadmap and grow out a data platform infrastructure team.

Responsibilities

  • Execute on tactical issues with our current data pipelines: address development velocity issues, consolidate and converge approaches e.g. move to managed airflow on AWS
  • Develop and maintain a detailed execution roadmap as we tackle our legacy monolith and move data and processing into the cloud
  • Build out our data architecture to meet low latency, batch and other emerging use cases
  • Drive observability and cost optimization

Technical skills required

  • Sufficient skills overlap as an IC with our current technical stack:
  • SQL, S3-based data lakes, AWS Athena, AWS Glue
  • AWS DMS, CDC and other replication approaches
  • Hadoop; Spark/Scala;  AWS EMR
  • Airflow
  • Kafka stream processing
  • AWS; Terraform
  • Oracle; AWS RDS
  • Prior experience of building out data warehouse solutions to meet disparate use cases. In particular experience with approaches to fulfilling low latency (sub second) use cases.
  • Awareness of the current big data technology landscape including hadoop and cloud datawarehouse solutions such as Snowflake or Databricks

Soft skills required

  • Self starter and able to deal with a certain level of ambiguity
  • Delivery focused and able to convey a sense of urgency to a team
  • Ability to partner with senior/staff engineers to move our tech stack forward.
  • Foster a healthy and collaborative environment aligned with Kobalt values.

Apply for this Job

* Required