Distributed Systems Architect
Seeking a Senior Tech Lead/Architect to join the Data Infrastructure organization at Wayfair focused on large scale distributed systems. The team runs and provides a variety of horizontally scalable batch, interactive, and streaming data management platforms. Our main responsibility is building and operating systems for data transport, data reliability and high availability of a wide array of structured and unstructured data streams and data processing/compute for the purpose of providing analytical insights and enhancing the customer experience and engagement. The Data Warehousing Team is responsible for maintaining distributed real-time and batch Analytics processing environments at Wayfair and associated services.
The ideal candidate is an expert in building Big Data Analytics infrastructure and services both on-prem and in the public cloud (experience with GCP is a plus). The candidate will be responsible for working with key stakeholders inside the company, building strong relationships and ensuring that our business goals and objectives are met.
What You'll Do
- You have the ability to architect large distributed applications at scale and understand how reliability of the computing environment impacts these applications, and how to guide application developers to make good choices in their designs.
- You will regularly generate and lead engineering efforts to develop and execute on new technical or architectural ideas that have a material and lasting impact on business objectives, reducing the latency of development, increasing the efficiency of development, increasing the number of initiatives that can safely run in parallel, or otherwise broadly increase the leverage of individual engineers across Wayfair. You need to demonstrate a depth and breadth of systems-building and end-to-end data pipeline architectures
- You will own and aggressively drive forward specific areas of Big Data stack (Hadoop/HDFS/Hive/Presto/Spark), Messaging (Kafka primarily) and Data Warehousing (Vertica & BigQuery) technology architecture. This includes data management architectures involving batch, micro-batch, and real-time streaming of data for both cloud and on-premise solutions
- You will work with Product Owner/Business Analysts to understand functional requirements and interact with other cross-functional teams to architect, design, develop, test, and release end-user services. You will work with your management and peers to drive common vision, practices and capabilities across teams. You have the ability to connect the technology to the business objectives for both non-technical audiences and junior staff who will be performing much of the work. This will most often cross team boundaries even within the technical teams.
- You have broad knowledge on platform automation, system configuration, CI/CD pipelines and measurement supporting performance analysis and reliability.
- You will help to define and represent Wayfair’s engineering brand externally - engineers want to join Wayfair specifically to work with you . You will work to represent Wayfair as a technology-leading shop through speaking at conferences, organizing meetups, etc
- You are recognized by other engineers as an expert mentor and teacher; consistently in demand for design and code review; regularly provide training and support for new methods, tools, and patterns
- You will significantly contribute to shaping the tech stack of the company; have significant influence in making major "organization-wide" architectural choices and scaling out repeatable architectural patterns to other teams
What You'll Need
- Minimum 10 years of professional experience with Distributed Systems and Data Management technologies and a minimum of 5 years of Hadoop, Kafka, and Open-Source data management technologies.
- Experience with Cloud Big Data & Messaging systems and architecture
- Expert programmer in at least 2 major languages (java and python preferably)
- Experience designing and architecting Big Data Pipelines supporting Business Intelligence & Data Science Use Cases.
- Proficient in authoring, editing and presenting technical documents
- Outstanding written and oral communication skills; ability to communicate effectively with technical and non-technical staff
- Excellent interpersonal and team building skills
- A positive, people-oriented and energetic attitude
- An interest in working hard and being challenged in a fast-paced environment, and having fun while doing it
Wayfair is one of the world’s largest online destinations for the home. Whether you work in our global headquarters in Boston or Berlin, or in our warehouses or offices throughout the world, we’re reinventing the way people shop for their homes. Through our commitment to industry-leading technology and creative problem-solving, we are confident that Wayfair will be home to the most rewarding work of your career. If you’re looking for rapid growth, constant learning, and dynamic challenges, then you’ll find that amazing career opportunities are knocking. No matter who you are, Wayfair is a place you can call home. We’re a community of innovators, risk-takers, and trailblazers who celebrate our differences, and know that our unique perspectives make us stronger, smarter, and well-positioned for success. We value and rely on the collective voices of our employees, customers, community, and suppliers to help guide us as we build a better Wayfair – and world – for all. Every voice, every perspective matters. That’s why we’re proud to be an equal opportunity employer. We do not discriminate on the basis of race, color, ethnicity, ancestry, religion, sex, national origin, sexual orientation, age, citizenship status, marital status, disability, gender identity, gender expression, veteran status, or genetic information.