Senior Software Engineer, Finance, Ecommerce, Data Pipeline
We are building a dynamic financial platform that operates at Wayfair’s scale of growth across the US, UK, and EU. Thinking outside the box to apply innovative solutions is not only encouraged, but expected. Technology on our team is growing at an accelerating pace as we rethink what it means to provide actionable insight to every line of business within Wayfair, as well as our external suppliers and partners
The Real-Time Visibility Team, part of Wayfair Finance Engineering, is focused on real time profitability and performance reporting. We provide Revenue and Cost metrics for every item that is viewed, stocked, and sold across all our brands. Having these real-time profitability metrics is paramount to running a successful, and profitable, business. The data Real-Time produces is consumed by all groups at Wayfair; we feed into real time pricing, business intelligence, and financial reporting.
Having a wide group of consumers requires speed and accuracy. We have adopted both batch and streaming technologies and our team utilizes new technologies such as Kafka and AeroSpike. We were recently able to re-analyze 1.25 years’ worth of data in less than 12 hours. Impressive considering, we estimated our legacy system would take 1,500 hours to do the same amount of work. The Real-Time Visibility team creates most of the software to support Wayfair’s global Finance team, and continues to push the limits of data processing at Wayfair with a build over buy mentality. We expect to be the go-to-group when it comes to data pipeline processing.
Real-Time Visibility looks to maintain processing speed by defining SLAs and Throughput Targets. Some of our recent projects entail:
- Transition to Python for large scale, parallel, data processing. Converting our legacy monolithic processing procedure into modular components with a focus on speed and simplicity.
- Incorporate real time, distributed, message queues for more resilient pipelines.
- Develop real time data validation and alerting to ensure that users are aware of anomalies AS they are happening. This requires building intelligent models to both monitor and learn from our real-time data processing.
- Develop a data streaming pipeline framework, Streampipe. The framework is used to manage multiple streaming jobs as part of a larger pipeline. The framework allows users to build processing components and hook into the larger system using a simplified configuration. The entire system is built on top of a distributed message system which allows for impressive horizontal scalability and speed.
What You'll Do
- Mentor new hires and other engineers to help them “level up” and become more proficient by example, tech talks, pairing, and other avenues to increase technical efficiency across the organization
- Handle ambiguity with limited oversight; leverage technical acumen, experience, and network to answer questions and remove roadblocks
- Propose and own initiatives to completion while balancing various tradeoffs including speed to delivery vs ongoing maintainability
- Build scalable applications with Python, PHP or Scala, while managing database schema and creating complex queries, aggregate, joins, etc. when needed
- We are currently working on building new systems that scale horizontally, so experience with Storm/Spark & Kafka is a plus, along with experience using cloud technologies
- Additionally, the Finance Engineering team builds and maintains web-based tools used by our vendors and finance team, so you will have exposure to integrating with client-side technologies like React/JS, CSS & HTML as well
What You'll Need
- 5+ years of professional experience as a full stack, full lifecycle software engineer
- 3+ years of experience working with Object Oriented Programming (OOP)
- 3+ years of creating complex SQL (Microsoft, Oracle, IBM, must be willing to learn T-SQL) queries & relational database schema design
- Proven ability to collaborate with product teams to gather requirements and create roadmaps
- Excellent communication skills and ability to share technical concepts with non-technical business users
Added bonus, but not required:
- Knowledge of Generally Accepted Accounting Principles (GAAP)
- Understanding of retail operations and ecommerce
- Previous experience supporting finance & accounting operations
- Comfortable working with large & complex datasets
- Exposure to scaling architecture
- Experience working with Spark & Kafka or other distributed data processing systems