Consumer Edge is the leading provider of consumer data for some of the largest hedge funds, venture capital and private equity firms, and corporations in the United States and Europe. We arm our clients with actionable consumer, competitive, and market insights that drive better investment and strategic decisions.
Consumer Edge is at its crux a data company. We deal with collecting, aggregating, wrangling, modeling, and then surfacing important data that our clients use to make business-driving decisions. That means that our Data Engineering team sits at the center of the business and plays a huge role in its success.
Data Engineers at CE own and are responsible for the entire data lifecycle: ingestion, ETL, warehousing, and reporting. We are data pipeline builders and data wranglers who enjoy optimizing data systems and building them from the ground up. We are looking to add another person to our team that will own every single piece of a data stack and be able to plug and play at every single step of the way. We support CE’s Developers, Product team, Data Analysts and Data Scientists on data initiatives and we ensure optimal data delivery architecture is consistent throughout ongoing projects.
Responsibilities For Data Engineer
- Assemble large, complex data sets that meet functional / non-functional business requirements.
- Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
- Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using a combination of SQL, GCP, and open-source big data’ technologies.
- Communicate insights and recommendations to key stakeholders, engineering, data science and product partners
- Work in a supportive team that offers engineers the flexibility to be creative and chase interesting ideas
- Work closely with the product manager, end-users and stakeholders to understand, document, troubleshoot and analyze requirements for complex data solutions
- Design and build data integration methods to guarantee accuracy as well as accessibility of all valuable data while understanding what data is important for the business and why.
- Provide technical assistance and cross training to other team members and help with operational data needs when required
- Participate in data architecture and engineering decision making/planning
Qualifications For Data Engineer
- 1-5 years of experience as Data Engineer or a similar role
- Experiencing dealing with large volumes of data, preferably successfully managing multiple heterogenous datasets into a unified data warehouse
- Creating ELT or ETL production data pipelines using Python/SQL
- Working knowledge of containers and workflow orchestration/automation tools: Prefect or Airflow. Bonus: DBT (com)
- AWS, GCS and/or Azure cloud-based data stacks, specifically experience with large-scale databases such as BigQuery, Redshift, or Snowflake
- Expert knowledge in SQL (window functions, partitioned or clustered databases, MPP, etc).
- Python data analysis libraries, such as Pandas and NumPy
- Bonus points if you feel comfortable working with DevOps tooling
What you will have at Consumer Edge:
- Competitive Salary
- Work-from-home flexibility. As of Q2 2022, CE is remote-first, with an office in Midtown Manhattan
- 401k with employer match
- Flexible vacation and unlimited sick days
- Paid family leave
- An incredible product & powerful data that “wows” clients
- Great people: surround yourself with a team of people with a shared vision & focus, drive, a passion for CE’s customers, and camaraderie
- Career growth opportunities