Data Engineer

Summary

The Wikimedia Foundation is looking for a Data Engineer to join our team, reporting to the Director of Data Engineering. As a Data Engineer, you will be responsible for building, maintaining and expanding the shared data infrastructure that powers a big part of decision making in the Foundation as well as the Wiki Movement. This includes everything from building scalable pipelines using big data technology to defining and creating a suite of datasets which adhere to our privacy principles.

We are a fully remote, internationally distributed team. When there isn’t a pandemic around, we see each other in person 2-3 times a year during one of our off-sites (the last few have been in places like Majorca, New York and Prague), the Wikimedia All Hands (once a year in San Francisco), or Wikimania, the annual international conference for the Wiki community.

You will be responsible for:

  • Integrating data from multiple sources to gain insights in areas such as content, traffic, editors, readership and fundraising
  • Building scalable data pipelines in collaboration with other data engineers as well as teams across the foundation including product analytics, platform engineering, survey, research and machine learning teams
  • Designing the shared data platform that supports use cases for critical aspects of the Wikimedia mission: harassment prevention, image classification, bot detection, DDoS attacks flagging and many more
  • Building and maintaining public metrics and datasets
  • Implementing data quality monitoring that alerts the team of possible data issues
  • Implementing a data governance and lineage solution for all Wikimedia data

Skills and Experience:

  • 2+ years of relevant industry experience
  • Advanced working knowledge of SQL, relational databases, query authoring, ideally in a variety of flavors (in our team alone we deal with MariaDB, HiveQL, CassandraQL, Spark SQL and Presto)
  • Experience with one or more programming languages such as Python, Scala, and Java
  • Experience building data pipelines using tools such as Airflow, Spark, Gobblin, Oozie, Yarn
  • Familiarity with stream processing systems using Kafka, Spark streaming and/or Flink
  • Excellent written and verbal communication skills
  • Strong interpersonal and collaboration skills
  • BS or MS degree, preferably in Computer Science, or equivalent work experience

Qualities that are important to us:

  • Commitment to the mission of the organization and our values
  • Commitment to our guiding principles
  • Commitment to diversity, equity, and inclusion
  • Cross-cultural sensitivity and awareness
  • Collaborative working experience

Additionally, we’d love it if you have:

  • Experience with Hadoop
  • Understanding of related disciplines including Machine Learning, Statistics, Privacy  and Algorithms
  • Experience working with site reliability engineers

The Wikimedia Foundation is... 

...the nonprofit organization that hosts and operates Wikipedia and the other Wikimedia free knowledge projects. Our vision is a world in which every single human can freely share in the sum of all knowledge. We believe that everyone has the potential to contribute something to our shared knowledge, and that everyone should be able to access that knowledge, free of interference. We host the Wikimedia projects, build software experiences for reading, contributing, and sharing Wikimedia content, support the volunteer communities and partners who make Wikimedia possible, and advocate for policies that enable Wikimedia and free knowledge to thrive. The Wikimedia Foundation is a charitable, not-for-profit organization that relies on donations. We receive financial support from millions of individuals around the world, with an average donation of about $15. We also receive donations through institutional grants and gifts. The Wikimedia Foundation is a United States 501(c)(3) tax-exempt organization with offices in San Francisco, California, USA.

As an equal opportunity employer, the Wikimedia Foundation values having a diverse workforce and continuously strives to maintain an inclusive and equitable workplace. We encourage people with a diverse range of backgrounds to apply. We do not discriminate against any person based upon their race, traits historically associated with race, religion, color, national origin, sex, pregnancy or related medical conditions, parental status, sexual orientation, gender identity, gender expression, age, status as a protected veteran, status as an individual with a disability, genetic information, or any other legally protected characteristics.

If you are a qualified applicant requiring assistance or an accommodation to complete any step of the application process due to a disability, you may contact us at recruiting@wikimedia.org or (415) 839-6885.

U.S. Benefits & Perks*

  • Health Care Benefits Covered at 100%: We cover 100% of premiums for medical, dental and vision plans for all full time employees and eligible dependents
  • Wellness Reimbursement Program: Up to 1,800.00 USD per year for reimbursement for staff wellness expenses, such as gym fees, educational expenses, and more
  • Technology and Equipment Stipend: In addition to receiving a brand new laptop, monitor, & docking station, all new hires receive 600.00 USD stipend to set up their space for working virtually 
  • Professional Development Program: Up to 750.00 USD reimbursement per year to encourage continuous learning through attending conferences, courses, workshops and the purchase of educational materials 
  • 401(k) Retirement Plan: Employer match of up to 4% of employee contributions dollar for dollar with no vesting period
  • Paid Time Off: Generous paid time off policy of over 45 days, which includes: vacation days, at least one observed holiday a month, sick leave, and volunteer days
  • Flexible Schedules: Options available to balance your personal and remote-work life
  • Silent Fridays: No scheduled meetings so you can get caught up at the end of each week
  • New Parent Leave: Fully paid new parent leave for seven weeks plus an additional five weeks for pregnancy, and flexible options as you embark on your return to work
  • Fertility and Adoption Reimbursement Plan: Reimburses staff up to 5,000.00 USD in expenses per year, with a lifetime maximum of 10,000.00 USD
  • Assistance for those unexpected life events: Long and short term disability, life insurance, and an employee assistance program
  • Pre-tax Savings Plans: Generously funded health savings accounts (HSAs), pre-tax contribution options for health care, child care & elder care, public transportation and parking expenses

Please note that for remote roles located outside of the U.S., we defer to our PEO to ensure alignment with local labor laws.

More information

Wikimedia Foundation
Applicant Privacy Policy
Blog
Wikimedia 2030
Our Commitment to Equity
This is Wikimedia Foundation 
Facts Matter
Our Projects
Our Tech Stack

 

Apply for this Job

* Required
  
  


U.S. Equal Opportunity Employment Information (Completion is voluntary)

Individuals seeking employment at Wikimedia Foundation are considered without regards to race, color, religion, national origin, age, sex, marital status, ancestry, physical or mental disability, veteran status, gender identity, or sexual orientation. You are being given the opportunity to provide the following information in order to help us comply with federal and state Equal Employment Opportunity/Affirmative Action record keeping, reporting, and other legal requirements.

Completion of the form is entirely voluntary. Whatever your decision, it will not be considered in the hiring process or thereafter. Any information that you do provide will be recorded and maintained in a confidential file.

Race & Ethnicity Definitions

If you believe you belong to any of the categories of protected veterans listed below, please indicate by making the appropriate selection. As a government contractor subject to Vietnam Era Veterans Readjustment Assistance Act (VEVRAA), we request this information in order to measure the effectiveness of the outreach and positive recruitment efforts we undertake pursuant to VEVRAA. Classification of protected categories is as follows:

A "disabled veteran" is one of the following: a veteran of the U.S. military, ground, naval or air service who is entitled to compensation (or who but for the receipt of military retired pay would be entitled to compensation) under laws administered by the Secretary of Veterans Affairs; or a person who was discharged or released from active duty because of a service-connected disability.

A "recently separated veteran" means any veteran during the three-year period beginning on the date of such veteran's discharge or release from active duty in the U.S. military, ground, naval, or air service.

An "active duty wartime or campaign badge veteran" means a veteran who served on active duty in the U.S. military, ground, naval or air service during a war, or in a campaign or expedition for which a campaign badge has been authorized under the laws administered by the Department of Defense.

An "Armed forces service medal veteran" means a veteran who, while serving on active duty in the U.S. military, ground, naval or air service, participated in a United States military operation for which an Armed Forces service medal was awarded pursuant to Executive Order 12985.


Form CC-305

OMB Control Number 1250-0005

Expires 05/31/2023

Voluntary Self-Identification of Disability

Why are you being asked to complete this form?

We are a federal contractor or subcontractor required by law to provide equal employment opportunity to qualified people with disabilities. We are also required to measure our progress toward having at least 7% of our workforce be individuals with disabilities. To do this, we must ask applicants and employees if they have a disability or have ever had a disability. Because a person may become disabled at any time, we ask all of our employees to update their information at least every five years.

Identifying yourself as an individual with a disability is voluntary, and we hope that you will choose to do so. Your answer will be maintained confidentially and not be seen by selecting officials or anyone else involved in making personnel decisions. Completing the form will not negatively impact you in any way, regardless of whether you have self-identified in the past. For more information about this form or the equal employment obligations of federal contractors under Section 503 of the Rehabilitation Act, visit the U.S. Department of Labor’s Office of Federal Contract Compliance Programs (OFCCP) website at www.dol.gov/ofccp.

How do you know if you have a disability?

You are considered to have a disability if you have a physical or mental impairment or medical condition that substantially limits a major life activity, or if you have a history or record of such an impairment or medical condition.

Disabilities include, but are not limited to:

  • Autism
  • Autoimmune disorder, for example, lupus, fibromyalgia, rheumatoid arthritis, or HIV/AIDS
  • Blind or low vision
  • Cancer
  • Cardiovascular or heart disease
  • Celiac disease
  • Cerebral palsy
  • Deaf or hard of hearing
  • Depression or anxiety
  • Diabetes
  • Epilepsy
  • Gastrointestinal disorders, for example, Crohn's Disease, or irritable bowel syndrome
  • Intellectual disability
  • Missing limbs or partially missing limbs
  • Nervous system condition for example, migraine headaches, Parkinson’s disease, or Multiple sclerosis (MS)
  • Psychiatric condition, for example, bipolar disorder, schizophrenia, PTSD, or major depression

1Section 503 of the Rehabilitation Act of 1973, as amended. For more information about this form or the equal employment obligations of Federal contractors, visit the U.S. Department of Labor's Office of Federal Contract Compliance Programs (OFCCP) website at www.dol.gov/ofccp.

PUBLIC BURDEN STATEMENT: According to the Paperwork Reduction Act of 1995 no persons are required to respond to a collection of information unless such collection displays a valid OMB control number. This survey should take about 5 minutes to complete.