Wikitech-l March 2023

51 participants
47 discussions

EBNF grammar project status?
by Steve Bennett 01 Apr '25

01 Apr '25

What's the status of the project to create a grammar for Wikitext in EBNF? There are two pages: http://meta.wikimedia.org/wiki/Wikitext_Metasyntax http://www.mediawiki.org/wiki/Markup_spec Nothing seems to have happened since January this year. Also the comments on the latter page seem to indicate a lack of clear goal: is this just a fun project, is it to improve the existing parser, or is it to facilititate a new parser? It's obviously a lot of work, so it needs to be of clear benefit. Brion requested the grammar IIRC (and there's a comment to that effect at http://bugzilla.wikimedia.org/show_bug.cgi?id=7 ), so I'm wondering what became of it. Is there still a goal of replacing the parser? Or is there some alternative plan? Steve

26 217

Phabricator upstream shutdown
by Brian Wolff 21 Mar '25

21 Mar '25

It sounds like phabricator upstream is going away: https://admin.phacility.com/phame/post/view/11/phacility_is_winding_down_op… Just curious, are we planning to continue using it long term or move to something else? -- Brian

10 12

Missing Section Headings
by Marc Riddell 12 Sep '24

12 Sep '24

Hello, I have been a WP editor since 2006. I hope you can help me. For some reason I no longer have Section Heading titles showing in the Articles. This is true of all Headings including the one that carries the Article subject's name. When there is a Table of Contents, it appears fine and, when I click on a particular Section, it goes to that Section, but all that is there is a straight line separating the Sections. There is also no button to edit a Section. If I edit the page and remove the "== ==" markers from the Section Titles, the Title then shows up, but not as a Section Heading. Also, I don't have any Date separators on my Want List. This started 2 days ago. Any thoughts? Thanks, Marc Riddell [[User:Michael David]]

10 11

How to make commits from mirrored Wikimedia repos show up on your GitHub profile
by Bartosz Dziewoński 27 Jun '24

27 Jun '24

I know it has been annoying a couple of people other than me, so now that I've learned how to make it work I'll share the knowledge here. tl;dr: Star the repositories. No, seriously. (And yes, you need to star each extension repo separately.) (Is there a place on mw.org to put this tidbit on?) ------- Forwarded message ------- From: "Brian Levine" <support(a)github.com> (GitHub Staff) To: matma.rex(a)gmail.com Cc: Subject: Re: Commits in mirrored repositories not showing up on my profile Date: Tue, 09 Jul 2013 06:47:19 +0200 Hi Bartosz In order to link your commits to your GitHub account, you need to have some association with the repository other than authoring the commit. Usually, having push access gives you that connection. In this case, you don't have push permission, so we don't link you to the commit. The easy solution here is for you to star the repository. If you star it - along with the other repositories that are giving you this problem - we'll see that you're connected to the repository and you'll get contribution credit for those commits. Cheers Brian -- Matma Rex

3 3

Research FAQ gets a facelift
by Dario Taraborelli 25 Jun '24

25 Jun '24

We just released a new version of Research:FAQ on Meta [1], significantly expanded and updated, to make our processes at WMF more transparent and to meet an explicit FDC request to clarify the role and responsibilities of individual teams involved in research across the organization. The previous version – written from the perspective of the (now inactive) Research:Committee, and mostly obsolete since the release of WMF's open access policy [2] – can still be found here [3]. Comments and bold edits to the new version of the document are welcome. For any question or concern, you can drop me a line or ping my username on-wiki. Thanks, Dario [1] https://meta.wikimedia.org/wiki/Research:FAQ [2] https://wikimediafoundation.org/wiki/Open_access_policy [3] https://meta.wikimedia.org/w/index.php?title=Research:FAQ&oldid=15176953 *Dario Taraborelli *Head of Research, Wikimedia Foundation nitens.org • @readermeter <http://twitter.com/readermeter>

2 1

Announcing: Path review board
by Brian Wolff 04 Oct '23

04 Oct '23

Yesterday there was a conversation about code review on irc and among other things, how sometimes patches can get "stuck". I had an idea for a way to improve things. I'm not sure if it is a good idea, but there's only one way to find out. So without further ado, announcing the Code Review Patch Board: https://www.mediawiki.org/wiki/Code_review/patch_board In short - each person is allowed to list one of their patches on the board that they would really like to see reviewed. You can only list one patch at a time, and it should be a patch that you have been unable to get review for for at least a week through normal means. See the page for the full list of guidelines. I encourage people to give it a try. Add a patch you wrote that you cannot get a review for. Or if you have +2 rights, try giving some love to these underloved patches. I would also love to hear feedback on the general idea as well as the current guidelines. To repeat, the url is: https://www.mediawiki.org/wiki/Code_review/patch_board Thanks, bawolff

4 5

Service Decommission Notice: Mobile Content Service - July 2023
by Adam Baso 06 Jul '23

06 Jul '23

TL;DR: The legacy Mobile Content Service is going away in July 2023. Please switch to Parsoid or another API before then to ensure service continuity. Hello World, I'm writing about a service decommission we hope to complete mid-July 2023. The service to be decommissioned is the legacy Mobile Content Service ("MCS"), which is maintained by the Wikimedia Foundation's Content Transform Team. We will be marking this service as deprecated soon. We hope that with this notice, people will have ample time to update their systems for use of other endpoints such as Parsoid [1] (n.b., MCS uses Parsoid HTML). The MCS endpoints are the ones with the relative URL path pattern /page/mobile-sections* on the Wikipedias. For examples of the URLs see the "Mobile" section on the online Swagger (OpenAPI) specification documentation with matching URLs here: https://en.wikipedia.org/api/rest_v1/#/Mobile == History == The Mobile Content Service ("MCS") is the historical aggregate service that originally provided support for the article reading experience on the Wikipedia for Android native app, as well as some other experiences. We have noticed that there are other users of the service. We are not able to determine all of the users, as it's hard to tell with confidence from the web logs. The Wikimedia Foundation had already transitioned the Wikipedia for Android and iOS apps to the newer Page Content Service ("PCS") several years ago. PCS has some similarities with MCS in terms of its mobility focus, but it also has different request-response signatures in practice. PCS, as with MCS, is intended to primarily satisfy Wikimedia Foundation-maintained user experiences only, and so this is classified with the "unstable" moniker. == Looking ahead == Generally, as noted in the lead, we recommend that folks who use MCS (or PCS, for that matter) switch over to Parsoid for accessing Wikipedia article content programmatically for the most predictable service. The HTML produced by Parsoid has a versioned specification [2] and because Parsoid is accessed regularly by a number of components across the globe tends to have fairly well cached responses. However, please note that Parsoid may be subject to stricter rate limits that can apply under certain traffic patterns. At this point, I do also want to note that in order to keep up with contemporary HTML standards, particularly those favoring accessibility and machine readability enhancements, Parsoid HTML will undergo change as we further converge parsing stacks [3]. Generally, you should expect iteration on the Parsoid HTML spec, and of course as you may have come to appreciate that the shape of HTML in practice can vary nontrivially wiki-by-wiki as practices across wikis vary. You may also want to consider Wikimedia Enterprise API options, which range from no cost to higher volume access paid options. https://meta.wikimedia.org/wiki/Wikimedia_Enterprise#Access == Forking okay, but not recommended == Because MCS acts as a service aggregate and makes multiple backend API calls, caveats can apply for those subresources - possibility of API changes, deprecation, and the like. We do not recommend a plain fork of MCS code because of the subresource fetch behavior. This said, of course you are welcome to fork in a way compatible with MCS's license. == Help spread the word == Although we are aware of the top two remaining consumers of MCS, we also are not sure who else is accessing MCS and anticipate that some downstream tech may break when MCS is turned off. As we are cross-posting this message, we hope most people who have come to rely upon MCS will see this message. Please feel free to forward this message to contacts if you know they are using MCS. == Help == Although we intend to decommission MCS in July 2023, we would like to share resources if you need some help. We plan to hold office hours in case you would like to meet with us to discuss this or other Content Transform Team matters. We will host these events on Google Meet. We will provide notice of these office hours on the wikitech-l mailing list in the coming weeks and months. Additionally, if you would like to discuss your MCS transition plans, please visit the Content Transform Team talk page: https://www.mediawiki.org/wiki/Talk:Content_Transform_Team Finally, some Content Transform Team members will also be at the Wikimedia Hackathon [4] if you would like some in-person support. Thank you. Adam Baso (he/him/his/Adam), on behalf of the Content Transform Team Director of Engineering Wikimedia Foundation [1] https://www.mediawiki.org/wiki/Parsoid [2] https://www.mediawiki.org/wiki/Specs/HTML [3] https://www.mediawiki.org/wiki/Parsoid/Parser_Unification/Updates [4] https://www.mediawiki.org/wiki/Wikimedia_Hackathon_2023

3 7

Wikimedia developer satisfaction survey 2023 🧑‍💻📊
by Tyler Cipriani 01 Jun '23

01 Jun '23

Hello! Please take our annual* *Developer Satisfaction Survey*! Link: https://wikimediafoundation.limesurvey.net/484133 The survey is open until Fri, 17 Feb 2023—two weeks from today. ____ This survey is for members of the *Wikimedia Developer Community* and covers the following topics: - Code review tooling and process - Code quality - Phabricator - Continuous Integration - MediaWiki development environments - Beta cluster / Staging Please take the survey if you’ve used the above tools as part of your role developing software for the Wikimedia community. We’re soliciting your feedback to: - Measure developer satisfaction, and - determine where to invest resources in the future We will anonymize, explore, and report the data we gather on mediawiki.org. View previous years' survey results: https://www.mediawiki.org/wiki/Developer_Satisfaction_Survey Privacy statement: This survey will be conducted via a third-party service, which may subject it to additional terms. For more information on privacy and data-handling, see the survey privacy statement <https://foundation.wikimedia.org/wiki/Legal:2023_Developer_Satisfaction_Sur…> . Thank you! Tyler Cipriani (he/him) Engineering Manager, Release Engineering Wikimedia Foundation ____ *: “annual,” except we missed 2022 🙁

2 3

March 2023 Datacenter Switchover
by Clément Goubert 20 Apr '23

20 Apr '23

Dear Wikitechians, On *Wednesday March 1st*, the SRE team will run a planned data center switchover, moving all wikis from our primary data center in Virginia to the secondary data center in Texas. This is an important periodic test of our tools and procedures, to ensure the wikis will continue to be available even in the event of major technical issues in our primary home. It also gives all our SRE and ops teams a chance to do maintenance and upgrades on systems in Virginia that normally run 24 hours a day. The switchover process requires a *brief read-only period for all Foundation-hosted wikis*, which will start at *14:00 UTC on Wednesday March 1st*, and will last for a few minutes while we execute the migration as efficiently as possible. All our public and private wikis will be continuously available for reading as usual, but no one will be able to save edits during the process. Users will see a notification of the upcoming maintenance, and anyone still editing will be asked to try again in a few minutes. CommRel has already begun notifying communities of the read-only window. A similar event will follow a few weeks later, when we move back to Virginia. This is currently scheduled for *Wednesday, April 26th*. If you like, you can follow along on the day in the public #wikimedia-operations channel on IRC (instructions for joining here <https://meta.wikimedia.org/wiki/IRC/Instructions>). To report any issues, you can reach us in #wikimedia-sre on IRC, or file a Phabricator ticket with the *datacenter-switchover* tag (pre-filled form here <https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?projects=Data…>); we'll be monitoring closely for reports of trouble during and after the switchover. (If you're new to Phab, there's more information at Phabricator/Help.) The switchover and its preparation are tracked tracked in Phabricator Task T327920 <https://phabricator.wikimedia.org/T327920> On behalf of the SRE team, please excuse the disruption, and our thanks to everyone in a number of departments who've been involved in planning this work for the past weeks. Feel free to reply directly to me with any questions. Thank you, -- Clément Goubert (they/them) Senior SRE Wikimedia Foundation

7 9

Technical Decision Making Retro
by Tajh Taylor 20 Apr '23

20 Apr '23

*Fixing my list addressing errors...* TLDR: The Foundation will be conducting a retrospective on the Technical Decision Making Process. To the entire Wiki technical community, For quite some time now, we have experienced issues with the Technical Decision Making Process (TDMP). Volunteer contributors and staff have asked if we are still operating the Technical Decision Forum (TDF, the member body that participates in the TDMP). Communication about it from the Foundation has been inconsistent, and interest from the volunteer community in joining has been low. Some of our most senior engineers on Foundation staff have expressed that the process is flawed, doesn’t create room for discussion about the technical issues surrounding a decision, and doesn’t ensure participation by all stakeholders who may be affected by the decision. Suffice it to say, the current state of affairs leaves many participants wanting more. We must also remind ourselves of the purpose of a decision making process. The decisions are not meant to be random or isolated. They should be aligned to our technical strategy, and we should be able to look at the decisions we have made and understand how they advance our progress against that strategy. If the process is working as it should, the decisions that are produced should represent settled wisdom, and not need to be revisited too quickly. The goals for a well-run process include: - A straightforward, widely understood decision making process, that - Facilitates impactful technical decisions to be made in a timely manner, - Incorporates input from staff and volunteers in our technical community, with - Decisions that align with accountability for decision outcomes, and - Clear communication and transparent operations throughout the process. On examination of the contributing factors that have led us to this point, the factor that stands out to me is the need for clear accountability: accountability for the TDMP itself and accountability for each of the decisions we make. Technical decision making, beyond a certain magnitude, is a core organizational process for any engineering organization. It is therefore important for us to examine and improve this process from time to time to ensure organizational effectiveness. Not unrelated, regular retrospectives are a routine agile software engineering practice to enact continuous improvement. To keep our decision making process effective and efficient, we need to conduct regular retros. Overall accountability for maintaining an effective decision making process should rest with a person who is sufficiently able to marshal resources and address problems at a large scale – here at the Foundation, that resides in the executive level. The Foundation will be conducting a retro on the TDMP over the next couple of months. Because we don’t yet have a habit of doing retros on this process, and because there is a wide range of stakeholders we seek to hear from, the process will be a bit more structured than an ordinary retro, and will take more time. As we do more of these, we should get better at them. The feedback gathered through the retro will be used to make changes to improve the TDMP. Foundation staff will follow up with more information about the kickoff of the retro and what steps will follow. I am looking forward to wide participation in this retro. Here are the links to the relevant wiki page and Phab ticket: - Wiki page <https://www.mediawiki.org/wiki/Technical_decision_making/Technical_Decision…> - Phabricator ticket <https://phabricator.wikimedia.org/T333235> Thank you! And apologies for all the crossposting. Tajh Taylor (he/him/his) VP, Data Science & Engineering Wikimedia Foundation <https://wikimediafoundation.org/>

4 4

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l March 2023