Page MenuHomeKCVelaga_WMFJul 8 2024, 9:42 AM
Tags
Referenced Files
None
Subscribers
KCVelaga_WMF
SonjaPerry

Description

Need
Automoderator has been deployed on the the first pilot wikis, with plans to rollout to couple of other wikis soon. At the moment we're tracking the pilot metrics manually with a weekly run. Eventually, a publicly accessible dashboard as communities would also be interested in AM's activity is needed, that is updated at least daily.

Solution

  • A public Superset dashboard to monitor activity of Automoderator, both by the team and the community as required.
  • Addition of new wikis, which mainly a work on the data pipeline side should be easy.

Steps involved

#StepTaskNotes
1Identify all the metrics (+dependent data sources) to be tracked with v1 of the dashboardT371981Pilot metrics are in T362610 (weekly reports)
-Make an exhaustive list of available metrics for future consideration✅
2Iterate based on the initial dashboard mockup designmockup based on testwiki (iterations can be done after the data is available)
3Develop Airflow-based pipelines for the necessary metrics to be calculatedT374705sub-tasks: T375062, T375153
4Privacy review of the data to be published (L3SC request as it would be Medium Risk according to request ✅
5Jobs to publish the datasets to https://analytics.wikimedia.org/published/datasets/ -this will not be a separate job
6Jobs to read and update Toolforge ToolsDB from the published datasetsT374705
7Develop the queries required for the dashboard on Superset✅
8Build and publish the first version of the dashboard & communicate✅

Note: This is a goal for FY 24 Q1, so this should be ready by the end of quarter.

Details

Due Date
Sep 29 2024, 6:30 PM

Related Objects
Search...

StatusSubtypeAssignedTask
 ResolvedKCVelaga_WMFT369488 Develop a unified Automoderator Activity Dashboard (v1)
 ResolvedKCVelaga_WMFT371981 Finalize metrics for Automoderator activity dashboard (v1)
DuplicateKCVelaga_WMFT373415 Develop queries for Automoderator activity dashboard metrics (v1)
ResolvedKCVelaga_WMFT374705 Develop ETL pipelines to calculate metrics for Automoderator Activity Dashboard (v1)
 ResolvedKCVelaga_WMFT375062 ETL pipeline to update Automoderator config (weekly)
 ResolvedKCVelaga_WMFT375153 ETL pipeline for Automoderator daily monitoring metrics
 ResolvedKCVelaga_WMFT375480 ETL pipeline for Automoderator monthly key metrics
 ResolvedKCVelaga_WMFT375405 Develop a script to classify wikis for small wikis filter for Automoderator metrics
 ResolvedKCVelaga_WMFT376456 ETL pipeline to calculate potential vandalism handed by Automoderator