Subject | Repo | Branch | Lines +/- | ||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Track feature-use activity for formatting | mediawiki/extensions/VisualEditor | master | +21 -0 | ||||||||||||||||
Customize query in gerrit
Related Objects
Event TimelineComment Actions If we can instrument all actions in one fell swoop, then we should do that. Instrumenting everything like that will definitely require per-session sampling though, which is a question that @Neil_P._Quinn_WMF can best answer. Neil, what kind of sampling rate should we use for that? Comment Actions
Good plan! Arranged something for later today. Comment Actions Meeting outcome: we seemed happy with the idea that we'd log some specific window-opened events for everything non-annotation, as we're more concerned with interacting with features rather than success. (I.e. "add image", "modify image", "opened the image inspector and didn't modify it", and "opened the add image dialog and then canceled" would all be in the same "opened the image dialog" event.) I'm going to go investigate some technical details of how annotations are set and cleared, to see what events might be triggered. (e.g. does clearing an annotation go through AnnotationAction->clear or surfaceFragment->annotateContent? When toggling via keyboard shortcut vs toolbar button vs context remove-button?) Basically, do we need to add logging to more than just ve.ui.AnnotationAction? Esanders moved this task from Incoming to In progress on the VisualEditor (Current work) board.Aug 25 2018, 6:33 PM2018-08-25 18:33:32 (UTC+0) Comment Actions Summary of what I observed: SurfaceFragment.annotateContent is the underpinning of most of this. Logging on it would catch use of formatting annotations and links, but not using formatting annotations without any text content -- that's done by changing the insertion annotations, not actually setting any annotation. (If we wanted to learn things like how common it is to turn on formatting and then type, versus adding formatting to existing text, the already-existing distinction in AnnotationAction.toggle would make that really easy to log. I don't know if it matters, but it does intuitively seem like a thing which could change greatly between mobile and desktop.) Clearing formatting via the toolbar command (or cmd-\) goes through AnnotationAction. It results in a flurry of annotateContent calls, because it individually clears each one. Link actions use SurfaceFragment.anotateContent directly rather than going through AnnotationAction. The context item's remove-annotation button uses SurfaceFragment directly, unlike the other clear-formatting method. Setting or editing a link via the popup does an annotateContent clear and then set. Because of the live preview, opening the add-link dialog at all gets us a set, though opening it to edit an existing link doesn't. If we log via SurfaceFragment.anotateContent, we should either filter out all the link-related actions in favor of logging attached to the inspector, or exempt link/language dialogs from the inspector logging. That said, it's slightly more work for us in terms of places to add logging to, but I think we'd get cleaner results by logging the various AnnotationAction methods, and separately adding logging to opening the link dialog and the link context's remove-link button. Avoids the mixing up of link annotation actions with the rest of them, and doesn't overrepresent the user-action weight of things like clearing annotations from a heavily-annotated bit of text. (I.e. Neil probably has to do less data-wrangling after we've logged it.) Comment Actions To add to @DLynch's comment, @Esanders proposed adding auto-save recovery to our logging in order to provide a metric that would get us close to understanding how many users are affected by mid-edit memory unloading due to tab or application switching on mobile (especially iOS). Though that is not the only reason we would see autosave recoveries on mobile, it may help us understand whether this is a significant use case and prioritize it appropriately. From the mobile report: Mobile OSes often unload pages from memory if you switch tab or application, especially iOS. This would result in the editor being reloaded when you return to that tab. Fortunately with auto-save this is somewhat less catastrophic, but there may be more work to remember other user state (e.g. if the user was in the middle of creating a citation) nshahquinn-wmf added a comment.Sep 17 2018, 10:35 PM2018-09-17 22:35:53 (UTC+0) Comment Actions
For the record, yes, it will. EventLogging events are automatically bundled with the wiki, among other things.
I don't see any problem with adding these (except potentially data volume, but we'd deal with that by dialing back the sampling rather than removing instrumentation). Auto-save recovery in particular is a bit different from the things we have in there currently, since it's not manually triggered by the user, but that's fine. Comment Actions
It is somewhat conceptually akin to the bits in there already. It's sort of a variant on init or ready, after all. Comment Actions Oh, thing to consider with this: we explicitly disable tracking for any session where we have to ask about restoring the autosave (i.e. any session where a new revision has been made since the autosave happened). This is probably a small source of sessions which have an init and then nothing else, since I think the sequencing would work out that way. nshahquinn-wmf added a comment.Sep 18 2018, 5:14 PM2018-09-18 17:14:56 (UTC+0) Comment Actions
Oh, interesting. Out of curiosity, why do we disable tracking? Would those sessions produce weird data? Comment Actions It's because those sessions stick a confirmation dialog in the middle of the loading process, between init and ready, and pause the process until the user chooses what to do with their autosave. Since most of this stuff was added for us to do performance testing, introducing a fairly randomly large delay into the timing there would throw the numbers off. (The exact method chosen for disabling it is also going to turn out to disable the activity metrics we're adding here. I could work around that if you want, or not if you don't think it's a big deal.) patch for the actual instrumentation, and Comment Actions Change 466695 had a related patch set uploaded (by DLynch; owner: DLynch): Comment Actions Change 466695 merged by jenkins-bot: • marcella moved this task from QA to Product owner review on the VisualEditor (Current work) board.Jan 7 2019, 3:47 PM2019-01-07 15:47:04 (UTC+0) Comment Actions Change 490661 had a related patch set uploaded (by DLynch; owner: DLynch): nshahquinn-wmf added a comment.Feb 14 2019, 6:58 PM2019-02-14 18:58:57 (UTC+0)
@DLynch, if you look at nshahquinn-wmf added a comment.Feb 14 2019, 7:01 PM2019-02-14 19:01:35 (UTC+0)
Oh, your commit message says tables are "the only part of the "insert" menu which wasn't being tracked by the dialog-tracking (apart from opening the properties on an existing table.)" That makes sense. Thank you, this is a very nice addition! Comment Actions Change 490661 merged by jenkins-bot: Comment Actions Change 491389 had a related patch set uploaded (by Bartosz Dziewoński; owner: Bartosz Dziewoński): Comment Actions Change 491389 merged by jenkins-bot: Comment Actions Change 491807 had a related patch set uploaded (by DLynch; owner: DLynch): Comment Actions Change 491807 merged by jenkins-bot: Comment Actions Change 492195 had a related patch set uploaded (by Bartosz Dziewoński; owner: Bartosz Dziewoński): Comment Actions Change 492195 merged by jenkins-bot: |
Tags
Referenced Files
None
Subscribers
Schema:Edit on Meta-Wiki, so computing these statistics will not be possible right now. The pipeline should be instrumented to record the necessary data.
From the description of T202133, the common tasks that need instrumentation adding are:
There are lots of other tasks that could also have instrumentation added, but this is the set we'll start with. The changes to the schema should be checked with @Neil_P._Quinn_WMF to make sure that they will give him the data he needs to compute the statistics.
Details
|