Write select participation scores to statsd

[jimchamp]

Advances #12281

Gathers and writes the following statistics to statsd:

• Total lists created during the last hour
• Total distinct list creators during the last hour
• Total star ratings left during the last hour
• Total distinct star raters during the last hour
• Total reading log events during the last hour
• Total distinct patrons logging books during the last hour

Reading log event counts are broken down by shelf name.

These stats will only be gathered/persisted during the hourly store counts cron jobs (where ndays == 1).

Data returned by the new function will look something like this:

{
  'star_ratings': 7, 
  'distinct_star_ratings': 2, 
  'reading_logs': {
    'want_to_read': 80, 
    'currently_reading': 38, 
    'already_read': 8, 
    'stopped_reading': 0
  }, 
  'distinct_reading_logs': {
    'want_to_read': 52, 
    'currently_reading': 10, 
    'already_read': 3, 
    'stopped_reading': 0
  }, 
  'list_counts': 3, 
  'distinct_list_counts': 3
}

Technical

Testing

Screenshot

Stakeholders

[Copilot]

Pull request overview

This PR adds hourly participation metric collection (lists created, star ratings, and reading log events/unique users) and publishes those counts to StatsD as part of the existing openlibrary/admin/stats.py cron workflow when ndays == 1.

Changes:

• Add a new openlibrary/admin/vitals.py module to query hourly participation counts from Postgres and emit them as StatsD gauges.
• Hook the new participation-score gathering + StatsD publishing into the existing admin stats cron script for the hourly run.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File	Description
openlibrary/admin/vitals.py	New participation metric queries and StatsD gauge emission logic
openlibrary/admin/stats.py	Invokes participation metric gathering/emission during the hourly (ndays == 1) cron execution

[mekarpeles]

prefix of stats.ol.

[mekarpeles]

Is there any way for some of these queries to live on the core/models themselves (e.g. classmethod)?

[jimchamp]

I think so. I'll place the queries in the appropriate openlibrary.core modules.

[jimchamp]

I moved the ratings query and the reading log query to Ratings and Bookshelves, respectively.

I'm not sure where to put the list count query. We get list data from Infogami, so there is no existing openlibrary.core module for list-related DB queries (happy to create one, if wanted).

We could add this function to numbers.py, though it may be a bit out-of-place there. If we do that, it may make sense to move the remaining functions from vitals.py to stats.py.

I've no strong opinions either way...

[mekarpeles]

Looks much cleaner, thank you for leading this!