[DENG-11194] Create graphics_dashboard job, migrated from spark#533
Draft
BenWu wants to merge 4 commits into
Draft
[DENG-11194] Create graphics_dashboard job, migrated from spark#533BenWu wants to merge 4 commits into
BenWu wants to merge 4 commits into
Conversation
ed71382 to
ee73991
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
https://mozilla-hub.atlassian.net/browse/DENG-11194
Note that this is largely AI-generated code and documentation and it looks right to me, but I don't have all the domain knowledge to be certain.
This is a migration of the spark jobs in https://github.com/mozilla/python_mozetl/tree/main/mozetl/graphics. The output is almost the same. The main difference is that it's now using the glean metrics ping instead of the legacy main ping. This makes the output for the graphs not 1:1 but usually close (differences are documented in the readme).
This can be run locally with a copy of the frontend with:
Checklist for reviewer:
Commits should reference a bug or github issue, if relevant (if a bug is referenced, the pull request should include the bug number in the title)
Scan the PR and verify that no changes (particularly to
.circleci/config.yml) will cause environment variables (particularly credentials) to be exposed in test logsEnsure the container image will be using permissions granted to telemetry-airflow responsibly.