Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Filter attributes neuron metrics #1262

Merged
merged 13 commits into from
Aug 7, 2024

Conversation

aditya-purang
Copy link
Contributor

@aditya-purang aditya-purang commented Jul 26, 2024

Description of the issue

We saw that the EMF logs for neuron were having too many bytes causing an inflation in bills. This was caught by our monitoring and while investigating we saw some attributes which were redundant and could've been filtered out making the logs a bit lighter.

Description of changes

This PR has 2 major changes for the GPU processor:

  1. We drop all the attributes from resource metrics object if they are coming from the DCGM or the neuron monitor scraper.
  2. The attribute filtering logic has been moved from the neuron metric modifier to the gpu processor. It has now been combined with GPU metrics and the filters have been moved to a different files making the code a bit more readable.

Tests

Tested in eks playground cluster

Requirements

  1. Run make fmt and make fmt-sh : done
  2. Run make lint : done

@aditya-purang aditya-purang requested review from movence and sam6134 July 26, 2024 11:50
@codecov-commenter
Copy link

Codecov Report

Attention: Patch coverage is 72.72727% with 15 lines in your changes missing coverage. Please review.

Project coverage is 64.41%. Comparing base (96d4763) to head (63be809).
Report is 587 commits behind head on main.

Files Patch % Lines
plugins/processors/gpuattributes/processor.go 59.45% 13 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1262      +/-   ##
==========================================
+ Coverage   57.58%   64.41%   +6.82%     
==========================================
  Files         370      469      +99     
  Lines       17548    23679    +6131     
==========================================
+ Hits        10105    15252    +5147     
- Misses       6848     7677     +829     
- Partials      595      750     +155     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@aditya-purang aditya-purang marked this pull request as ready for review July 26, 2024 13:23
@aditya-purang aditya-purang requested a review from a team as a code owner July 26, 2024 13:23
sam6134
sam6134 previously approved these changes Jul 30, 2024
@sam6134 sam6134 mentioned this pull request Aug 7, 2024
@aditya-purang aditya-purang merged commit 55a3d88 into aws:main Aug 7, 2024
6 checks passed
mitali-salvi added a commit that referenced this pull request Aug 8, 2024
mitali-salvi added a commit to mitali-salvi/amazon-cloudwatch-agent that referenced this pull request Dec 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants