Added enhancement to slack publish with build failure cause #1819

prudhvigodithi · 2022-03-24T12:24:38Z

Signed-off-by: pgodithi pgodithi@amazon.com

Description

With this PR, the slack build failure message will be enhanced with actually error.
Example while distribution-build-opensearch with manifest 2.0.0/opensearch-2.0.0.yml

MESSAGE=2022-03-24 01:47:24 ERROR    Error building job-scheduler
 retry with: ./build.sh manifests/2.0.0/opensearch-2.0.0.yml --component job-scheduler --snapshot
2022-03-24 01:48:51 ERROR    Error building job-scheduler
 retry with: ./build.sh manifests/2.0.0/opensearch-2.0.0.yml --component job-scheduler --snapshot

Issues Resolved

#1672

Check List

Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: pgodithi <pgodithi@amazon.com>

codecov-commenter · 2022-03-24T13:57:26Z

Codecov Report

Merging #1819 (c913b22) into main (6a06d7d) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##               main    #1819   +/-   ##
=========================================
  Coverage     94.53%   94.53%           
  Complexity       19       19           
=========================================
  Files           176      177    +1     
  Lines          3622     3623    +1     
  Branches         27       27           
=========================================
+ Hits           3424     3425    +1     
  Misses          194      194           
  Partials          4        4

Impacted Files	Coverage Δ
tests/jenkins/jobs/PublishNotification_Jenkinsfile	`100.00% <ø> (ø)`
tests/jenkins/jobs/BuildFailureMessage_Jenkinsfile	`100.00% <100.00%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6a06d7d...c913b22. Read the comment docs.

Signed-off-by: pgodithi <pgodithi@amazon.com>

peterzhuamazon

@abhinavGupta16 do we want @prudhvigodithi to use this new way of testing scripts or using the old ways of lib-testers?

vars/buildFailureMessage.groovy

abhinavGupta16 · 2022-03-24T19:00:12Z

vars/buildFailureMessage.groovy

+    message=message.replaceAll(",", "\n")
+    if(message.isEmpty()){
+        message="Build failed"
+    }


Nit:

Suggested change

message=message.replaceAll(",", "\n")

if(message.isEmpty()){

message="Build failed"

}

if(message.isEmpty()){

message="Build failed"

} else {

message=message.replaceAll(",", "\n")

}

We want this to message.replaceAll(",", "\n") even-though message.isEmpty() is true as want to separate the Error messages one per line
Example slack output

MESSAGE=2022-03-24 15:30:20 ERROR Error building job-scheduler retry with: ./build.sh manifests/2.0.0/opensearch-2.0.0.yml --component job-scheduler --snapshot 2022-03-24 15:31:41 ERROR Error building job-scheduler retry with: ./build.sh manifests/2.0.0/opensearch-2.0.0.yml --component job-scheduler --snapshot

if message is an empty string, what will .replaceAll replace?

abhinavGupta16 · 2022-03-24T19:02:38Z

vars/buildFailureMessage.groovy

+    def performance_log = currentBuild.rawBuild.getLog(Integer.MAX_VALUE).join('\n')
+    performance_log.eachLine() { line ->
+        if (line.matches(".*$ERROR_STRING.*")) {
+            message+=(line+delimiter)


This message can be long if there are a log of errors. Let's make it a StringBuffer or a StringBuilder instead

Thats a good point, Added Reader reading encoded character and pushed in my latest commit.

vars/buildFailureMessage.groovy

abhinavGupta16 · 2022-03-24T19:05:58Z

vars/publishNotification.groovy

-        args.manifest,
+        "${args.icon}", 
+        "JOB_NAME=${JOB_NAME}",
+        "BUILD_NUMBER=[${BUILD_NUMBER}]",


Why use an array here? Can there be multiple build numbers in one notification?

This is part of already existing code, I have just made more readable.

Ohh.... I see. I wonder why is it an array though

zelinh · 2022-03-24T19:18:03Z

vars/buildFailureMessage.groovy

+    def message = ""
+    String delimiter = ","
+    def performance_log = currentBuild.rawBuild.getLog(Integer.MAX_VALUE).join('\n')
+    performance_log.eachLine() { line ->


I'm wonder how long this normally takes since it's iterating through the log? This could be inefficient if the log is long. Could we possibly read through the bottom of the log where the error is generally close to?

Hey @zelinh, currentBuild.rawBuild.getLog(Integer.MAX_VALUE) we can add a value in place of Integer.MAX_VALUE, but this will get only last few lines of that value, and there are high chances we miss the ERROR message in those lines, so which is why I have added all lines and go over each line in running log, also this is part of post() script, which runs as part of same pipeline, so it should go over all the currentBuild log to get that ERROR string.

Got it. Thanks!

Is it possible to read the file line by line and process it, keeping in mind, string creation is an expensive operation? This goes through the entire file, reads and creates a string out of it with line breaks. It then goes over it again to filter error messages to a new string. WDTY?

yes right now this code reads line by line, and filters for the log error message, I'm looking for a buffered stream here, as explained above this is part of Same pipeline as post() call, so we need to read all lines each line as part of same pipeline.

Yes, but it currently goes through all the lines twice and makes 2 strings. Can we change it to only once?

@abhinavGupta16 check my latest commit, this time I have added Reader buffer.

Signed-off-by: pgodithi <pgodithi@amazon.com>

abhinavGupta16 · 2022-03-24T22:55:12Z

@abhinavGupta16 do we want @prudhvigodithi to use this new way of testing scripts or using the old ways of lib-testers?

We'll need libtesters

Signed-off-by: pgodithi <pgodithi@amazon.com>

abhinavGupta16 · 2022-03-25T21:33:29Z

vars/buildFailureMessage.groovy

+    Reader performance_log = currentBuild.getRawBuild().getLogReader()
+    String logContent = IOUtils.toString(performance_log)
+    performance_log.close()
+    performance_log = null


Nit: Why do we reassign it to null?

Stream object will be eligible for garbage collection

Signed-off-by: pgodithi <pgodithi@amazon.com>

gaiksaya

Looks good! Thanks!

Added enhancement to slack publish with build failure cause

c47cb5e

Signed-off-by: pgodithi <pgodithi@amazon.com>

prudhvigodithi requested a review from a team as a code owner March 24, 2022 12:24

Added Unit tests

ce2e3bd

Signed-off-by: pgodithi <pgodithi@amazon.com>

prudhvigodithi added 3 commits March 24, 2022 07:23

Added Unit tests

b546c86

Signed-off-by: pgodithi <pgodithi@amazon.com>

Added Unit tests

d093f9f

Signed-off-by: pgodithi <pgodithi@amazon.com>

Added Unit tests

070ab1c

Signed-off-by: pgodithi <pgodithi@amazon.com>

prudhvigodithi marked this pull request as draft March 24, 2022 15:45

prudhvigodithi requested review from peterzhuamazon and removed request for peterzhuamazon March 24, 2022 15:45

peterzhuamazon reviewed Mar 24, 2022

View reviewed changes

vars/buildFailureMessage.groovy Outdated Show resolved Hide resolved

abhinavGupta16 reviewed Mar 24, 2022

View reviewed changes

vars/buildFailureMessage.groovy Outdated Show resolved Hide resolved

abhinavGupta16 reviewed Mar 24, 2022

View reviewed changes

zelinh reviewed Mar 24, 2022

View reviewed changes

prudhvigodithi added 8 commits March 24, 2022 12:54

Added data types

053b3fa

Signed-off-by: pgodithi <pgodithi@amazon.com>

Added data types

5fdde69

Signed-off-by: pgodithi <pgodithi@amazon.com>

Added data types

617c2f4

Signed-off-by: pgodithi <pgodithi@amazon.com>

Added data types

ec4dfb4

Signed-off-by: pgodithi <pgodithi@amazon.com>

Added data types

6f2c75a

Signed-off-by: pgodithi <pgodithi@amazon.com>

Added data types

5838956

Signed-off-by: pgodithi <pgodithi@amazon.com>

Added data types

ff6ed4a

Signed-off-by: pgodithi <pgodithi@amazon.com>

Added data types

75946d4

Signed-off-by: pgodithi <pgodithi@amazon.com>

prudhvigodithi added 4 commits March 25, 2022 08:30

Buffer Reader to store the jenkins log and read from this stream

b2dd3c6

Signed-off-by: pgodithi <pgodithi@amazon.com>

Removed delimiter

709f4bd

Signed-off-by: pgodithi <pgodithi@amazon.com>

Removed delimiter

1d66804

Signed-off-by: pgodithi <pgodithi@amazon.com>

Added test files

381c21c

Signed-off-by: pgodithi <pgodithi@amazon.com>

abhinavGupta16 reviewed Mar 25, 2022

View reviewed changes

Added jenkins BuildFailureMessage shared lib test cases

1d20384

Signed-off-by: pgodithi <pgodithi@amazon.com>

prudhvigodithi added 8 commits March 27, 2022 18:54

Merge remote-tracking branch 'upstream/main' into slack-publish-v3

c46a002

Fixing all failed test cases

257c937

Signed-off-by: pgodithi <pgodithi@amazon.com>

Fixing all failed test cases: testing

a0aeddc

Signed-off-by: pgodithi <pgodithi@amazon.com>

Fixing all failed test cases: testing

7ed98b4

Signed-off-by: pgodithi <pgodithi@amazon.com>

Fixing double-quotes bug in buildFailureMessage.groovy file

6f222b7

Signed-off-by: pgodithi <pgodithi@amazon.com>

Fixing double-quotes bug in buildFailureMessage.groovy file

129277f

Signed-off-by: pgodithi <pgodithi@amazon.com>

Added buildFailureMessage() pipelines

7c34ea2

Signed-off-by: pgodithi <pgodithi@amazon.com>

Added buildFailureMessage() pipelines

e9bfb12

Signed-off-by: pgodithi <pgodithi@amazon.com>

prudhvigodithi marked this pull request as ready for review March 29, 2022 00:40

removing buildFailureMessage() for dashboard jenkinsfile

c913b22

Signed-off-by: pgodithi <pgodithi@amazon.com>

gaiksaya approved these changes Mar 29, 2022

View reviewed changes

prudhvigodithi merged commit aa6b829 into opensearch-project:main Mar 29, 2022

prudhvigodithi mentioned this pull request Mar 29, 2022

[Enhancement] Make build notification in chat show failed build command #1672

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added enhancement to slack publish with build failure cause #1819

Added enhancement to slack publish with build failure cause #1819

prudhvigodithi commented Mar 24, 2022 •

edited

Loading

codecov-commenter commented Mar 24, 2022 •

edited

Loading

peterzhuamazon left a comment

abhinavGupta16 Mar 24, 2022 •

edited

Loading

prudhvigodithi Mar 24, 2022

abhinavGupta16 Mar 24, 2022

abhinavGupta16 Mar 24, 2022

prudhvigodithi Mar 25, 2022 •

edited

Loading

abhinavGupta16 Mar 24, 2022

prudhvigodithi Mar 25, 2022

abhinavGupta16 Mar 25, 2022

zelinh Mar 24, 2022

prudhvigodithi Mar 24, 2022

zelinh Mar 24, 2022

abhinavGupta16 Mar 24, 2022

prudhvigodithi Mar 25, 2022

abhinavGupta16 Mar 25, 2022

prudhvigodithi Mar 25, 2022 •

edited

Loading

abhinavGupta16 commented Mar 24, 2022 •

edited

Loading

abhinavGupta16 Mar 25, 2022

prudhvigodithi Mar 25, 2022

gaiksaya left a comment

Added enhancement to slack publish with build failure cause #1819

Added enhancement to slack publish with build failure cause #1819

Conversation

prudhvigodithi commented Mar 24, 2022 • edited Loading

Description

Issues Resolved

Check List

codecov-commenter commented Mar 24, 2022 • edited Loading

Codecov Report

peterzhuamazon left a comment

Choose a reason for hiding this comment

abhinavGupta16 Mar 24, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

prudhvigodithi Mar 25, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

prudhvigodithi Mar 25, 2022 • edited Loading

Choose a reason for hiding this comment

abhinavGupta16 commented Mar 24, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gaiksaya left a comment

Choose a reason for hiding this comment

prudhvigodithi commented Mar 24, 2022 •

edited

Loading

codecov-commenter commented Mar 24, 2022 •

edited

Loading

abhinavGupta16 Mar 24, 2022 •

edited

Loading

prudhvigodithi Mar 25, 2022 •

edited

Loading

prudhvigodithi Mar 25, 2022 •

edited

Loading

abhinavGupta16 commented Mar 24, 2022 •

edited

Loading