Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move JobTelemetry and StepsTelemetry into GlobalContext. #1680

Merged
merged 3 commits into from
Feb 11, 2022
Merged

Move JobTelemetry and StepsTelemetry into GlobalContext. #1680

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
4bc6d98
Move JobTelemetry and StepsTelemetry into GlobalContext.
TingluoHuang Feb 11, 2022
e97a766
.
TingluoHuang Feb 11, 2022
071bb6e
.
TingluoHuang Feb 11, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/Runner.Worker/ActionCommandManager.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -178,7 +178,7 @@ private void ValidateStopToken(IExecutionContext context, string stopToken)
Message = $"Invoked ::stopCommand:: with token: [{stopToken}]",
Type = JobTelemetryType.ActionCommand
};
context.JobTelemetry.Add(telemetry);
context.Global.JobTelemetry.Add(telemetry);
}

if (isTokenInvalid && !allowUnsecureStopCommandTokens)
Expand Down
68 changes: 54 additions & 14 deletions src/Runner.Worker/ExecutionContext.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@
using GitHub.DistributedTask.Pipelines.ContextData;
using GitHub.DistributedTask.Pipelines.ObjectTemplating;
using GitHub.DistributedTask.WebApi;
using GitHub.Runner.Common.Util;
using GitHub.Runner.Common;
using GitHub.Runner.Common.Util;
using GitHub.Runner.Sdk;
using GitHub.Runner.Worker.Container;
using GitHub.Services.WebApi;
Expand Down Expand Up @@ -52,8 +52,7 @@ public interface IExecutionContext : IRunnerService
Dictionary<string, string> IntraActionState { get; }
Dictionary<string, VariableValue> JobOutputs { get; }
ActionsEnvironmentReference ActionsEnvironment { get; }
List<ActionsStepTelemetry> ActionsStepsTelemetry { get; }
List<JobTelemetry> JobTelemetry { get; }
ActionsStepTelemetry StepTelemetry { get; }
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Each ExecutionContext will hold a single ActionsStepTelemetry object, the object will get append to the GlobalContext.ActionsStepsTelemetry when ExecutionContext.Complete() is called.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For composite child steps we don't call complete() so I don't think you will be getting that data currently

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for pointing this out, i will take a look.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like we only call .Complete() in one place under composite action.
https://github.com/actions/runner/blob/main/src/Runner.Worker/Handlers/CompositeActionHandler.cs#L296

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am adding a new PublishStepTelemetry() to executionContext, so composite actions can use that to publish step telemetry.

DictionaryContextData ExpressionValues { get; }
IList<IFunctionInfo> ExpressionFunctions { get; }
JobContext JobContext { get; }
Expand Down Expand Up @@ -109,6 +108,7 @@ public interface IExecutionContext : IRunnerService
// others
void ForceTaskComplete();
void RegisterPostJobStep(IStep step);
void PublishStepTelemetry();
}

public sealed class ExecutionContext : RunnerService, IExecutionContext
Expand Down Expand Up @@ -139,6 +139,7 @@ public sealed class ExecutionContext : RunnerService, IExecutionContext

// only job level ExecutionContext will track throttling delay.
private long _totalThrottlingDelayInMilliseconds = 0;
private bool _stepTelemetryPublished = false;

public Guid Id => _record.Id;
public Guid EmbeddedId { get; private set; }
Expand All @@ -152,8 +153,7 @@ public sealed class ExecutionContext : RunnerService, IExecutionContext
public Dictionary<string, VariableValue> JobOutputs { get; private set; }

public ActionsEnvironmentReference ActionsEnvironment { get; private set; }
public List<ActionsStepTelemetry> ActionsStepsTelemetry { get; private set; }
public List<JobTelemetry> JobTelemetry { get; private set; }
public ActionsStepTelemetry StepTelemetry { get; } = new ActionsStepTelemetry();
public DictionaryContextData ExpressionValues { get; } = new DictionaryContextData();
public IList<IFunctionInfo> ExpressionFunctions { get; } = new List<IFunctionInfo>();

Expand Down Expand Up @@ -273,9 +273,9 @@ public void RegisterPostJobStep(IStep step)
{
Trace.Info($"'post' of '{actionRunner.DisplayName}' already push to child post step stack.");
}
else
else
{
Root.EmbeddedStepsWithPostRegistered[actionRunner.Action.Id] = actionRunner.Condition;
Root.EmbeddedStepsWithPostRegistered[actionRunner.Action.Id] = actionRunner.Condition;
}
return;
}
Expand All @@ -294,7 +294,20 @@ public void RegisterPostJobStep(IStep step)
Root.PostJobSteps.Push(step);
}

public IExecutionContext CreateChild(Guid recordId, string displayName, string refName, string scopeName, string contextName, ActionRunStage stage, Dictionary<string, string> intraActionState = null, int? recordOrder = null, IPagingLogger logger = null, bool isEmbedded = false, CancellationTokenSource cancellationTokenSource = null, Guid embeddedId = default(Guid), string siblingScopeName = null)
public IExecutionContext CreateChild(
Guid recordId,
string displayName,
string refName,
string scopeName,
string contextName,
ActionRunStage stage,
Dictionary<string, string> intraActionState = null,
int? recordOrder = null,
IPagingLogger logger = null,
bool isEmbedded = false,
CancellationTokenSource cancellationTokenSource = null,
Guid embeddedId = default(Guid),
string siblingScopeName = null)
{
Trace.Entering();

Expand All @@ -306,7 +319,6 @@ public void RegisterPostJobStep(IStep step)
child.Stage = stage;
child.EmbeddedId = embeddedId;
child.SiblingScopeName = siblingScopeName;
child.JobTelemetry = JobTelemetry;
if (intraActionState == null)
{
child.IntraActionState = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
Expand Down Expand Up @@ -354,7 +366,13 @@ public void RegisterPostJobStep(IStep step)
/// An embedded execution context shares the same record ID, record name, logger,
/// and a linked cancellation token.
/// </summary>
public IExecutionContext CreateEmbeddedChild(string scopeName, string contextName, Guid embeddedId, ActionRunStage stage, Dictionary<string, string> intraActionState = null, string siblingScopeName = null)
public IExecutionContext CreateEmbeddedChild(
string scopeName,
string contextName,
Guid embeddedId,
ActionRunStage stage,
Dictionary<string, string> intraActionState = null,
string siblingScopeName = null)
{
return Root.CreateChild(_record.Id, _record.Name, _record.Id.ToString("N"), scopeName, contextName, stage, logger: _logger, isEmbedded: true, cancellationTokenSource: CancellationTokenSource.CreateLinkedTokenSource(_cancellationTokenSource.Token), intraActionState: intraActionState, embeddedId: embeddedId, siblingScopeName: siblingScopeName);
}
Expand Down Expand Up @@ -404,6 +422,8 @@ public TaskResult Complete(TaskResult? result = null, string currentOperation =
}
}

PublishStepTelemetry();

if (Root != this)
{
// only dispose TokenSource for step level ExecutionContext
Expand Down Expand Up @@ -654,16 +674,18 @@ public void InitializeJob(Pipelines.AgentJobRequestMessage message, Cancellation
// Job defaults shared across all actions
Global.JobDefaults = new Dictionary<string, IDictionary<string, string>>(StringComparer.OrdinalIgnoreCase);

// Job Telemetry
Global.JobTelemetry = new List<JobTelemetry>();

// ActionsStepTelemetry for entire job
Global.StepsTelemetry = new List<ActionsStepTelemetry>();

// Job Outputs
JobOutputs = new Dictionary<string, VariableValue>(StringComparer.OrdinalIgnoreCase);

// Actions environment
ActionsEnvironment = message.ActionsEnvironment;

// ActionsStepTelemetry
ActionsStepsTelemetry = new List<ActionsStepTelemetry>();

JobTelemetry = new List<JobTelemetry>();

// Service container info
Global.ServiceContainers = new List<ContainerInfo>();
Expand Down Expand Up @@ -895,6 +917,24 @@ public IEnumerable<IssueMatcherConfig> GetMatchers()
return Root._matchers ?? Array.Empty<IssueMatcherConfig>();
}

public void PublishStepTelemetry()
{
if (!_stepTelemetryPublished)
{
// Add to the global steps telemetry only if we have something to log.
if (!string.IsNullOrEmpty(StepTelemetry?.Type))
{
Trace.Info($"Publish step telemetry for current step {StringUtil.ConvertToJson(StepTelemetry)}.");
Global.StepsTelemetry.Add(StepTelemetry);
_stepTelemetryPublished = true;
}
}
else
{
Trace.Info($"Step telemetry has already been published.");
}
}

private void InitializeTimelineRecord(Guid timelineId, Guid timelineRecordId, Guid? parentTimelineRecordId, string recordType, string displayName, string refName, int? order)
{
_mainTimelineId = timelineId;
Expand Down
2 changes: 2 additions & 0 deletions src/Runner.Worker/GlobalContext.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@ public sealed class GlobalContext
public PlanFeatures Features { get; set; }
public IList<String> FileTable { get; set; }
public IDictionary<String, IDictionary<String, String>> JobDefaults { get; set; }
public List<ActionsStepTelemetry> StepsTelemetry { get; set; }
public List<JobTelemetry> JobTelemetry { get; set; }
public TaskOrchestrationPlanReference Plan { get; set; }
public List<string> PrependPath { get; set; }
public List<ContainerInfo> ServiceContainers { get; set; }
Expand Down
36 changes: 17 additions & 19 deletions src/Runner.Worker/Handlers/CompositeActionHandler.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ public async Task RunAsync(ActionRunStage stage)
{
ArgUtil.NotNull(Data.PreSteps, nameof(Data.PreSteps));
steps = Data.PreSteps;
}
}
else if (stage == ActionRunStage.Post)
{
ArgUtil.NotNull(Data.PostSteps, nameof(Data.PostSteps));
Expand All @@ -60,7 +60,7 @@ public async Task RunAsync(ActionRunStage stage)
Trace.Info($"Skipping executing post step id: {step.Id}, name: ${step.DisplayName}");
}
}
}
}
else
{
ArgUtil.NotNull(Data.Steps, nameof(Data.Steps));
Expand All @@ -83,20 +83,17 @@ public async Task RunAsync(ActionRunStage stage)
hasUsesStep = true;
}
}
var pathReference = Action as Pipelines.RepositoryPathReference;
var telemetry = new ActionsStepTelemetry {
Ref = GetActionRef(),
HasPreStep = Data.HasPre,
HasPostStep = Data.HasPost,
IsEmbedded = ExecutionContext.IsEmbedded,
Type = "composite",
HasRunsStep = hasRunsStep,
HasUsesStep = hasUsesStep,
StepCount = steps.Count
};
ExecutionContext.Root.ActionsStepsTelemetry.Add(telemetry);

ExecutionContext.StepTelemetry.Ref = GetActionRef();
ExecutionContext.StepTelemetry.HasPreStep = Data.HasPre;
ExecutionContext.StepTelemetry.HasPostStep = Data.HasPost;
ExecutionContext.StepTelemetry.IsEmbedded = ExecutionContext.IsEmbedded;
ExecutionContext.StepTelemetry.Type = "composite";
ExecutionContext.StepTelemetry.HasRunsStep = hasRunsStep;
ExecutionContext.StepTelemetry.HasUsesStep = hasUsesStep;
ExecutionContext.StepTelemetry.StepCount = steps.Count;
}

try
{
// Inputs of the composite step
Expand All @@ -117,7 +114,7 @@ public async Task RunAsync(ActionRunStage stage)
// Create embedded steps
var embeddedSteps = new List<IStep>();

// If we need to setup containers beforehand, do it
// If we need to setup containers beforehand, do it
// only relevant for local composite actions that need to JIT download/setup containers
if (LocalActionContainerSetupSteps != null && LocalActionContainerSetupSteps.Count > 0)
{
Expand Down Expand Up @@ -152,7 +149,7 @@ public async Task RunAsync(ActionRunStage stage)
}
else
{
step.ExecutionContext.ExpressionValues["steps"] = ExecutionContext.Global.StepsContext.GetScope(childScopeName);
step.ExecutionContext.ExpressionValues["steps"] = ExecutionContext.Global.StepsContext.GetScope(childScopeName);
}

// Shallow copy github context
Expand Down Expand Up @@ -309,7 +306,7 @@ private async Task RunStepsAsync(List<IStep> embeddedSteps, ActionRunStage stage
// Mark job as cancelled
ExecutionContext.Root.Result = TaskResult.Canceled;
ExecutionContext.Root.JobContext.Status = ExecutionContext.Root.Result?.ToActionResult();

step.ExecutionContext.Debug($"Re-evaluate condition on job cancellation for step: '{step.DisplayName}'.");
var conditionReTestTraceWriter = new ConditionTraceWriter(Trace, null); // host tracing only
var conditionReTestResult = false;
Expand Down Expand Up @@ -393,7 +390,7 @@ private async Task RunStepsAsync(List<IStep> embeddedSteps, ActionRunStage stage
{
await RunStepAsync(step);
}

}
finally
{
Expand Down Expand Up @@ -458,6 +455,7 @@ private async Task RunStepAsync(IStep step)

Trace.Info($"Step result: {step.ExecutionContext.Result}");
step.ExecutionContext.Debug($"Finished: {step.DisplayName}");
step.ExecutionContext.PublishStepTelemetry();
}

private void SetStepConclusion(IStep step, TaskResult result)
Expand Down
27 changes: 12 additions & 15 deletions src/Runner.Worker/Handlers/ContainerActionHandler.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
using System.Collections.Generic;
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Threading.Tasks;
using System;
using GitHub.Runner.Worker.Container;
using Pipelines = GitHub.DistributedTask.Pipelines;
using GitHub.DistributedTask.Pipelines.ContextData;
using GitHub.DistributedTask.WebApi;
using GitHub.Runner.Common;
using GitHub.Runner.Sdk;
using GitHub.DistributedTask.WebApi;
using GitHub.DistributedTask.Pipelines.ContextData;
using System.Linq;
using GitHub.Runner.Worker.Container;
using Pipelines = GitHub.DistributedTask.Pipelines;

namespace GitHub.Runner.Worker.Handlers
{
Expand Down Expand Up @@ -73,14 +73,11 @@ public async Task RunAsync(ActionRunStage stage)
// Add Telemetry to JobContext to send with JobCompleteMessage
if (stage == ActionRunStage.Main)
{
var telemetry = new ActionsStepTelemetry {
Ref = GetActionRef(),
HasPreStep = Data.HasPre,
HasPostStep = Data.HasPost,
IsEmbedded = ExecutionContext.IsEmbedded,
Type = type
};
ExecutionContext.Root.ActionsStepsTelemetry.Add(telemetry);
ExecutionContext.StepTelemetry.Ref = GetActionRef();
ExecutionContext.StepTelemetry.HasPreStep = Data.HasPre;
ExecutionContext.StepTelemetry.HasPostStep = Data.HasPost;
ExecutionContext.StepTelemetry.IsEmbedded = ExecutionContext.IsEmbedded;
ExecutionContext.StepTelemetry.Type = type;
}

// run container
Expand Down
Loading