Adding GitHub workflow test job

[Gordonby]

I've added a Test job to the .NET workflow as i wanted to see the tests run.
However it looks like it crashes the 😲

Determining projects to restore...
  Restored /home/runner/work/arm-estimator/arm-estimator/arm-estimator/arm-estimator.csproj (in 2.7[8](https://github.com/Gordonby/arm-estimator/actions/runs/3551022538/jobs/5964906827#step:4:9) sec).
  Restored /home/runner/work/arm-estimator/arm-estimator/arm-estimator-tests/arm-estimator-tests.csproj (in 3.03 sec).
/home/runner/work/arm-estimator/arm-estimator/arm-estimator/EstimateOptionsBinder.cs(56,[13](https://github.com/Gordonby/arm-estimator/actions/runs/3551022538/jobs/5964906827#step:4:14)): warning CS8604: Possible null reference argument for parameter 'inlineParameters' in 'EstimateOptions.EstimateOptions(DeploymentMode mode, int threshold, FileInfo? parametersFile, CurrencyCode currency, bool shouldGenerateOutput, bool shouldBeSilent, bool stdout, bool disableDetailedMetrics, string? jsonOutputFilename, bool shouldGenerateHtmlOutput, IEnumerable<string> inlineParameters)'. [/home/runner/work/arm-estimator/arm-estimator/arm-estimator/arm-estimator.csproj]
  arm-estimator -> /home/runner/work/arm-estimator/arm-estimator/arm-estimator/bin/Debug/net6.0/linux-x64/arm-estimator.dll
  arm-estimator-tests -> /home/runner/work/arm-estimator/arm-estimator/arm-estimator-tests/bin/Debug/net6.0/arm-estimator-tests.dll
Test run for /home/runner/work/arm-estimator/arm-estimator/arm-estimator-tests/bin/Debug/net6.0/arm-estimator-tests.dll (.NETCoreApp,Version=v6.0)
Microsoft (R) Test Execution Command Line Tool Version [17](https://github.com/Gordonby/arm-estimator/actions/runs/3551022538/jobs/5964906827#step:4:18).4.0 (x64)
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
The active test run was aborted. Reason: Test host process crashed

Test Run Aborted with error System.Exception: One or more errors occurred.
 ---> System.Exception: Unable to read beyond the end of the stream.
   at System.IO.BinaryReader.Read7BitEncodedInt()
   at System.IO.BinaryReader.ReadString()
   at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.LengthPrefixCommunicationChannel.NotifyDataAvailable()
   at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.TcpClientExtensions.MessageLoopAsync(TcpClient client, ICommunicationChannel channel, Action`1 errorHandler, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---.
Error: Process completed with exit code 1.

[kamil-mrzyglod]

Hmm, my first thought be, that it may be connected to microsoft/vstest#3658 as those tests run ACE as a separate process, which, in case of an error, crashes the runner. I may have not seen this before because I run those tests mostly on a local instance of VS.

I'm thinking if wrapping all the similar lines:

var exitCode = await Program.Main(new[] {
"templates/bicep/nested.bicep",
"f81e70a7-e819-49b2-a980-8e9c433743dd",
"arm-estimator-rg",
"--generateJsonOutput",
"--jsonOutputFilename",
outputFilename
});

with try/catch would help here.

[kamil-mrzyglod]

Unless you're using --threshold option somewhere, which uses Environment.Exit(1) under the hood, what could also crash the runner? :)

[kamil-mrzyglod]

Not sure if that's connected, but I run all the tests locally and realized, that some ARM Templates are not properly flagged as "Copy Always" when run:

• mysqldb.json
• azuresqlmi.json

At least one the tests caused ACE process to crash exiting with non-zero code. So that may be the problem here - that's on me, I need also to fix the dictionary storing IDs of resources cause when all the tests are run within the same AppDomain, there're some concurrency issues.

[Gordonby]

I'm finding that locally the tests weren't running consistently and often flagged as skipped in the test manager console.

I need to refamiliarise myself with vs testing setups I think 😆

[kamil-mrzyglod]

I pushed some fixes for tests to the main branch. You can try merge them into your branch and see if that's solves the issue.

[Gordonby]

Unfortunately not.

Could just be an mstest config thing.
I'll look next week :)

https://github.com/Gordonby/arm-estimator/actions/runs/3553386200/jobs/5968884186#step:4:1

[kamil-mrzyglod]

@Gordonby The crash was caused by ACE exiting with Environment.Exit(1) when What If operation returned null due to e.g. invalid Azure CLI credentials / missing role assignment. It's been fixed and merged into main branch.

[github-actions]

This PR is stale because it has been open 7 days with no activity. Remove stale label or comment or this will be closed in 5 days.