CI pipeline for installer #14

jkoritzinsky · 2019-11-05T00:27:38Z

These were the changes I needed to get the CI pipeline building for the installer partition of the repository.

update install-nuget-credprovider-then-build.sh to understand subsets (not strictly necessary, but it made things a bit cleaner in the initial prototyping day).
Fix paths to the various YAML files included by the pipeline.
Update the path mounted in the docker container to be root/runtime instead of root/coresetup.
Fix paths for the manpages for the packaging steps.
Disable signing during the installer build.
Fix path to the cross directory
Fix placeholder detection in tools-local to work correctly within the docker container.
Fix parameters to installer.cmd and installer.sh to correctly pass --build and --restore on all platforms to restore the tools needed.

* Packaging for crossgen2 as a framework dependent application. This requires us to manually emit a runtimeconfig json for now, since the build produces a self-contained version of crossgen2. Also includes some minor cleanups to the way we build crossgen2: - Removed from build.proj since we explicitly build it as a self-contained application from build.cmd/build.sh - Removed the output path property from the csproj files. We use an explicit -o argument when publishing to specify the output folder

…025.3 (#8671) - System.Windows.Extensions - 5.0.0-alpha.1.19525.3 - System.CodeDom - 5.0.0-alpha.1.19525.3 - System.Configuration.ConfigurationManager - 5.0.0-alpha.1.19525.3 - Microsoft.NETCore.Platforms - 5.0.0-alpha.1.19525.3 - Microsoft.NETCore.Targets - 5.0.0-alpha.1.19525.3 - Microsoft.Private.CoreFx.NETCoreApp - 5.0.0-alpha.1.19525.3 - Microsoft.Win32.Registry - 5.0.0-alpha.1.19525.3 - Microsoft.Win32.SystemEvents - 5.0.0-alpha.1.19525.3 - Microsoft.Windows.Compatibility - 5.0.0-alpha.1.19525.3 - System.Diagnostics.EventLog - 5.0.0-alpha.1.19525.3 - System.Security.AccessControl - 5.0.0-alpha.1.19525.3 - System.Security.Cryptography.Cng - 5.0.0-alpha.1.19525.3 - System.Security.Cryptography.Pkcs - 5.0.0-alpha.1.19525.3 - System.Security.Cryptography.ProtectedData - 5.0.0-alpha.1.19525.3 - System.Security.Cryptography.Xml - 5.0.0-alpha.1.19525.3 - System.Security.Permissions - 5.0.0-alpha.1.19525.3 - System.Security.Principal.Windows - 5.0.0-alpha.1.19525.3 - System.Text.Encodings.Web - 5.0.0-alpha.1.19525.3 - System.Text.Json - 5.0.0-alpha.1.19525.3 - System.Threading.AccessControl - 5.0.0-alpha.1.19525.3 - System.Resources.Extensions - 5.0.0-alpha.1.19525.3 - System.DirectoryServices - 5.0.0-alpha.1.19525.3 - System.Drawing.Common - 5.0.0-alpha.1.19525.3 - System.IO.FileSystem.AccessControl - 5.0.0-alpha.1.19525.3 - System.IO.Packaging - 5.0.0-alpha.1.19525.3

This change modifies R2RDump to use the new System.CommandLine.Experimental module instead of the obsoleted System.CommandLine parser we were using before. Apart from general hygiene w.r.t. using up-to-date components one practical reason for this switch-over is that the old parser is only available on the unreliable myget.org nuget feed we plan to drop as part of our planned repo consolidation. During debugging of the change I noticed a bug in the implementation of the "--verbose" option - it shouldn't be setting EntryPoints, otherwise we actually end up with a super-terse output including just the entry points. I have anecdotally verified that, after fixing this in both the "new" and "old" R2RDump both produce identical output. Thanks Tomas

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

Improved performance of complex casts by caching results of cast analysis. * Easy out for same types. * Introduce cast cache * Use managed heap for the cache * Fixes. Get rid of a fake array typedesc. * TypeDesc conversions caching * Removed NoGC helpers * Cache typedesc conversions * Renamed remaining trivial NoGC casting helpers * Some cleanups. Comments, redundant code. * PR feedback.

* Update dependencies from https://github.com/dotnet/coreclr build 20191025.1 - Microsoft.NET.Sdk.IL - 5.0.0-alpha1.19525.1 - Microsoft.NETCore.ILAsm - 5.0.0-alpha1.19525.1 - Microsoft.NETCore.Runtime.CoreCLR - 5.0.0-alpha1.19525.1 * remove Stopwatch.*.cs from System.Runtime.Extensions build * Disable tests against CoreCLR bug

…aces

…aces Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

…025.2 (#8673) - Microsoft.DotNet.XUnitExtensions - 5.0.0-beta.19525.2 - Microsoft.DotNet.VersionTools.Tasks - 5.0.0-beta.19525.2 - Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19525.2 - Microsoft.DotNet.Build.Tasks.Feed - 5.0.0-beta.19525.2 - Microsoft.DotNet.Build.Tasks.Packaging - 5.0.0-beta.19525.2 - Microsoft.DotNet.Build.Tasks.SharedFramework.Sdk - 5.0.0-beta.19525.2

…025.2 (#42137) - Microsoft.DotNet.XUnitExtensions - 5.0.0-beta.19525.2 - Microsoft.DotNet.XUnitConsoleRunner - 2.5.1-beta.19525.2 - Microsoft.DotNet.VersionTools.Tasks - 5.0.0-beta.19525.2 - Microsoft.DotNet.ApiCompat - 5.0.0-beta.19525.2 - Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19525.2 - Microsoft.DotNet.Build.Tasks.Configuration - 5.0.0-beta.19525.2 - Microsoft.DotNet.Build.Tasks.Feed - 5.0.0-beta.19525.2 - Microsoft.DotNet.Build.Tasks.Packaging - 5.0.0-beta.19525.2 - Microsoft.DotNet.CodeAnalysis - 5.0.0-beta.19525.2 - Microsoft.DotNet.CoreFxTesting - 5.0.0-beta.19525.2 - Microsoft.DotNet.GenAPI - 5.0.0-beta.19525.2 - Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19525.2 - Microsoft.DotNet.RemoteExecutor - 5.0.0-beta.19525.2 - Microsoft.DotNet.GenFacades - 5.0.0-beta.19525.2

…91026.1 (#8674) - NETStandard.Library - 2.2.0-prerelease.19526.1

* Enable building with clang 9 Update the docker images that we use for building. * Add missing clang9 option to build.sh / build-test.sh * Fix clang version matching and MUSL build * Prevent Clang 9 from optimizing out s_gsCookie write

* Can't declare variable and also have extern c * Remove extern c from variable declartion

) Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

The file was there to keep anonymous types used by DiagnosticsHandler from having their getters removed. But a) it was relying on the implementation detail name generated by Roslyn, and b) it was keeping alive other state from other anonymous types in the assembly. I changed it to use concrete types instead. This also happens to result in less code, because we don't need the generated Equals/GetHashCode methods generated for anonymous types.

It's not necessary to specify the assembly name when the consumer is in the same assembly.

* Update dependencies from https://github.com/dotnet/corefx build 20191024.13 - Microsoft.NETCore.Platforms - 5.0.0-alpha.1.19524.13 - Microsoft.Private.CoreFx.NETCoreApp - 5.0.0-alpha.1.19524.13 * Update dependencies from https://github.com/dotnet/corefx build 20191024.14 - Microsoft.NETCore.Platforms - 5.0.0-alpha.1.19524.14 - Microsoft.Private.CoreFx.NETCoreApp - 5.0.0-alpha.1.19524.14 * Update dependencies from https://github.com/dotnet/corefx build 20191025.6 - Microsoft.NETCore.Platforms - 5.0.0-alpha.1.19525.6 - Microsoft.Private.CoreFx.NETCoreApp - 5.0.0-alpha.1.19525.6 * Workaround https://github.com/dotnet/coreclr/issues/27465

- Use PreserveDependencyAttribute for ctors used via reflection by FetcherForProperty - Remove reflection use entirely when looking up Activity events

… changes to coreclr/libraries-specific pipelines.

eng/install-nuget-credprovider-then-build.sh

eng/pipelines/installer/azure-pipelines.yml

src/installer/signing/Directory.Build.props

ViktorHofer · 2019-11-05T07:50:27Z

tools-local/Directory.Build.props

  <!-- TODO: (Consolidation) Remove when consolidated. -->
  <PropertyGroup>
-    <IsRuntimeRepository Condition="Exists('..\..\..\.dotnet-runtime-placeholder')">true</IsRuntimeRepository>
+    <IsRuntimeRepository Condition="Exists('$(MSBuildThisFileDirectory)..\.dotnet-runtime-placeholder')">true</IsRuntimeRepository>


this is already fixed in upstream :)

ViktorHofer · 2019-11-05T08:00:42Z

eng/pipelines/installer/jobs/bash-build.yml

@@ -118,8 +118,8 @@ jobs:
    - script: |
        set -x
        df -h
-        $(RunArguments) $(BuildScript) $(BuildArguments)
-      displayName: Build
+        $(RunArguments) $(BuildScript) --restore --build --test $(BuildArguments)


please revert these changes, I submitted the correct fix in the consolidation repo: https://github.com/dotnet/consolidation/pull/16.

I'll make sure not to backport these specific changes when backporting to core-setup.

ViktorHofer · 2019-11-05T08:05:13Z

Thanks @jkoritzinsky! What's your strategy for backporting the changes to upstream? @safern is currently doing the same for corefx so you might want to sync-up with him.

dagood · 2019-11-05T15:09:59Z

src/installer/pkg/packaging/deb/package.targets

@@ -32,7 +32,7 @@
      <DebPackageVersion>$(HostPackageVersion)</DebPackageVersion>
      <InputRoot>$(SharedHostPublishRoot)</InputRoot>
      <DebFile>$(SharedHostInstallerFile)</DebFile>
-      <ManPagesDir>$(RepoRoot)Documentation/manpages</ManPagesDir>
+      <ManPagesDir>$(RepoRoot)docs/installer/manpages</ManPagesDir>


Also fixed in upstream with dotnet/core-setup#8767 and https://github.com/dotnet/consolidation/pull/17, to point it out. This change is good to get builds going though.

…ller

jkoritzinsky · 2019-11-05T22:50:40Z

/azp run runtime-installer

azure-pipelines · 2019-11-05T22:50:46Z

Azure Pipelines could not run because the pipeline triggers exclude this branch/path.

Fadi Hanna and others added 30 commits October 25, 2019 14:58

[crossgen2] Avoid ambiguous devirtualization (#27424)

d374f8c

Profiler API to modify environment variables (#23383) (#27157)

457449f

Fix build break

5861d42

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

Remove class declarator (#27456)

6fe1637

Use ValueStringBuilder instead of StringBuidler{Cache} in multiple pl…

2969452

…aces

Avoid unnecessary allocations in Exception.ToString

373a7b2

Cleanup SELECTANY (#27459)

7e11363

Remove extra qualifier (#27455)

a2f10df

Use ValueStringBuilder instead of StringBuidler{Cache} in multiple pl…

ea46cd6

…aces Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

Avoid unnecessary allocations in Exception.ToString

b123ba4

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

Update dependencies from https://github.com/dotnet/standard build 201…

73e44b8

…91026.1 (#8674) - NETStandard.Library - 2.2.0-prerelease.19526.1

Avoid mod operator when fast alternative available (#27299)

e532bf6

Removing unneeded function calls and empty bodies (#27420)

9a60a6e

Removed unused strings from resources (#27304)

6f2791f

Delete a few more instances of StringBuilderCache (#42139)

bc78f93

Enable building with clang 9 (#27415)

33905fb

* Enable building with clang 9 Update the docker images that we use for building. * Add missing clang9 option to build.sh / build-test.sh * Fix clang version matching and MUSL build * Prevent Clang 9 from optimizing out s_gsCookie write

Can't declare variable and also have extern c (#27457)

9b832e6

* Can't declare variable and also have extern c * Remove extern c from variable declartion

Avoid mod operator when fast alternative available (dotnet/coreclr#27299

36d9ba1

) Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

Fix build break

3e6e68f

Clean up use of PreserveDependencyAttribute in System.Text.Json (#42146)

e13b93b

It's not necessary to specify the assembly name when the consumer is in the same assembly.

Remove ILLinkTrim.xml from System.Diagnostics.DiagnosticSource (#42124)

a5d7583

- Use PreserveDependencyAttribute for ctors used via reflection by FetcherForProperty - Remove reflection use entirely when looking up Activity events

jkoritzinsky added 3 commits November 4, 2019 15:22

Fix typo.

b9fa831

Fix disabling signing condition.

c1eb006

Add back original triggers in prep for PR. Extend excludes to exclude…

c8ca7f2

… changes to coreclr/libraries-specific pipelines.

jkoritzinsky requested review from ViktorHofer and dagood November 5, 2019 00:27

ViktorHofer reviewed Nov 5, 2019

View reviewed changes

eng/install-nuget-credprovider-then-build.sh Outdated Show resolved Hide resolved

ViktorHofer reviewed Nov 5, 2019

View reviewed changes

eng/pipelines/installer/azure-pipelines.yml Outdated Show resolved Hide resolved

ViktorHofer reviewed Nov 5, 2019

View reviewed changes

src/installer/signing/Directory.Build.props Outdated Show resolved Hide resolved

ViktorHofer reviewed Nov 5, 2019

View reviewed changes

dagood reviewed Nov 5, 2019

View reviewed changes

jkoritzinsky added 2 commits November 5, 2019 14:43

Merge branch 'master' of https://github.com/dotnet/runtime into insta…

af0d26c

…ller

PR Feedback. Run build on changes to manpages.

948e89a

Include all files except those excluded.

3e2b45a

ViktorHofer closed this Nov 7, 2019

ViktorHofer force-pushed the master branch from 534783a to 9d685ea Compare November 7, 2019 18:11

safern mentioned this pull request Dec 11, 2019

Fix relro, now and PIE for host and libraries #685

Merged

jkotas mentioned this pull request Jan 31, 2020

Change debug info default variable tracking system #2107

Merged

jbatt33 mentioned this pull request Feb 5, 2020

[runtime] Fix the --enable-minimal=aot build. #31684

Merged

fanyang-mono mentioned this pull request Feb 18, 2020

Infinite recursion during resource lookup within System.Private.CoreLib #32510

Closed

oldzhu mentioned this pull request Mar 8, 2020

[ARM32] illegal instruction when run dotnet to create a console project #33344

Closed

fanyang-mono mentioned this pull request Aug 3, 2020

[interp] DynamicMethod with a CALL of a P/Invoke throws "BadImageFormatException: Method has no body" #38897

Closed

karelz added this to the 5.0.0 milestone Aug 18, 2020

ghost locked as resolved and limited conversation to collaborators Dec 11, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CI pipeline for installer #14

CI pipeline for installer #14

jkoritzinsky commented Nov 5, 2019

ViktorHofer Nov 5, 2019

ViktorHofer Nov 5, 2019

jkoritzinsky Nov 5, 2019

ViktorHofer commented Nov 5, 2019

dagood Nov 5, 2019

jkoritzinsky commented Nov 5, 2019

azure-pipelines bot commented Nov 5, 2019

CI pipeline for installer #14

CI pipeline for installer #14

Conversation

jkoritzinsky commented Nov 5, 2019

ViktorHofer Nov 5, 2019

Choose a reason for hiding this comment

ViktorHofer Nov 5, 2019

Choose a reason for hiding this comment

jkoritzinsky Nov 5, 2019

Choose a reason for hiding this comment

ViktorHofer commented Nov 5, 2019

dagood Nov 5, 2019

Choose a reason for hiding this comment

jkoritzinsky commented Nov 5, 2019

azure-pipelines bot commented Nov 5, 2019