One more: actually build the packages from source. The bootstrap tool takes a set of prebuilt images, replaces a few things and declares it a new image. That means we can't fix bugs in some components because those are not built when this package is built using the bootstrap tool.

@omajid Then we wouldn't need the rover tool?

I saw that you build the payload for RHEL, can you please give me some basic steps for this?

Then we wouldn't need the rover tool?

I believe that we wouldn't need the rover tool for that particular version (or further bugfixes to that version). I don't know if we could use 1.0 to build 1.1, for example. In my experience, .NET Core often needs exact versions to be able to build itself.

I saw that you build the payload for RHEL, can you please give me some basic steps for this?

These are the rough steps I used for 1.0. From what I understand, there are changes happening upstream (the term they are using is "vertical build" or some such) to make this whole thing easier.

Some of the numbers (such as the BuildIds) are still magic to me, so I am not sure how those work (they are computed from the date). Parts were built only on Windows.

Most projects will download a .NET Core build, so you will have to go into the build scripts and patch them to use the version built by rover.

I used the following tags/branches:

• coreclr (branch release/1.0.0)
• corefx (branch 1.0.0)
• ApplicationInsights-dotnet (tag v2.0.0)
• NuGet.Client (tag 3.5.0-beta-final)
• Newtonsoft.Json (tag 7.0.1)
• Roslyn (tag 1.3.0-beta1)
• Roslyn-analyzers
• PlatformAbstractions (branch release)
• symreader
• core-setup (branch 1.0.0)
• cli (branch 1.0.0-preview2)

The build is done in 3 stages (4 in your case, I suppose):

• Stage 0: Use rover to build a working .NET Core
• Stage 1: build coreclr, corefx, ApplicationInsights-dotnet, NuGet.Client, Newtonsoft.Json, roslyn, roslyn-analyzers, symreader, PlatformAbstractions. This is a list of everything that builds completely from source (and doesn't bundle in prebuilt artifacts).
• Stage 2: core-setup (let it use binaries for cli packages). It bundles in various bits from stage1
• Stage 3: cli. Bundles in core-setup from stage 2 and many packages from stage 1. Don't let it use cli nuget packages built in stage 1 or from NuGet.org.

For all the stages, I let everything download whatever it wants from NuGet for build-only dependencies. So if an artifact doesn't end up in the final built cli image, I don't bother building it. This is not acceptable for Fedora, of course, but should be fine for copr.

You can control what NuGet packages each build has access to by overriding NuGet.Config files. For stages 2 and 3, I let it only access a local repository where I manually placed the packages I built in stage 1.

CoreCLR

./build.sh release clean skiptests 2>&1 | tee build.log 
./build-packages.sh release 2>&1 | tee build-packages.log

Take the nuget packages and put it into a repository for stage 2.

CoreFX

./build.sh managed skiptests /t:GenerateVersionHeader /p:GenerateNativeVersionInfo=true /p:OfficialBuildId=20160627-01
./build.sh skiptests release /p:BuildTests=false /p:GenerateNativeVersionInfo=true /p:OfficialBuildId=20160627-01
./build-packages.sh /p:ConfigurationGroup=Release

It needs to be built on Windows too:

.\clean.cmd /all
.\sync.cmd /p
.\build.cmd managed /t:GenerateVersionHeader /p:GenerateNativeVersionInfo=true /p:OfficialBuildId=20160627-01
.\build.cmd /p:Platform=AnyCPU /p:ConfigurationGroup=Release /p:BuildTests=false /p:BuildPackages=false t/p:RestoreDuringBuild=false
.\build-packages.cmd /p:ConfigurationGroup=Release

Take all the nuget packages and put it into a repository for stage 2. Let the windows-built packages override the Fedora-built packages.

ApplicationInsights-dotnet

.\buildRelease.cmd

Newtonsoft.Json

Build it on Windows.

This requires some local changes. First, you will need to find a pre-built Newtonsoft.Json.dll from NuGet.org, and extract the delayed signing key. Use a command like sn -e your.dll <path-to-output> to extract the key.

Then place it somewhere where Build\\Build.ps1 can find it. Also add "/p:DelaySign=true" to the build command and "delaySign" = $true; to the $options variable in Build.ps1.

Use the build scripts in Build directory.

NuGet.Client

Extract the key (using a command similar to that in Newtonsoft.Json). Then

 .\build.ps1 -Configuration Release -ReleaseLabel beta2 -BuildNumber 1484 -SkipTests -SkipCSProj -MSPFXPath <full-path-to-NuGet.Client.snk>

Roslyn

Modify cibuild.cmd to pass /p:OfficialBuild=true and /t:Build to the invocation of MSBuild for BuildAndTest.proj

.\cibuild.cmd /release
mkdir .\Binaries\Packages
 .\Binaries/Release/csi.exe build/BuildNuGets.csx Binaries/Release 1.3.0 Binaries\Packages

Roslyn-analyzers

.\src\.nuget\NuGet.exe Enable -Name https://www.nuget.org/api/v2
cibuild.cmd /release

PlatformAbstractions

set Configuration=Release & build.cmd

core-setup

./build.sh --configuration Release --targets Init,Compile,Package

It should produce a dotnet-$distro-x64-*.tar.gz file.

libuv

This is actually a hack. I feel dirty just writing it out. Here's what I did: take a prebuilt libuv from your distro, grab the libuv NuGet package, unzip it, replace the libuv.so with your distro's libuv.so and zip it back up.

cli

export DOTNET_SKIP_FIRST_TIME_EXPERIENCE=true
# (this prevents dotnet from doing some extraction into a package cache you don't want it to do)
export DOTNET_CLI_FRAMEWORK_HOST_COMPRESSED_FILE=<path to local package source>/dotnet-<distro>-x64.1.0.0.tar.gz
./build.sh --configuration Release --targets Init,Prepare,Compile,Package

The second export is very important. Otherwise cli will download prebuilt binary for core-setup. The build should produce a dotnet-dev-$distro.*.tar.gz file. That's the .NET Core binary.

First of all, thank you so much for this info. Really means a lot.

It's really complicated this build process isn't it, especially since we have to build parts on Windows. :(

"vertical build"

This is it, right?

https://github.com/dotnet/corefx/issues/14592

https://github.com/dotnet/corefx/issues/14492

I believe that we wouldn't need the rover tool for that particular version (or further bugfixes to that version). I don't know if we could use 1.0 to build 1.1, for example. In my experience, .NET Core often needs exact versions to be able to build itself.

I can confirm this, 1.1 didn't want to build with 1.0 payload.

Please bring this comment up next dotnet-sig meeting to see what should we do, we are still so far away it seems.

Thank you so much, again.

RIL issue fixed in the latest build.

Metadata Update from @rhea:
- Issue untagged with: package

Metadata Update from @rhea:
- Issue tagged with: package

Metadata Update from @rhea:
- Issue status updated to: Closed (was: Open)