Add platform support for FreeBSD

[Thefrank]

PR Summary

Adds build and platform support for FreeBSD
Adds tests and updates comments for FreeBSD relevant items

PR Context

FreeBSD is a community supported dotNET RID and this PR expands Powershell's ability to run under FreeBSD

PR Checklist

• PR has a meaningful title
  • Use the present tense and imperative mood when describing your changes
• Summarized changes
• Make sure all .h, .cpp, .cs, .ps1 and .psm1 files have the correct copyright header
• This PR is ready to merge and is not Work in Progress.
  • If the PR is work in progress, please add the prefix WIP: or [ WIP ] to the beginning of the title (the WIP bot will keep its status check at Pending while the prefix is present) and remove the prefix when the PR is ready.
• Breaking changes
  • None
  • OR
  • Experimental feature(s) needed
    • Experimental feature name(s):
• User-facing changes
  • Not Applicable
  • OR
  • Documentation needed
    • Issue filed:
• Testing - New and feature
  • N/A or can only be tested interactively
  • OR
  • Make sure you've added a new test if existing tests do not effectively test the code changed
• Tooling
  • I have considered the user experience from a tooling perspective and don't believe tooling will be impacted.
  • OR
  • I have considered the user experience from a tooling perspective and opened an issue in the relevant tool repository. This may include:
    • Impact on PowerShell Editor Services which is used in the PowerShell extension for VSCode
      (which runs in a different PS Host).
      • Issue filed:
    • Impact on Completions (both in the console and in editors) - one of PowerShell's most powerful features.
      • Issue filed:
    • Impact on PSScriptAnalyzer (which provides linting & formatting in the editor extensions).
      • Issue filed:
    • Impact on EditorSyntax (which provides syntax highlighting with in VSCode, GitHub, and many other editors).
      • Issue filed:

I have left this as WIP for two reasons:

• It requires Fix FreeBSD build failures. Update tests for FreeBSD PowerShell-Native#92 for updated systemcalls
• Feedback

Finally, this PR is roughly based on the work jasonpugsley@a3aeb12 and directly uses the PID and PPID functionality from it.

[pull-request-quantifier-deprecated]

This PR has 296 quantified lines of changes. In general, a change size of upto 200 lines is ideal for the best PR experience!

---

Quantification details

Label      : Large
Size       : +228 -68
Percentile : 69.6%

Total files changed: 48

Change summary by file extension:
.psm1 : +49 -20
.csproj : +7 -7
.cs : +78 -5
.resx : +4 -1
.ps1 : +87 -34
.sh : +3 -1

Change counts above are quantified counts, based on the PullRequestQuantifier customizations.

Why proper sizing of changes matters

Optimal pull request sizes drive a better predictable PR flow as they strike a
balance between between PR complexity and PR review overhead. PRs within the
optimal size (typical small, or medium sized PRs) mean:

• Fast and predictable releases to production:
  • Optimal size changes are more likely to be reviewed faster with fewer
    iterations.
  • Similarity in low PR complexity drives similar review times.
• Review quality is likely higher as complexity is lower:
  • Bugs are more likely to be detected.
  • Code inconsistencies are more likely to be detected.
• Knowledge sharing is improved within the participants:
  • Small portions can be assimilated better.
• Better engineering practices are exercised:
  • Solving big problems by dividing them in well contained, smaller problems.
  • Exercising separation of concerns within the code changes.

What can I do to optimize my changes

• Use the PullRequestQuantifier to quantify your PR accurately
  • Create a context profile for your repo using the context generator
  • Exclude files that are not necessary to be reviewed or do not increase the review complexity. Example: Autogenerated code, docs, project IDE setting files, binaries, etc. Check out the Excluded section from your prquantifier.yaml context profile.
  • Understand your typical change complexity, drive towards the desired complexity by adjusting the label mapping in your prquantifier.yaml context profile.
  • Only use the labels that matter to you, see context specification to customize your prquantifier.yaml context profile.
• Change your engineering behaviors
  • For PRs that fall outside of the desired spectrum, review the details and check if:
    • Your PR could be split in smaller, self-contained PRs instead
    • Your PR only solves one particular issue. (For example, don't refactor and code new features in the same PR).

How to interpret the change counts in git diff output

• One line was added: +1 -0
• One line was deleted: +0 -1
• One line was modified: +1 -1 (git diff doesn't know about modified, it will
  interpret that line like one addition plus one deletion)
• Change percentiles: Change characteristics (addition, deletion, modification)
  of this PR in relation to all other PRs within the repository.

---

Was this comment helpful? 👍  :ok_hand:  :thumbsdown: (Email)
Customize PullRequestQuantifier for this repository.

[Thefrank]

CI tooling does not like IsFreeBSD apparently. I will need to find a way to work around that.

[Thefrank]

PowerShell/PowerShell-Native#92 has been merged in.

I will try and get this PR rebased this weekend.

Edit: Initial rebase done. Still needs work

[Thefrank]

As I am working back over this I am noticing a few things:

• Bootstrapping issue: Checks system does not like IsFreeBSD. This builds under FreeBSD without bootstrapping without a working pwsh. It will build with bootstrapping with a working pwsh. I am unsure how to work around this for the checks system.

• Testing with Pester: Tests require an update to Pester (There is a pending PR). Otherwise this needs to be run via Start-PSPester and wait for it to error out. Add FreeBSD as a supported OS. Rerun with Start-PSPester -SkipTestToolBuild

• WebCmdlets.Tests.ps1 have similar issues. This is unfixable upstream. The repo has been archived for almost 3 years. https://github.com/PowerShell/SelfSignedCertificate/blob/5f720d3d5b28c2cb2e9b33296cfd802ec6f1a668/SelfSignedCertificate/SelfSignedCertificate.psm1#L24. About 10 tests fail with Exception setting "FriendlyName": "The FriendlyName value cannot be set on Unix.". Once FreeBSD is added here there are no more test failures from this.

• Remaining test failures: 5. I am not sure why they are failing as similar ones in their test packages pass.

No idea on why just this fails.

              <test-case description="Word wrapping for string with escape sequences" name="Handle strings with escape sequences in formatting.Word wrapping for string with escape sequences" time="0.0343" asserts="0" success="False" result="Failure" executed="True">
                <failure>
                  <message>Expected strings to be the same, but they were different.
Expected length: 123
Actual length:   95
Strings differ at index 45.
Expected: '...hell &amp;27;[0m\n...'
But was:  '...hell scrip...'</message>
                  <stack-trace>at &lt;ScriptBlock&gt;, /root/Powershell/test/powershell/engine/Formatting/PSStyle.Tests.ps1: line 501
501:         $text.Trim().Replace("`r", "") | Should -BeExactly $expected.Replace("`r", "")</stack-trace>
                </failure>
              </test-case>

This one appears to come from runtime and not pwsh

              <test-suite type="TestFixture" name="Data, Config, and Cache locations" executed="True" result="Failure" success="False" time="0.844" asserts="0" description="Data, Config, and Cache locations">
                <results>
                  <test-case description="Should start if Data, Config, and Cache location is not accessible" name="ConsoleHost unit tests.Data, Config, and Cache locations.Should start if Data, Config, and Cache location is not accessible" time="0.6604" asserts="0" success="False" result="Failure" executed="True">
                    <failure>
                      <message>Expected the actual value to be greater than 0, but got 0.</message>
                      <stack-trace>at &lt;ScriptBlock&gt;, /root/Powershell/test/powershell/Host/ConsoleHost.Tests.ps1: line 675
675:             [int]$output | Should -BeGreaterThan 0</stack-trace>
                    </failure>
                  </test-case>
                </results>
              </test-suite>

This one might be filesystem or runtime related. Not 100% sure.

                  <test-case description="Resolve-Path -Path './test*.txt' -RelativeBasePath '/tmp/cf3edf0f-4706-4da4-b2f8-ce6d79acf9e2' -Force:$false should return '$null'" name="Resolve-Path returns proper path.Resolve-Path -Path './test*.txt' -RelativeBasePath '/tmp/cf3edf0f-4706-4da4-b2f8-ce6d79acf9e2' -Force:$false should return '$null'" time="0.0389" asserts="0" success="False" result="Failure" executed="True">
                    <failure>
                      <message>Expected exactly $null, but got @('/tmp/cf3edf0f-4706-4da4-b2f8-ce6d79acf9e2/test1.txt', '/tmp/cf3edf0f-4706-4da4-b2f8-ce6d79acf9e2/test2.txt').</message>
                      <stack-trace>at &lt;ScriptBlock&gt;, /root/Powershell/test/powershell/Modules/Microsoft.PowerShell.Management/Resolve-Path.Tests.ps1: line 176
176:         (Resolve-Path -Path $Path -RelativeBasePath $BasePath -Force:$Force).Path | Should -BeExactly $ExpectedResult</stack-trace>
                    </failure>
                  </test-case>

These two are likely FreeBSD related

              <test-case description="Should be able to import string using default culture" name="Import-LocalizedData.Should be able to import string using default culture" time="0.0171" asserts="0" success="False" result="Failure" executed="True">
                <failure>
                  <message>Expected strings to be the same, but they were different.
Expected length: 5
Actual length:   0
Strings differ at index 0.
Expected: 'en-US'
But was:  ''</message>
                  <stack-trace>at &lt;ScriptBlock&gt;, /root/Powershell/test/powershell/Modules/Microsoft.PowerShell.Utility/Import-LocalizedData.Tests.ps1: line 16
16: $culture.Name     | Should -BeExactly "en-US"</stack-trace>
                </failure>
              </test-case>

              <test-case description="using -displayhint produces the correct output" name="Get-Date DRT Unit Tests.using -displayhint produces the correct output" time="0.0134" asserts="0" success="False" result="Failure" executed="True">
                <failure>
                  <message>Expected strings to be the same, but they were different.
String lengths are both 26.
Strings differ at index 11.
Expected: 'Wednesday, January 1, 2020'
But was:  'Wednesday, 01 January 2020'</message>
                  <stack-trace>at &lt;ScriptBlock&gt;, /root/Powershell/test/powershell/Modules/Microsoft.PowerShell.Utility/Get-Date.Tests.ps1: line 20
20:         $d.Trim() | Should -Be "Wednesday, January 1, 2020"</stack-trace>
                </failure>

Some more details

PS /root/Powershell> Get-Culture | Fl *

Parent                         : 
LCID                           : 127
KeyboardLayoutId               : 127
Name                           : 
IetfLanguageTag                : 
DisplayName                    : Invariant Language (Invariant Country)
NativeName                     : Invariant Language (Invariant Country)
EnglishName                    : Invariant Language (Invariant Country)
TwoLetterISOLanguageName       : iv
ThreeLetterISOLanguageName     : ivl
ThreeLetterWindowsLanguageName : IVL
CompareInfo                    : CompareInfo - 
TextInfo                       : TextInfo - 
IsNeutralCulture               : False
CultureTypes                   : SpecificCultures
NumberFormat                   : System.Globalization.NumberFormatInfo
DateTimeFormat                 : System.Globalization.DateTimeFormatInfo
Calendar                       : System.Globalization.GregorianCalendar
OptionalCalendars              : {System.Globalization.GregorianCalendar}
UseUserOverride                : False
IsReadOnly                     : True

PS /root/Powershell> locale
LANG=C.UTF-8
LC_CTYPE="C.UTF-8"
LC_COLLATE="C.UTF-8"
LC_TIME="C.UTF-8"
LC_NUMERIC="C.UTF-8"
LC_MONETARY="C.UTF-8"
LC_MESSAGES="C.UTF-8"
LC_ALL=

GitHub

SelfSignedCertificate/SelfSignedCertificate/SelfSignedCertificate.psm1 at 5f720d3d5b28c2cb2e9b33296cfd802ec6f1a668 · PowerShell/SelfSignedCertificate

A module for generating self-signed certificates in PowerShell Core - PowerShell/SelfSignedCertificate

[Thefrank]

net9 now has better support for FreeBSD and the SDK will under FreeBSD RIDs.

This does not change the fact that this PR still has a bootstrapping issue as it requires a pwsh that understands what FreeBSD is first and I am unsure how to resolve that in this PR. Is this something that needs to be done outside of a PR? A different PR?

[iSazonov]

Is this something that needs to be done outside of a PR? A different PR?

We would manually compile and publish initial pwsh package before CIs on Github could use it. Although I do not even know if the MSFT team wants to have it officially. @Thefrank Did you know that FreeBSD is used in Azure? Such scenarios could be a trigger for solving this issue.

[Thefrank]

I think official support for FreeBSD is unlikely even in the near future and I am OK with that. I have no issue removing FreeBSD references in the download scripts. Officially supported (with or without a Microsoft download) can imply to users that they can get official support. This adds a support burden even if it is just "Issue Closed: File with FreeBSD bugs tracker". If pwsh was just aware of FreeBSD, even abstractly as a platform, (e.g., isFreeBSD) it will allow the checks system here to make sure I have not impacted other platforms. I tried to be defensive as possible, where possible.

---

More complex ecosystem details:

• dotNET SDK knows about FreeBSD (x64 and arm64) so those using it can target FreeBSD. There is no official dotNET download for FreeBSD right now as it remains a "community supported platform". Understandable, FreeBSD is not even close to OSX in terms of usage.
  • FreeBSD users can pkg install dotnet as there is a pkg version that is maintained by the FreeBSD community. https://www.freshports.org/lang/dotnet/
• The Azure Agent will run under FreeBSD.
  • There is a package for it: https://www.freshports.org/sysutils/azure-agent
• The Azure Marketplace has FreeBSD images.
  • I have not tried them but they exist here https://azuremarketplace.microsoft.com/en-us/marketplace/apps/thefreebsdfoundation.freebsd-14_0 .
• The Azure Pipelines Agent works under FreeBSD
  • It requires the user to patch and build with: Adjustments for FreeBSD microsoft/azure-pipelines-agent#3266 but it runs well. I like this more than some other CI/CD solutions.

FreshPorts -- lang/dotnet: Open-source developer platform and a cross-platform runtime for cloud, mobile, desktop, and IoT apps

.NET is an open source developer platform, created by Microsoft, for building many different types of applications. It provides a standard set of base class libraries and APIs that are common to all .NET applications. .NET apps can be written in such languages as C#, F#, or Visual Basic.

FreshPorts -- sysutils/azure-agent: Microsoft Azure Linux Agent

The Microsoft Azure Linux Agent

LICENSE: APACHE20

Microsoft Azure Marketplace

[kasini3000]

Looking forward to the release of the for freebsd version soon.
Perhaps we can release an alpha version for fans to test first.

[microsoft-github-policy-service]

This pull request has been automatically marked as Review Needed because it has been there has not been any activity for 7 days.
Maintainer, please provide feedback and/or mark it as Waiting on Author

[sec]

Any chance to get this upstreamed to maybe help with testing/fixing rest of the things for more ppl?

[sec]

We have working port build of PowerShell 7.5.0 - https://github.com/sec/freebsd-ports/tree/powershell.v7.5.0/shells/powershell - that soon maybe will come into upstream - would be nice to know the status of those changes

GitHub

freebsd-ports/shells/powershell at powershell.v7.5.0 · sec/freebsd-ports

FreeBSD ports tree (read-only mirror). Contribute to sec/freebsd-ports development by creating an account on GitHub.

[Thefrank]

@sec AFAIK this PR is unlikely to be merged due mainly to a bootstrapping issue (e.g., Checks system can't run without knowing about FreeBSD) and to a lesser extent no official support for community supported dotNET platforms.

Looking over it right now, it may also require a rebase. This is something I can get to this weekend. Edit: Not needed.

I have no issue in closing this in favor of a more likely/qualified/coded PR.

[Thefrank]

@sec Is this safe to close? I see we have ports version on FreeBSD now 🎉

• This PR might conflict with it as it makes changes to similar files. This might create more work the port maintainer.
• I do not have the time right now to troubleshoot the final failing five tests. I assume these have been resolved in the port.
• This is designed for portable RIDs. FreeBSD does not use them and opts for non-portable OS version specific runtimes.
• While an official version of pwsh published by Microsoft is unlikely, I also understand that even adding code for an unofficial platform creates a future code burden further reducing this PR's chances of a merge.
• This PR is over one year old and there is likely a better solution by someone else at this point (e.g., FreeBSD ports)
• Finally, I have no idea who else to ping on this PR for feedback.

[sec]

The port was based on some previous work to make pwsh build and just adjusted to 7.5.0 - I don't think anything was done related to test, as to run the test some changes are needed (which are part of your PR as I saw) - looking that there is no move or will to make those merged, only thing we can do is to incorporate changes from your PR into FreeBSD port as patch files, if that make sense.

[kasini3000]

Thanks a lot, the Kasini3000 project has been supported freebsd by your hard work.
It would be even better if PowerShell could be turned into freebsd “pkg”.

[Thefrank]

@kasini3000 it is in ports as of last month: https://www.freshports.org/shells/powershell/
@sec and @arrowd did the work getting that and the library for pwsh working in ports. This PR at best duplicates that work and at worst conflicts with it making future ports harder to maintain (i.e. patches are upstream instead of ports)

FreshPorts -- shells/powershell: Microsoft's shell with support for .NET objects

PowerShell is a modern command shell that includes the best features of other popular shells. Unlike most shells that only accept and return text, PowerShell accepts and returns .NET objects.