[Proposal] Ignored Directives Support

[333fred]

Ignored Directives

• Proposed: https://github.com/dotnet/csharplang/blob/main/proposals/ignored-directives.md
• Prototype
• Implementation
• Specification

Summary

As we move to allow C# statements in the top level of a file, it's reasonable to assume that we might someday get support for dotnet runing a .cs file. For further use of C# as a scripting language in this vein, I propose that we introduce a new preprocessor directive who's purpose is to exist solely to be ignored by the language: the hashbang directive. Linux and Unix scripts will often use #!<path to interpreter> in order to direct the program loader to the interpreter that can run them. We should add a similar ignore to the C# language.

Detailed design

We introduce a new directive:

pp_directive
    : ... // Existing directives
    | pp_hashbang
    | pp_reference
    ;

pp_hashbang
    : whitespace? '#' '!' input_character*
    ;

pp_reference
    : whitespace? '#' 'r' whitespace input_character*
    ;

The language ignores the contents of the hashbang and reference directives and moves to the next line.

Drawbacks

Any change adds complication to the language. If we implement support for this and it starts appearing in files without actual scripting support, then it could be confusing for users expecting it to work.

Alternatives

N/A

Unresolved questions

1. If we use these directives to support tooling such as nuget references, we will need to consider how they interact with other pragmas, such as #if. We have a few options for this:
   1. Require that ignored directives are the first directives in a file. We could even go further and require they're all at the beginning of the file.
   2. Less restrictive than 1, but forbid ignored directives inside conditional directives.
   3. Least restrictive, require a true preprocessing step in the compiler. Today, we can process directives up front without having to do a full multi-step preprocessing pass before lexing, but if we want to support nuget directives inside conditional directives we may need the full two-step process.

Design meetings

• https://github.com/dotnet/csharplang/blob/main/meetings/2021/LDM-2021-05-12.md#simple-c-programs
• https://github.com/dotnet/csharplang/blob/main/meetings/2022/LDM-2022-08-31.md#ignored-directives-support
• https://github.com/dotnet/csharplang/blob/main/meetings/2022/LDM-2022-09-26.md#ungrouped

[ufcpp]

I'm worried about BOM. The presence of the BOM before the shebang will prevent the script interpreter.

Visual Studio for Win uses Encoding.Default (usually not UTF8) if the BOM doesn't present.

[333fred]

Sure, and we could certainly consider warning if the file uses anything other than UTF-8 w/o BOM.

[yaakov-h]

I thought shebang was already supported... is it only in the .csx variant?

[yaakov-h]

ah, looks like its only currently supported in the scripting mode.

https://github.com/dotnet/roslyn/blob/1b266a1e67434578ec09737a4e9427e52f44a420/src/Compilers/CSharp/Portable/Parser/DirectiveParser.cs#L108-L111

[svick]

1. As @yaakov-h mentions, shebang is already supported in the scripting dialect. Is this effectively formalizing that support and making it work in both dialects?
2. If I understand it correctly, the shebang that's supported by the scripting mode has to be literally the first thing in the file: no whitespace before it, no other lines before it. Since that's the only way shebang works, does it make sense to support whitespace before it and to support it on lines other than the first line in the file?

[HaloFour]

I'm a little confused. Do "simple programs" not require a project file? How would a single file "script" in this manner establish references or change any compiler properties?

This feels like trying to bridge a gap between C# "simple programs" and CSX, which already has this support plus the directive support to make it actually useful. Short of adding that directive support this feels extremely limited.

Again, C# would benefit so much more from embracing CSX and improving the tooling support than it ever will by chasing CSX through some separate dialect that will always be much less capable.

[dersia]

I agree with @HaloFour without directives for assembly imports this might not be very useful. On the other hand having a script that downloads nuget packages or loading complex and somewhat big assemblies doesn't feel right for scripting.

The only thing that I kind of see as a big plus is that you can just take the cs file to any machine architecture and call dotnet run on it so you don't have to compile aot.

What is still unclear for me is if the purpose of this proposal is that the cs-file is being interpreted or is it going to be compiled on running it.

If the purpose is to "just" interpreted on run, than I agree even more with @HaloFour we should focus on making csx a better experience.

[hez2010]

I don't think it should introduce a new #! directive to C#, for C# doesn't use # for comment syntax.

How about:

/// <interpreter path="" />

Which will allow more complex definition, for example:

/// <project target="net5.0">
///   <properties>
///     <nullable>enable</nullable>
///   </properties>
///   <items>
///     <package include="Newtonsoft.Json" version="12.0.3" />
///   </items>
/// </project>

[HaloFour]

The point is to use a syntax that existing shells already recognize, otherwise there would be no benefit. Unix shells recognize shebang syntax and it is widely used to enable a script file to instruct the shell in how that file is to be interpreted. No shell will understand a C# comment or a blob of XML.

…

On Thu, May 28, 2020, 10:31 PM Steve ***@***.***> wrote: How about: /// <interpreter path="" /> I don't think it should introduce a new #! directive to C#, for C# doesn't use # for comment syntax. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3507 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACNCMEWBCWZ6OAC346MHQHTRT4M6VANCNFSM4NMYL2BQ> .

[aka-nse]

I summarize my understanding of this proposal at followings. Is it right?

• C# compiler will simply ignore a shebang in the first line of the source. Because the lines start with # are preprocessor directives even on conventional spec, this modification looks not to impact other features.

• You can write inline csproj XML on and after 2nd line, but the interpretation of them should be done by build tool (e.g. dotnet-cli), not compiler. Such an inline csproj will be written under the conventional comment syntax, and the extension spec will be defined by tools.

#!/bin/dotnet run
//+ <Project Sdk="Microsoft.NET.Sdk">
//+     <PropertyGroup>
//+         <TargetFramework>netcoreapp3.1</TargetFramework>
//+     </PropertyGroup>
//+ </Project>

/*
 * In this example, it is assumed that dotnet-cli will interprets the comments start with `//+` as inline csproj
 */

using System;

Console.WriteLine("Hello, world!");

[333fred]

The first point is correct, and is the only thing this proposal covers. This proposal makes no statement on how a program would communicate dependencies to a tool.

[rolfbjarne]

One of the design meetings says:

Would be nice to have, but only in conjunction with the rest of the simple programs feature. Implementation might be one of the simplest ever features in the compiler, but without the tooling to accompany it we don't know that this is the right design.

I'm not sure I agree, because this doesn't actually need official tooling to be useful, it's entirely possible to extend by anyone.

Consider this:

#!/bin/my-custom-program

using System;

Console.WriteLine("Hello, world!");

where /bin/my-custom-program can be a program that takes the script and does whatever it wants to - which could be creating a project file for the file and running dotnet run, but it an also be anything else.

One example could be:

#!/bin/run-unit-test

[TestFixture]
public class Test {
    [Test]
    public void IsValid (int number)
    {
        Assert.AreEqual (42, number, "everything");
    }
}

where run-unit-test would create a project, add the corresponding NUnit references and usings, compile & run the tests.

[CyrusNajmabadi]

it's entirely possible to extend by anyone.

The point is that without that tooling we don't know if this will be good enough to actually solve those cases.

[rolfbjarne]

it's entirely possible to extend by anyone.

The point is that without that tooling we don't know if this will be good enough to actually solve those cases.

I agree that the reference directive (#r) might be in this category, but the shebang directive (#!) can stand on its own and would be useful by itself, both without the reference directive, or any other tooling.

[CyrusNajmabadi]

it's entirely possible to extend by anyone.

The point is that without that tooling we don't know if this will be good enough to actually solve those cases.

I agree that the reference directive (#r) might be in this category, but the shebang directive (#!) can stand on its own and would be useful by itself, both without the reference directive, or any other tooling.

What is it's not good enough? What if we do it, and a month later people try to use it and say it's not sufficient and it needs to change or be redone.

We can't add features without some way of actually validating that it solves the problems we want it to solve. We need some actual tooling adoption to demonstrate it is sufficient.

[jaredpar]

Parts of the runtime arguably do (source generators), but this is besides the point

Can you list an example? How does the runtime implicitly add or modify inputs to the compilations? How are source generators relevant to this discussion? Source generators, ideally, follow the rules of the compiler with respect to inputs (don't read anything that wasn't explicitly passed to the compilation). Yes some violate this but that is considered a bug.

If they want to do something unprecedented, they can

Sure we can ... but we have to ask if doing so fits into or subverts users expectations. Given that #pragma today is purely about controlling user diagnostics, having it suddenly be a way to change inputs to the compilation is very likely to fit into the subverts expectations category.

Like, the language doesn't (and shouldn't) have a concept of a package or SDK and there are good reasons for this.

This proposal doesn't change the C# language to understand packages. It is stating that it supports a directive that compilation tools can choose to, or choose not to, use. Similar to #region where the compiler hosts in IDEs leverage the directive but the command line compiler largely ignores it.

[333fred]

Do nothing (and use #pragma)

This option is simply a non-starter, as has been detailed numerous times in this discussion. Shells use #!. We don't control that. At the barest minimum, that directive must be added.

[jkrejcha]

Re source generators: they add new source that's then compiled (regex gen, libraryimport, etc). They're relevant if only to say that "adding a magic attribute adds a compilation unit" isn't unprecedented, in fact, it's pretty standard, and in fact I'd say the major current use of source generators. That's not against spec or anything, compilation units aren't limited to on-disk files. I feel like though there is some miscommunication here and it's more illustrative that similar things exist in .NET already and used the existing tooling just fine.

It's orthogonal to my main point ultimately.

---

Sure we can ... but we have to ask if doing so fits into or subverts users expectations.

Which is that #pragma does something implementation-defined and not much more than that. That it happens to be useful for controlling diagnostics is a specific feature .NET provides along with... I guess checksumming. Especially considering that "pragma" generally is understood to mean "controls how input is processed in a implementation-defined way", I don't really see how this is all that confusing.

This proposal doesn't change the C# language to understand packages. It is stating that it supports a directive that compilation tools can choose to, or choose not to, use.

I think this is partially my point. There's no defined semantics for packages (and nor should there be). Why then should the language have anything to say about it?

---

To be clear, I'm actually pretty in favor of something like conditionally-supported directives. It'd change the error state of unknown directives to something more of an implementation-defined thing and increases backwards compatibility somewhat, and solves the fact that the *nix-like OS loader cares about shebangs for those actually running C# code as a script or whatever

It also has the advantage of not tying these directives to the language version and makes it so that if you want a snazzy new directive you don't have to go to LDM every single time.

The only disadvantage ig is that pragma probably becomes much less attractive as an option even where it's the right choice (which would probably be most of the time).

[jkrejcha]

This option is simply a non-starter, as has been detailed numerous times in this discussion. Shells use #!. We don't control that. At the barest minimum, that directive must be added.

It's the status quo... most programming languages don't special case this (iirc Python used to do some weird related hack at some point for DOS (the operating system) related reasons but they dropped that a very very long time ago). Non-starter seems a lil much in my view.

Imo, either supporting the prefix for implementation-defined anything or just supporting anything (in implementation-defined manner if unknown) imo would be much less hacky and piecemeal.

[jaredpar]

Re source generators: they add new source that's then compiled (regex gen, libraryimport, etc)

Yes they add source as a code object but this is 100% internal to the compilation process. There is no reaching out of the compilation and grabbing new artifacts off of disk. The source generator, in combination with the other compiler inputs, perfectly describes the inputs to the compilation.

Especially considering that "pragma" generally is understood to mean "controls how input is processed in a implementation-defined way"

This claim is simply incorrect.

There's no defined semantics for packages (and nor should there be). Why then should the language have anything to say about it?

The language has nothing to say about it, just as the language has nothing to say about #region. It is just a directive the language says that the host can define semantics for.

[jkrejcha]

To be honest, I don't really seem the advantage this has in the language over just using pragma since this is already implementation specific.

The idea of sdk and package is effectively specific only to the .NET implementation (the tooling)

[jkrejcha]

The only weird one is #! but even that could just be an extension used by CSX or something (it already has its own implementation with #r).

Interestingly of note, it does seem that .NET effectively just ignores compiler directives it doesn't know about. (See below)

[jjonescz]

it does seem that .NET effectively just ignores compiler directives it doesn't know about.

If by "ignore" you mean "warns about", that's true but only for pragma directives, not arbitrary directives, i.e.:

#pragma test // warning CS1633: Unrecognized #pragma directive
#test // error CS1024: Preprocessor directive expected

[jkrejcha]

Ah ya, I checked again and it seemed I had a different file open when I tested the latter. Whoops.