test262harness - Test262Harness 1.0.0-preview-20240729-0910

Test262 Harness for .NET

PM> Install-Package Test262Harness -Version 1.0.0-preview-20240729-0910 -Source https://www.myget.org/F/test262harness/api/v3/index.json

Copy to clipboard

> nuget.exe install Test262Harness -Version 1.0.0-preview-20240729-0910 -Source https://www.myget.org/F/test262harness/api/v3/index.json

Copy to clipboard

> dotnet add package Test262Harness --version 1.0.0-preview-20240729-0910 --source https://www.myget.org/F/test262harness/api/v3/index.json

Copy to clipboard
<PackageReference Include="Test262Harness" Version="1.0.0-preview-20240729-0910" />
Copy to clipboard
source https://www.myget.org/F/test262harness/api/v3/index.json

nuget Test262Harness  ~> 1.0.0-preview-20240729-0910
Copy to clipboard

> choco install Test262Harness --version 1.0.0-preview-20240729-0910 --source https://www.myget.org/F/test262harness/api/v2

Copy to clipboard
Import-Module PowerShellGet
Register-PSRepository -Name "test262harness" -SourceLocation "https://www.myget.org/F/test262harness/api/v2"
Install-Module -Name "Test262Harness" -RequiredVersion "1.0.0-preview-20240729-0910" -Repository "test262harness" -AllowPreRelease
Copy to clipboard

Build NuGet NuGet MyGet

Test262-Harness-dotnet

This is a .NET test runner for Test262: ECMAScript Test Suite. It includes parsing and downloading logic for the test suite in package Test262Harness and test suite generator functionality via CLI too, Test262Harness.Console

Usage

Following projects are utilizing the test suite generation and show how to create NUnit based test suite that is being generated by downloaded snapshot from test262 GitHub repository.

  • Jint
    • Generates NUnit test suite to ensure compliance, suite can be run in parallel for faster feedback loop
  • esprima-dotnet
    • Generates NUnit test suite for parsing tests, also has custom console logic to compare allow-list.txt for problematic files and progress getting the to parse properly
  • acornima
    • Generates NUnit test suite for parsing tests

First you need need to install the required package to your test project, it should look similar to this:

  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.9.0" />
    <PackageReference Include="NUnit" Version="4.0.1" />
    <PackageReference Include="NUnit3TestAdapter" Version="4.5.0" />
    <PackageReference Include="Test262Harness" Version="1.0.0" />
  </ItemGroup>

Next you will need a configuration, similar to this (also check the configuration format section):

Test262Harness.settings.json example

{
  "SuiteGitSha": "28b31c0bf1960878abb36ab8597a0cae224a684d",
  "TargetPath": "./Generated",
  "Namespace": "My.Tests.Test262",
  "ExcludedFeatures": [
    "Atomics",
    "Temporal"
  ],
  "ExcludedFlags": [
    "async" 
  ],
  "ExcludedDirectories": [
    "annexB",
    "intl402"
  ],
  "ExcludedFiles": [
    "language/expressions/object/dstr-async-gen-meth-*",
    "language/expressions/assignment/fn-name-lhs-cover.js"
  ]
}

You need to create minimal test file stub to initialize your testing target, example from Jint project.

using System;
using System.IO;
using Esprima;
using Jint.Native;
using Jint.Native.ArrayBuffer;
using Jint.Runtime;
using Jint.Runtime.Descriptors;
using Jint.Runtime.Interop;
using Test262Harness;

namespace Jint.Tests.Test262;

public static partial class State
{
    /// <summary>
    /// Pre-compiled scripts for faster execution.
    /// </summary>
    public static readonly Dictionary<string, Script> Sources = new(StringComparer.OrdinalIgnoreCase);
}

/// <summary>
/// Handles initializing testing state.
/// </summary>
public partial class TestHarness
{
    private static partial Task InitializeCustomState()
    {
        foreach (var file in State.HarnessFiles)
        {
            var source = file.Program;
            State.Sources[Path.GetFileName(file.FileName)] = new JavaScriptParser(source, new ParserOptions(file.FileName)).ParseScript();
        }

        return Task.CompletedTask;
    }
}

public abstract partial class Test262Test
{
    private Engine BuildTestExecutor(Test262File file)
    {
        var engine = new Engine(cfg =>
        {
            var relativePath = Path.GetDirectoryName(file.FileName);
            cfg.EnableModules(new Test262ModuleLoader(State.Test262Stream.Options.FileSystem, relativePath));
        });

        if (file.Flags.Contains("raw"))
        {
            // nothing should be loaded
            return engine;
        }

        engine.Execute(State.Sources["assert.js"]);
        engine.Execute(State.Sources["sta.js"]);

        // initialize engine with Test262 expected host defined functions here
        // https://github.com/tc39/test262/blob/main/INTERPRETING.md#host-defined-functions

        engine.SetValue("print", new ClrFunction(engine, "print", (_, args) => TypeConverter.ToString(args.At(0))));
        // and more...

        // the cinded files that that are expected
        foreach (var include in file.Includes)
        {
            engine.Execute(State.Sources[include]);
        }

        return engine;
    }

    private static void ExecuteTest(Engine engine, Test262File file)
    {
        if (file.Type == ProgramType.Module)
        {
            engine.AddModule(file.FileName, builder => builder.AddSource(file.Program));
            engine.ImportModule(file.FileName);
        }
        else
        {
            engine.Execute(new JavaScriptParser(file.Program, new ParserOptions(file.FileName)).ParseScript());
        }
    }

    private partial bool ShouldThrow(Test262File testCase, bool strict)
    {
        return testCase.Negative;
    }
}

And also the CLI tool for generating the test suite, run this in you test project directory.

dotnet tool add Test262Harness.Console

When everything is installed, you should be able to run:

dotnet tool restore
dotnet test262 generate

Test262Harness.settings.json configuration file

List of most important things you can tweak in configuration file:

Key Default Description
SuiteGitSha none The GitHub commit to use when downloading the test suite
SuiteDirectory none Alternatively, you can point to local repository root
TargetPath none Where to generate the file to
Namespace Test262Harness.TestSuite Namespace for the generated source files
ExcludedFeatures [] Any feature you want to ignore
ExcludedFlags [] Any flag you want to ignore
ExcludedDirectories [] Any sub-directory you would like to ignore, for example annexB
ExcludedFiles [] List of specific files you would like to ignore

Exclusion maps to setting [Ignore] attribute in test suite.

Branches and releases

  • .NETFramework 6.0
    • SharpZipLib (>= 1.4.2)
    • YamlDotNet (>= 16.0.0)
    • Zio (>= 0.19.1)
    • ZString (>= 2.6.0)
  • .NETFramework 8.0
    • SharpZipLib (>= 1.4.2)
    • YamlDotNet (>= 16.0.0)
    • Zio (>= 0.19.1)
    • ZString (>= 2.6.0)
  • .NETStandard 2.0
    • SharpZipLib (>= 1.4.2)
    • System.Memory (>= 4.5.5)
    • YamlDotNet (>= 16.0.0)
    • Zio (>= 0.19.1)
    • ZString (>= 2.6.0)
  • .NETFramework 6.0: 6.0.0.0
  • .NETFramework 8.0: 8.0.0.0
  • .NETStandard 2.0: 2.0.0.0

Owners

Marko Lahma

Authors

Marko Lahma

Project URL

https://github.com/lahma/test262-harness-dotnet

License

Unknown

Tags

test262,javascript,ecmascript,test,harness

Info

88 total downloads
1 downloads for version 1.0.0-preview-20240729-0910
Download (57.55 KB)
Found on the current feed only

Package history

Version Size Last updated Downloads Mirrored?
1.0.0-preview-20241216-0300 57.53 KB Mon, 16 Dec 2024 03:01:21 GMT 1
1.0.0-preview-20241209-0620 57.55 KB Mon, 09 Dec 2024 06:21:15 GMT 1
1.0.0-preview-20241209-0619 57.53 KB Mon, 09 Dec 2024 06:20:28 GMT 1
1.0.0-preview-20241202-1104 57.53 KB Mon, 02 Dec 2024 11:05:02 GMT 1
1.0.0-preview-20241202-0657 57.53 KB Mon, 02 Dec 2024 06:58:15 GMT 1
1.0.0-preview-20241125-1547 57.53 KB Mon, 25 Nov 2024 15:48:15 GMT 1
1.0.0-preview-20241125-1546 57.53 KB Mon, 25 Nov 2024 15:47:16 GMT 1
1.0.0-preview-20241118-0716 57.54 KB Mon, 18 Nov 2024 07:17:32 GMT 1
1.0.0-preview-20241111-0718 57.55 KB Mon, 11 Nov 2024 07:19:30 GMT 1
1.0.0-preview-20241111-0515 57.55 KB Mon, 11 Nov 2024 05:15:35 GMT 1
1.0.0-preview-20241111-0511 57.53 KB Mon, 11 Nov 2024 05:12:14 GMT 1
1.0.0-preview-20241014-0600 57.53 KB Mon, 14 Oct 2024 06:01:04 GMT 1
1.0.0-preview-20241007-0504 57.54 KB Mon, 07 Oct 2024 05:04:43 GMT 1
1.0.0-preview-20240930-0559 57.54 KB Mon, 30 Sep 2024 05:59:42 GMT 1
1.0.0-preview-20240916-0605 57.54 KB Mon, 16 Sep 2024 06:05:57 GMT 1
1.0.0-preview-20240916-0604 57.54 KB Mon, 16 Sep 2024 06:05:06 GMT 1
1.0.0-preview-20240910-0525 57.53 KB Tue, 10 Sep 2024 05:25:56 GMT 1
1.0.0-preview-20240909-0532 57.53 KB Mon, 09 Sep 2024 05:33:12 GMT 1
1.0.0-preview-20240902-0511 57.53 KB Mon, 02 Sep 2024 05:12:22 GMT 1
1.0.0-preview-20240902-0510 57.53 KB Mon, 02 Sep 2024 05:11:23 GMT 1
1.0.0-preview-20240826-0405 57.55 KB Mon, 26 Aug 2024 04:06:03 GMT 1
1.0.0-preview-20240826-0322 57.54 KB Mon, 26 Aug 2024 03:22:59 GMT 1
1.0.0-preview-20240819-0324 57.54 KB Mon, 19 Aug 2024 03:25:12 GMT 1
1.0.0-preview-20240805-0541 57.51 KB Mon, 05 Aug 2024 05:42:03 GMT 0
1.0.0-preview-20240731-0730 57.54 KB Wed, 31 Jul 2024 07:30:41 GMT 1
1.0.0-preview-20240729-0910 57.55 KB Mon, 29 Jul 2024 09:11:17 GMT 1
1.0.0-preview-20240722-0510 57.54 KB Mon, 22 Jul 2024 05:10:38 GMT 1
1.0.0-preview-20240715-0802 57.53 KB Mon, 15 Jul 2024 08:03:15 GMT 1
1.0.0-preview-20240715-0609 53.94 KB Mon, 15 Jul 2024 06:10:22 GMT 1
1.0.0-preview-20240715-0608 53.93 KB Mon, 15 Jul 2024 06:09:10 GMT 1
1.0.0-preview-20240624-0519 53.95 KB Mon, 24 Jun 2024 05:20:23 GMT 1
1.0.0-preview-20240624-0518 53.95 KB Mon, 24 Jun 2024 05:18:39 GMT 1
1.0.0-preview-20240617-0501 53.94 KB Mon, 17 Jun 2024 05:02:05 GMT 1
1.0.0-preview-20240603-0428 53.93 KB Mon, 03 Jun 2024 04:28:50 GMT 1
1.0.0-preview-20240603-0413 53.94 KB Mon, 03 Jun 2024 04:13:48 GMT 1
1.0.0-preview-20240603-0412 53.94 KB Mon, 03 Jun 2024 04:13:22 GMT 2
1.0.0-preview-20240527-0413 53.95 KB Mon, 27 May 2024 04:14:05 GMT 1
1.0.0-preview-20240519-0655 53.94 KB Sun, 19 May 2024 06:55:34 GMT 1
1.0.0-preview-20240216-1831 53.59 KB Fri, 16 Feb 2024 18:32:00 GMT 2
0.0.23-preview-20240216-1828 53.61 KB Fri, 16 Feb 2024 18:29:00 GMT 2
0.0.23-preview-20240216-1815 54.36 KB Fri, 16 Feb 2024 18:16:22 GMT 2
0.0.23-preview-20240216-1806 54.37 KB Fri, 16 Feb 2024 18:07:17 GMT 2
0.0.23-preview-20231228-1625 54.37 KB Thu, 28 Dec 2023 16:26:56 GMT 2
0.0.22-preview-20230810-1927 35.12 KB Thu, 10 Aug 2023 19:28:00 GMT 2
0.0.22-preview-20230810-0754 33.25 KB Thu, 10 Aug 2023 07:55:00 GMT 4
0.0.22-preview-20230507-1809 33.5 KB Sun, 07 May 2023 18:10:21 GMT 2
0.0.22-preview-20220925-0721 33.26 KB Sun, 25 Sep 2022 07:22:01 GMT 3
0.0.19-preview-20220918-0624 33.28 KB Sun, 18 Sep 2022 06:25:39 GMT 2
0.0.19-preview-20220807-0752 33.41 KB Sun, 07 Aug 2022 07:53:12 GMT 2
0.0.19-preview-20220807-0731 33.21 KB Sun, 07 Aug 2022 07:34:34 GMT 4
0.0.19-preview-20220807-0723 33.21 KB Sun, 07 Aug 2022 07:24:18 GMT 2
0.0.19-preview-20220730-1217 33.55 KB Sat, 30 Jul 2022 12:18:08 GMT 2
0.0.19-preview-20220730-1213 33.54 KB Sat, 30 Jul 2022 12:13:37 GMT 2
0.0.19-preview-20220730-0917 33.42 KB Sat, 30 Jul 2022 09:18:10 GMT 4
0.0.19-preview-20220730-0914 33.42 KB Sat, 30 Jul 2022 09:14:24 GMT 2
0.0.17-preview-20220730-0911 33.43 KB Sat, 30 Jul 2022 09:11:32 GMT 2
0.0.17-preview-20220526-1709 33.44 KB Thu, 26 May 2022 17:10:13 GMT 2
0.0.17-preview-20220526-1659 33.54 KB Thu, 26 May 2022 16:59:44 GMT 2
0.0.16 33.4 KB Thu, 26 May 2022 16:44:59 GMT 3