warbo-utilities: a9a0ba3d76a75ec90ed669a2a23f08c35885bbd7

     1: From: Chris Warburton
     2: Date: Fri, 08 Jun 2018 13:21:36 +0100
     3: State: new
     4: Subject: Make trackGit ask whether to make ASV benchmarks
     5: Message-Id: <cc3f744da13eaf79-0-artemis@nixos>
     6: 
     7: Ask if we want some ASV benchmarks, if not already found. If so we
     8: should make a benchmarks/ dir, a skeleton benchmarks/default.nix which
     9: takes root and dir, imports pkgs.nix from dir, defines 'fixed' and
    10: 'measured' as imports of 'dir' and 'root' with 'packageOnly = false',
    11: and returns a wrapped default python binary (see below).
    12: 
    13: We should ask whether we should use 'mkBin' from nix-config. There
    14: should be three options:
    15: 
    16:  - No. Use makeWrapper in runCommand.
    17:  - Yes; we can get it from this project (assumes it's in pkgs.nix)
    18:  - Yes; we should fetch and import nix-config in the benchmarks.
    19: 
    20: Each of these should have a separate template in ~/.templates:
    21: 
    22:  - The "no" template should use runCommand and makeWrapper, assuming
    23:    that they're in pkgs.nix. 'dir' should be set, with var name
    24:    'measured'. An example of prefixing PATH should be given, but
    25:    commented-out.
    26:  - The "this project" template should use 'mkBin', assuming that our
    27:    import of pkgs.nix will do. An empty paths list should be included,
    28:    and vars should set 'measured' to 'dir'.
    29:  - The "fetch and import" template should define a nix-config variable
    30:    alongside pkgs, which we take mkBin from. Don't use 'with nix-config'
    31:    since that would override whatever we got from pkgs.nix. The import
    32:    of nix-config should use its defaults; if we cared about nixpkgs
    33:    versions we would be getting stuff from pkgs.nix anyway. The same
    34:    paths and vars as above should be used.
    35: 
    36: We should also populate an asv.conf.json template with reasonable
    37: defaults:
    38: 
    39:  - asv-nix plugin, environment, etc.
    40:  - Project name should be repo name (which trackGit should know)
    41:  - Repo can be chriswarbo.net/git/....git
    42:  - Builders and matrix should be included for 'dir'
    43:  - Installer should import benchmarks/ dir
    44: 
    45: We should ask whether to define a default benchmark which tracks open
    46: issues, defaulting to yes. If yes, this should include a template Python
    47: file in benchmarks/ which looks up 'measured' from the environment and
    48: (if it exists), loops through the contents checking whether each one is
    49: closed (e.g. via a simple search for some header string; whatever
    50: artemis uses).