Fearless Steps Challenge:
Phase III

Overview

NIST maintained web platform OpenSAT will conduct all evaluation activities. Each participant will register by creating an account on this web platform. This will allow them to perform various activities such as registering for the evaluation and uploading submissions.

After registering and agreeing to the NIST FSC-P3 Terms and Conditions, participants will be able to participate in the Fearless Steps Challenge: Phase 3 (FSC P3). This page contains step-by-step instructions for creating the evaluation account, joining a site and team, selecting tasks, and signing the relevant agreements.

Set up an Account

To sign up for an evaluation account, Please head to OpenSAT Series FSC P3 website and follow the steps below:

  1. Click the “Sign Up” button at the top right corner of the page.
  2. Provide a valid email address and create a password, then click “Sign up”. A confirmation email will then be sent to the provided email address.
  3. Click the “Confirm my account” link in the confirmation email. This will take you to a login page. Log in with our email address and password and click “Log in”. This will take you to the evaluation dashboard, which will display the registration steps.
  4. Please complete the steps in this dashboard to complete account creation and registration.

Create an account on site

After the evaluation account is confirmed, the participant will be asked to join a site (or create one if it does not exist). The participant is also asked to associate their site to a team or to create a team if one does not exist. This allows multiple members to perform activities on behalf of their site and/or team (e.g., make a submission). Clarifying the distinction between participants, sites, and teams:

  • site – a single organization (e.g., NIST)

  • team – a group of organizations collaborating on a task (e.g., Team1 consisting of NIST and LDC)

  • participant – a member or representative of a site who takes part in the evaluation (e.g., John Hansen)

Create/Join a site

From the “Registration Workflow” panel click “Create/Join a site” to create or join a site. This will bring up a list of all sites that you are a member of. To join an existing site, click “Join a Site”, then:

  1. Select a site from the “Choose a Site” dropdown menu
  2. Enter the site password (you will need to obtain this from the site owner)
  3. Click “Join this Site”
  4. To create a new site, click “Create New Site”, then:
    • Enter a site name
    • Enter a password for the site
    • Click “Create and join Site”

Register your team

To create or join a team, from the “Registration Workflow” panel click “Create/Join a team”. This will bring up a list of all teams that you are a member of.

Create your Team

  1. To create a new team, click “Create New Team”
  2. Enter a team name
  3. Choose one of the teams that you are registered with from “Choose one of the site(s) you own” dropdown menu
  4. Enter a password for the team
  5. Select whether or not you want the team name to be annonymized on the leaderboard during the challenge (you can change this later)
  6. click “Create Team”

Join Existing Team

To join an existing team, click “Join a Team”, then:

  1. Select the site from the “Choose one of the site(s) you own”dropdown menu
  2. Choose the team you wish to join from the “Choose a Team” dropdown menu
  3. Enter the team password (you will need to obtain this from the team owner)
  4. Click “Join this team”

Select task

To register for FSC P3 Challenge in the “Registration Workflow” panel, click “Select tasks”. This will bring up a list of available tasks in the system. These tasks are displayed as a list of TEAM/TASK pairs, each with a checkbox to it’s left; e.g., if your team name is TEAM:

  • TEAM - FSC_P3_SAD (evaluation)
  • TEAM - FSC_P3_SID (evaluation)
  • TEAM - FSC_P3_SD_TRACK1 (evaluation)
  • TEAM - FSC_P3_SD_TRACK2 (evaluation)
  • TEAM - FSC_P3_ASR_TRACK1 (evaluation)
  • TEAM - FSC_P3_ASR_TRACK2 (evaluation)
  • TEAM - FSC_P3_CONV_TRACK1 (evaluation)
  • TEAM - FSC_P3_CONV_TRACK2 (evaluation)

To register for a certain task, check the box to the left of “FSC_P3_Task_Name”. When you are finished, click the “Register” button.

Terms and Conditions

In order to complete your registration and gain access to the data, you will need to accept the FSC P3 terms and conditions and sign and upload the LDC data license agreement. To complete these tasks, from the “Registration Workflow” panel, click “Sign and upload license”, which will take you to a listing of all available agreements and their current status (i.e., signed/not signed, approved/not approved).

To agree to the FSC P3 terms and conditions locate the entry named “FSC P3 terms and conditions” and click “Show”. This will bring up the text of the terms and conditions. To agree to these terms, click the “Review” button located in the bottom left hand corner, check the box to the left of “I acknowledge that I have read and accepted…“, then click “Update License Agreement.”

Fearless Steps Challenge Phase-03 is being held in collaboration with NIST. Please register on this website to receive updates regarding receiving the data and registering on the NIST submission portal.

To participate in the challenge, Please click the button below to register!