Fearless Steps Challenge:
Phase IV

Interspeech Special Session

CRSS-UTDallas held the First Fearless Steps Challenge in 2019. The Challenge was held from February to June 2019, in which 16 organizations competed for the 5 Challenge Tasks, submitting 116 systems. The Second Fearless Steps Challenge (Phase-02) held form February to May 2020 had 17 teams competing with 111 system submissions. For the first two Fearless Steps Challenges released in Phases, the 100-hour audio used for the Challenge remained the same, while the supervised data size and Tasks evolved. The Phase-01 focused on developing unsupervised and semi-supervised systems. Thus, only 20 hours of development data was fully transcribed (in addition to the 20-hour blind evaluation set). As a natural progression, Phase-02 focused on the development of supervised systems, with 60 hours of manually transcribed training set, 20 hours each allocated for development and evaluation sets. Improvements on the Challenge Format proposed in Phase-02 were successful.
For Phase-03, 5 hours of an unseen Apollo-11 channel data was presented to test system robustness towards changes in channel characteristics. 5 additional hours of Apollo-13 mission data was provided to test the system’s ability to generalize performance over entirely separate missions.
We propose to continue this trend for the Phase-04, leveraging the experiences from the first three challenges, while restoring the most successful aspects of Phase-01, Phase-02, and Phase-03 based on community feedback. Due to the overwhelming increase in deep learning systems submissions outperforming the other machine learning strategies for the First Challenge, new and improved DNN based baseline systems will also be provided to the community to motivate robust system development on naturalistic data. The improved addition to Phase-04 of this Challenge are:

• Topic Detection task to evaluate language understanding in naturalistic speech through detection of conversational “hot-spots”


• 10 additional hours of previously unreleased Apollo-8 data for evaluation of inter-mission robustness., which will have multiple factors different from the Apollo-11 Mission data, which the systems will primarily use to train.

The major advantage we have for the Phase-04 of the Challenge are the learnings from the Phase-01, Phase-02, and Phase-03 from which we have been be able to use the existing infrastructure to further improve the Challenge Tasks, Baselines and Evaluation Plan. The Interspeech-2022 Special Session will be a great opportunity to reflect on the evolution of the Challenge and Systems performances in the past 4 years and focus on the promising future directions for extracting knowledge from massive naturalistic audio.