Introduction
NASA’s Apollo program relied on a massive team of dedicated scientists, engineers, and specialists working seamlessly together in a cohesive manner to accomplish one of mankind’s greatest technological achievements in history. This entire effort relied exclusively on voice communications between all participants – many of which were not all located physically together. To date, no attempt has ever been made to explore the intricate communication characteristics of problem solving on the scale as complex as going to the moon. Over the past 6 years, the Center for Robust Speech Systems (CRSS) at the University of Texas at Dallas, under an NSF funded project (Hansen, PI; Sangwan; Co-PI, UTDallas; Oard, Co-PI, Univ. Maryland) explored a solution for both analysis and diarization, as well as linking of audio content for the Apollo program. As part of the development of speech and language technologies and algorithms for this NSF (USA) sponsored project, outreach to the speech processing community was a core goal. CRSS-UTDallas have made the entire audio corpus publicly available (to recognize the 50th Anniversary of Apollo-11 Moon Landing in July 2019) with the intent to help advance the Speech and Language technologies on this challenging, yet important, data domain.
Figure shows the poster for Fearless Steps Challenge
The Fearless Steps Initiative by UTDallas-CRSS led to the digitization, recovery, and diarization of 19,000 hours of original analog audio data, as well as the development of algorithms to extract meaningful information from this multichannel naturalistic data resource. As an initial step to motivate a stream-lined and collaborative effort from the speech and language community, UTDallas-CRSS is hosting a series of progressively complex tasks to promote advanced research on naturalistic “Big Data” corpora. This began with ISCA INTERSPEECH-2019: "The FEARLESS STEPS Challenge: Massive Naturalistic Audio (FS-#1)". This first edition of this challenge encouraged the development of core unsupervised/semi-supervised speech and language systems for single-channel data with low resource availability, serving as the “First Step” towards extracting high-level information from such massive unlabeled corpora.This was followed with ISCA INTERSPEECH-2020 which held the Special Session for FEARLESS STEPS Challenge (FS-2), which focused on developing supervised learning strategies for the 100 hour Challenge Corpus. Further, the FEARLESS STEPS Challenge Phase-#3 (FSC P3) focuses on development of single-channel supervised learning strategies with an aim to test system generalizability to varying channel and mission data. FSC P3 also provides an additional challenge task of Conversational Analysis, motivating researchers to work on natural language understanding and group dynamics analysis. The results of the challenge was presented at ISCA INTERSPEECH-2021
In an effort to foster research directions on naturalistic data, the proposed Fearless Steps Challenge Phase-04 will include 10 additional hours of manually transcribed and previously unseen Apollo-8 mission data. Additional tasks of Speaker Verification (SV) and Topic Detection will also be available to FSC participants. Researchers will be able to participate in Six Core Speech and Language Technology tasks, with sub-tasks for each of the tasks (except SAD, SID, and SV) offered. In addition, one oral session time slot will be dedicated to having all top-5 performers serve as a Panel for a Q&A session from the audience. We have reached out to the speech and language community, and anticipate at least 50 teams participating. The 25,000-hour corpora of Apollo-1, 8, 11, and 13 data will also be released along with the pipeline diarization transcripts for the multi-channel time-synchronized data
Challenge Tasks in Phase-4 (FSC P4)
- TASK#1: Speech Activity Detection: SAD
- TASK#2: Speaker Diarization: SD
- 2a. Track 1: Diarization using system SAD
- 2b. Track 2: Diarization using reference SAD
- TASK#3: Speaker Identification: SID
- TASK#4: Speaker Verification: SV
- TASK#5: Automatic Speech Recognition: ASR
- 5a. Track 1: ASR using reference SAD
- 5b. Track 2: ASR using system SAD
- TASK#6: Topic Detection
- 6a. Track 1: Topic Detection using diarized segments
- 6b. Track 2: Topic Segment using identification from audio streams
Announcements
- Information about each task has been added. Check the appropriate tabs for more information!!
19k Corpus Access
The Entire Fearless Steps Corpus consisting of over 19,000 hours of audio from the Apollo-11 Mission is publicly available under the 'NASA Media Usage Guidelines'. For access to the complete 19,000 hours corpus, Please fill the form given in the link: Fearless Steps Challenge Phase-01. or directly contact us at FearlessSteps@utdallas.edu.
References
Below are some of the references you can use
[2014] Speech Activity Detection for NASA Apollo Space Missions: Challenges and Solutions [abstract] [paper] [citation]
[2017] I-vector/plda speaker recognition using support vectors with discriminant analysis [abstract] [paper] [citation]
[2018] Fearless Steps: Apollo-11 Corpus Advancements for Speech Technologies from Earth to the Moon [abstract] [paper] [citation]
[2019] The 2019 Inaugural Fearless Steps Challenge: A Giant Leap for Naturalistic Audio [abstract] [paper] [citation]
[2020] FEARLESS STEPS Challenge (FS-2): Supervised Learning with Massive Naturalistic Apollo Data [abstract] [paper] [citation]
Additional Information
The Fearless Steps corpus is derived fom a five-year NSF CISE funded project awarded to CRSS at the University of Texas at Dallas. UTDallas-CRSS established the hardware/software solutions to digitize and diarize 19,000 hrs of NASA Apollo data. All core Apollo data released as part of this challenge has been approved for public release by NASA Export Control. The full audio corpus is also available through UTDallas-CRSS. Any reference to or listing of organizations other than UTD is for information only; it does not imply recommendation or endorsement by UTDallas-CRSS nor does it imply that the products mentioned are necessarily the best available for that purpose.
Licensing
Details
All the conversations between Astronauts and and Mission Control Personnel during the Apollo-11 Mission were recorded by NASA. The tireless efforts of CRSS-UTD transcribers and researchers contributed to the shaping of this enormous amounts of data into a well-defined corpus to address various speech and language tasks for naturalistic audio, a portion of which is now made publicly available to the speech community through this Challenge via a creative commons license.
Note:The Creative Commons License is restricted to the efforts made by CRSS-UTD, which involves 100 hours of Challenge Corpus (audio) data sampled from 8Khz, along with its meta-data generated separately. The license also covers all the scripts which were used in the preparation of the corpus and systems built to support the tasks in this Challenge, along with the webpages developed to host the Challenge.
NASA Media Usage Guidelines
NASA content - images, audio, video, and computer files used in the rendition of 3-dimensional models, such as texture maps and polygon data in any format - generally are not copyrighted. You may use this material for educational or informational purposes, including photo collections, textbooks, public exhibits, computer graphical simulations and Internet Web pages. This general permission extends to personal Web pages.
News outlets, schools, and text-book authors may use NASA content without needing explicit permission. NASA content used in a factual manner that does not imply endorsement may be used without needing explicit permission. NASA should be acknowledged as the source of the material. NASA occasionally uses copyrighted material by permission on its website. Those images will be marked copyright with the name of the copyright holder. NASA's use does not convey any rights to others to use the same material. Those wishing to use copyrighted material must contact the copyright holder directly.
Creative Commons License
FEARLESS STEPS CHALLENGEbyAditya Joglekar, John H.L. Hansenis licensed under aCreative Commons Attribution 4.0 International License
Based on a work athttps://www.nasa.gov/mission_pages/apollo/apollo-11.html
Permissions beyond the scope of this license may be available athttps://www.nasa.gov/multimedia/guidelines/index.html
For Additional Information regarding Commercial and Non-Commercial Use:
Please visit: https://www.nasa.gov/multimedia/guidelines/index.html
Organizers
- John H.L. Hansen, University of Texas at Dallas
- Christopher Cieri, University of Pennsylvania
- Omid Sadjadi, NIST
- Aditya Joglekar, University of Texas at Dallas
- Meena Chandra Shekar, University of Texas at Dallas
- Abhijeet Sangwan, University of Texas at Dallas
Acknowledgements
This project was supported in part by AFRL under contractFA8750-15-1-0205, NSF-CISE Project 1219130, and partially by the University of Texas at Dallas from the DistinguishedUniversity Chair in Telecommunications Engineering held by J.H. L. Hansen. We would also like to thank Tatiana Korelsky and the National Science Foundation (NSF) for their support on this scientific and historical project. A special Thanks to Katelyn Foxworth for leading the ground-truth development efforts for the FS-02 and FS-03 Challenge Corpus.