Saurav Singh

Saurav Singh

Robotics and AI Researcher

Publications

You can also find my articles on my Google Scholar profile.

Regulating Modality Utilization within Multimodal Fusion Networks

Published in Sensors 2024, 24(18), 6054, Special Issue: Deep Learning Methods for Aerial Imagery, 2024

This paper proposes a modality utilization-based training method for multimodal fusion networks, ensuring balanced integration of diverse data in aerial imagery tasks while improving noise robustness and enhancing performance in noisy conditions.

Recommended citation: S. Singh, E. Saber, P. P. Markopoulos, and J. Heard, “Regulating Modality Utilization within Multimodal Fusion Networks,” Sensors, vol. 24, no. 18, p. 6054, 2024.
Download Paper

Measuring State Utilization During Decision Making in Human-Robot Teams

Published in Companion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction, 2024

This study introduces a state utilization (SU) metric to assess how reinforcement-based agents incorporate human internal states into decision-making, enhancing the explainability of human-robot teaming using data from the Cartpole and NASA MATB-II environments.

Recommended citation: S. Singh and J. Heard, “Measuring state utilization during decision making in human-robot teams,” in Companion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction, 2024, pp. 985–989.
Download Paper | Download Slides

Spatial and Temporal Attention-based emotion estimation on HRI-AVC dataset

Published in 2023 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2023

This paper introduces the HRI-AVC dataset, featuring self-reported emotional measures from human-robot interactions, and proposes an attention-based network for estimating human emotion through arousal and valence from image frames in real-time.

Recommended citation: K. Subramanian, S. Singh, J. Namba, J. Heard, C. Kanan, and F. Sahin, "Spatial and Temporal Attention-Based Emotion Estimation on HRI-AVC Dataset," 2023 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Honolulu, Oahu, HI, USA, 2023, pp. 4895-4900, doi: 10.1109/SMC53992.2023.10394066.
Download Paper

Probabilistic Policy Blending for Shared Autonomy using Deep Reinforcement Learning

Published in 2023 32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), 2023

This work introduces a probabilistic policy blending approach for shared control between a human operator and an intelligent agent, demonstrating effective assistance levels using the Lunar Lander game and analyzing the correlation between human physiological data, arbitration level, and task performance.

Recommended citation: S. Singh and J. Heard, “Probabilistic Policy Blending for Shared Autonomy using Deep Reinforcement Learning,” 2023 32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), Busan, Korea, Republic of, 2023, pp. 1537-1544.
Download Paper | Download Slides

Multimodal aerial view object classification with disjoint unimodal feature extraction and fully-connected-layer fusion

Published in Big Data V: Learning, Analytics, and Applications, SPIE, 2023, 2023

This paper presents a two-phase multi-stream fusion approach for training networks with limited multimodal data, addressing the challenges of collecting paired data and demonstrating improved performance on the NTIRE-21 dataset.

Recommended citation: S. Singh, M. Sharma, J. Heard, J. D. Lew, E. Saber, and P. P. Markopoulos, “Multimodal aerial view object classification with disjoint unimodal feature extraction and fully-connected-layer fusion,” in Big Data V: Learning, Analytics, and Applications, vol. 12522, p. 1252206, SPIE, 2023.
Download Paper | Download Slides

Measuring Modality Utilization in Multi-Modal Neural Networks

Published in 2023 IEEE Conference on Artificial Intelligence (CAI), 2023

This paper introduces a novel modality utilization metric to quantify neural network reliance on different modalities, enhancing the explainability of multimodal data fusion, validated on NTIRE-21 and MCubeS datasets.

Recommended citation: S. Singh, P. P. Markopoulos, E. Saber, J. D. Lew and J. Heard, "Measuring Modality Utilization in Multi-Modal Neural Networks," 2023 IEEE Conference on Artificial Intelligence (CAI), Santa Clara, CA, USA, 2023, pp. 11-14.
Download Paper | Download Slides

Understanding Differences in Human-Robot Teaming Dynamics between Deaf/Hard of Hearing and Hearing Individuals

Published in Companion of the 2023 ACM/IEEE International Conference on Human-Robot Interaction, 2023

This study examines the interactive differences between hearing and deaf or hard of hearing individuals with collaborative robots to improve inclusiveness and effectiveness in human-cobot interactions.

Recommended citation: A. Dust, C. Gonzalez-Lebron, S. Connell, S. Singh, R. Bailey, C. O. Alm, and J. Heard, “Understanding differences in human-robot teaming dynamics between deaf/hard of hearing and hearing individuals,” in Companion of the 2023 ACM/IEEE International Conference on HumanRobot Interaction, pp. 552–556, 2023.
Download Paper

A Human-Aware Decision Making System for Human-Robot Teams

Published in 2022 17th Annual System of Systems Engineering Conference (SOSE), 2022

This paper presents a human-aware system integrating human internal states such as workload into robot decision-making using reinforcement learning, validated on NASA MATB-II data to enhance human-robot team dynamics and performance.

Recommended citation: S. Singh and J. Heard, “A Human-Aware Decision Making System for Human-Robot Teams,” 2022 17th Annual System of Systems Engineering Conference (SOSE), 2022, pp. 268-273.
Download Paper | Download Slides

Diagnostic Human Fatigue Classification using Wearable Sensors for Intelligent Systems

Published in 2022 17th Annual System of Systems Engineering Conference (SOSE), 2022

This paper presents a multimodal approach to classify mental and physical fatigue for adaptive intelligent systems, aiming to optimize team performance by adapting to human fatigue levels.

Recommended citation: L. Nagahanumaiah, S. Singh and J. Heard, “Diagnostic Human Fatigue Classification using Wearable Sensors for Intelligent Systems,” 2022 17th Annual System of Systems Engineering Conference (SOSE), 2022, pp. 424-429.
Download Paper

Human-Aware Reinforcement Learning for Adaptive Human Robot Teaming

Published in 2022 17th ACM/IEEE International Conference on Human-Robot Interaction (HRI), 2022

This paper presents a reinforcement learning-based approach that leverages human workload states to adapt robot interactions, aiming to improve team performance in high-stress, multitasking environments like the NASA MATB-II task.

Recommended citation: S. Singh and J. Heard, “Human-aware reinforcement learning for adaptive human robot teaming,” in Proceedings of the 2022 ACM/IEEE International Conference on Human-Robot Interaction, ser. HRI ’22. IEEE Press, 2022, p. 1049–1052.
Download Paper | Download Slides

Push Recovery for Humanoid Robots using Linearized Double Inverted Pendulum

Published in ProQuest, 2020

This paper presents a novel balance control scheme using a Linearized Double Inverted Pendulum model to enhance a humanoid robots recovery from external disturbances, validated on the simulated TigerBot-VII.

Recommended citation: S. Singh, “Push Recovery for Humanoid Robots using Linearized Double Inverted Pendulum,” Research Master Thesis, Rochester Institute of Technology, Rochester, NY, 2020.
Download Paper

Electronic Guitar MIDI Controller for Various Musical Instruments Using Charlieplexing Method

Published in Innovations in Computer Science and Engineering: Proceedings of the Sixth ICICSE 2018, 2019

This paper introduces an electronic guitar that enables musicians to play various instrument sounds using guitar notes, utilizing Charlieplexing and computer connectivity for a versatile and silent practice experience.

Recommended citation: R. Devasia, A. Gupta, S. Sharma, S. Singh, and N. Rathee, “Electronic guitar midi controller for various musical instruments using charlieplexing method,” in Innovations in Computer Science and Engineering: Proceedings of the Sixth ICICSE 2018, pp. 315–325, Springer, 2019.
Download Paper

Digital resistance box: An approach to generate desired value of resistance by automatically varying the potentiometer

Published in 2016 IEEE 1st International Conference on Power Electronics, Intelligent Control and Energy Systems (ICPEICES), 2016

This publication introduces a digital resistance box using a microcontroller-controlled servo motor and binary search algorithm to achieve precise resistance values entered via a digital keypad, eliminating the manual tuning of potentiometers and ensuring accurate resistance selection.

Recommended citation: N. Rathee, A. Gupta, S. Singh, R. Devasia, and A. Bansal, “Digital resistance box: An approach to generate desired value of resistance by automatically varying the potentiometer,” in 2016 IEEE 1st International Conference on Power Electronics, Intelligent Control and Energy Systems (ICPEICES), pp. 1–4, IEEE, 2016.
Download Paper