A Global Correction Framework for Camera Registration in Video See-Through Augmented Reality Systems

Wenhao Yang, Yunbo Zhang

Journal of Computing and Information Science in Engineering, 2023

“I can see your password”: A case study about cybersecurity risks in mid-air interactions of Mixed reality-based smart manufacturing applications

Wenhao Yang, Xiwen Dengxiong, Xueting Wang, Yidan Hu, & Yunbo Zhang

Journal of Computing and Information Science in Engineering, 2023

HAR²bot: System Design of a Human-centered Augmented Reality Robot Programming Interface with Cognition Awareness

Wenhao Yang, Qinqin Xiao, & Yunbo Zhang

Journal of Intelligent Manufacturing, 2023

Visualization Error Analysis for Augmented Reality Stereo Video SeeThrough Head-Mounted Displays in Industry 4.0 Applications

Wenhao Yang, Yunbo Zhang

International Manufacturing Science and Engineering Conference, 2022

An Augmented-Reality Based Human-Robot Interface for Robotics Programming in the Complex Environment

Wenhao Yang, Qinqin Xiao, & Yunbo Zhang

International Manufacturing Science and Engineering Conference, 2021

A Digital Twin Framework for human-robot collaboration

Mar. 2023 – Present

Developed an innovative and comprehensive Digital Twin framework that facilitate seamless and effective collaboration between humans and robots. And a Human-Robot Collaboration (HRC) assembly use case is implemented. The Augmented reality technology is leveraged to enhance the user experience and enable robot programming by demonstration with the digital twin.

A MR-based programming by demonstration for pick-and-place tasks

Jan. 2023 – Present

Built a system to demonstrate the task to the robot by virtually performing it while wearing a MR headset. The system identifies the user's attention and uses machine learning algorithms to generate the robot's program. This approach offers a promising solution for programming pick-and-place tasks and has the potential to improve the efficiency and scalability of industrial automation.

3D Model Reconstruction from UAV's Mono RGB Camera: Building Virtual Metaverse Environments

Jun. 2023 – Aug. 2023

Our project focuses on the development of a cutting-edge 3D model reconstruction method utilizing video and flight record data from an Unmanned Aerial Vehicle (UAV). The primary objective is to rapidly create virtual 3D environments, contributing to the emerging metaverse technology.

A mobile manipulator for telepresence

Sep. 2022 – Present

Designed a mobile manipulator system capable of performing remote tasks through human interaction. This system enables the perception of the remote environment and control of the robot arm through Augmented Reality implemented system.

Cybersecurity risks in Mixed Reality-based smart manufacturing applications

Sep. 2022 – Nov. 2022

This work discusses a potential cybersecurity risk in MR-based smart manufacturing applications where digital passwords can be deciphered through mid-air gestures captured by a single RGB camera. The study develops a machine learning-based hand gesture recognition method with high accuracy to estimate passwords input.

https://youtu.be/Erm2c_CijZY

A Tele-robot System with Mixed-Reality Interaction

Jun. 2022 – Sep. 2022

This study describes the development of a tele-robot control system that utilizes a Mixed Reality headset, (HoloLens 2), to remotely control Automated Guided Vehicles (Rover Pro). The system features a user-friendly interface that offers three control methods: motion control, physical button control, and virtual button control.

A global calibration to solve the misregistration problems in VST AR system

Oct. 2021 – Feb.2023

The purpose of this study is to introduce a calibration method for the registration problem in a video seethrough Augmented Reality (AR) system. This study investigates error sources and presents an efficient calibration procedure to reduce the misalignment accuracy via HMD-to-Camera transformation.

Augmented reality assisted smart factory management system development

Dec. 2021 – Feb. 2022

An AR headset-based prototyping system has been developed to offer a clear and intuitive digital visualization for the purpose of exploring the potential applications of smart factories. This project presents a case study of a maintenance application in a CNC machine shop for technicians.

https://youtu.be/p6v-WwOH-og

Augmented reality assisted maintenance and repair system development

Dec. 2021 – Feb. 2022

This project demonstrates the potential of utilizing digital twin technology with an Augmented Reality interface for improved maintenance procedures. By presenting a demo system of a repair process, this work aims to highlight the benefits of using AR for clear and intuitive visualization of the device being repaired, resulting in improved accuracy and ease of instruction for technicians.

https://youtu.be/NfVmJfu_26A

An application of depth map completion and super-resolution in AR rendering

Sep. 2021 – Nov. 2021

This project presents an experiment on an AR-based human-robot interaction scenario, aimed at demonstrating the effectiveness of a proposed method for addressing occlusion problems in the rendering of virtual objects. The method involves the use of depth completion and super-resolution techniques to enhance the visual quality.

Visualization error analysis for augmented reality stereo VST system

May. 2021 – Nov. 2021

In this research study, the sources of registration errors were identified in stereo video see-through systems and a mathematical model was established to describe the propagation of these errors in the display process.

System design of a human-centered AR HRI with cognition awareness

Nov. 2019 – Oct. 2021

This work proposed a novel system, HAR2bot, for path planning tasks in complex environments. The proposed system effectively balances the communication between the operator and the robot to minimize the cognitive burden associated with the design loop, leading to improved task efficiency and accuracy.

https://youtu.be/JPeubcl6sQs

Augmented reality-based human-robot interaction system

Aug. 2019 – Nov. 2020

This project proposed and developed an intuitive and immersive AR-based interface for cobot (AUBO i5) programming, aiming to address the challenges in performing path planning tasks in complex environments.

https://youtu.be/PO4EoN_ix-Y

Course development: Introduction to industrial automation and Robots

Aug. 2023

This course offers college students a comprehensive introduction to the exciting and rapidly evolving field of industrial automation and robotics. In today's technology-driven world, automation plays a crucial role in enhancing productivity and efficiency across various industries. The course aims to equip students with fundamental knowledge and practical insights into automation systems and robotics, enabling them to understand, design, and implement automation solutions.