Haoyuan Zhang

Haoyuan Zhang

Why so serious

Posts by Collection

portfolio

publications

talks

The Face Detection and Replacement Package Design

Published:

source code
demo 1 / demo 2

  1. Utilized human skin color as feature to train GMM model to filter out face region candidates. Combined with edge mask to separate union face regions for better detection.
  2. Implemented PCA to construct Eigen Faces dataset and included the third-part package Face++ to improve detection performance. The final accuracy of detection reached 82.6%.
  3. Implemented image morphing such as TPS and gradient blending to complete face replacement task.

Fraction API Design

Published:

source code
Java provides several types of numbers, but it does not provide fractions, therefore to implement a Fraction API (Application Programmer’s Interface).

Orientation Estimation using Unscented Kalman Filter(UKF)

Published:

source code
demo 1 / demo 2 / demo 3 / demo 4

  1. Succeeded in implementing a Kalman filter to track three-dimensional orientation of a hand-handled camera.
  2. Given IMU sensor readings from gyroscopes and accelerometers, the algorithm will estimate the underlying 3D orientation by learning the appropriate model parameters from ground truth data given by a Vicon motion capture system.
  3. Generated a real-time panoramic image using images captured by the camera and information filtered by the 3D orientation filter.

Machine Learning applied in the modern robot

Published:

Implemented some of the most popular used machine learning algorithms and applied them into the real robot cases.

  1. Object Detection and Geometry distance estimation based on GMM: trained some GMMs based on the image color data, also combined image processing and geometry algorithms to detetct any read barrel in a robot’s view and estimate its real geometry distance from robot. source code
  2. Robot Gesture Recoginition using Hidden Markov Model (HMM): built an algorithm based on the HMM to recognize different robot arm motion gestures. Finally, it was able to classify unknown arm motions within almost real-time. source code
  3. Reinforcement Learning using Policy Gradient Methods: implemented several methods for dealing with the Robot Walking on the Frozen Lake problem. The situation can be represented via a Markov Decision Process(MDP) and a strategy for retrieving the frisbee can be obtained using value iteration (VI), policy iteration (PI), and policy gradient optimization (PGO). source code

SFM and Visual SLAM

Published:

Implemented Visual SLAM package and built 3D trajectory as well as 3D point cloud map.

  1. Structure From Motion(SFM): implemented Triangulation, PnP and Bundle Adjustment manually with Jacobian for camera state and 3D landmarks estimation and optimization. source code
  2. Visual SLAM Package on a Quadrotor: based on SFM structure, succeeded to implement Visual Odometry(VO), applied local BA(Bundle Adjustment) and combined g2O package to increase the SLAM efficiency as well as performance. source code

Faster RCNN Playing with Cifar10 and Cifar100

Published:

source code
Manually implemented Faster RCNN Deep Network (Tensorflow based) to achieve the object detection and classification task, here we used cifar10 data as the target object and cifar100 data as the background.

Generative Models Implementation

Published:

Mastered the algorithm and implementation of most generative models, including AE, VAE and multiple GANs.

  1. Implemented AE(Auto-Encoder) and VAE(Variational AutoEncoder) on people face data, e.g. cufs human faces. source code
  2. Researched on most popular GANs model, including DCGAN, WGAN and WGAN-GP, also wrote a detailed report to discuss their properties. source code
  3. Applied basic GAN model to more complicated problem, Image-to-Image Translation, my implementations included cGAN and cycleGAN.

3D Object Detection and Recognition

Published:

source code
Built a deep network to achieve the 3D-MNIST data classification, 3D bounding box estimation and cube plane depth estimation.

  1. 3D Data (3D MNIST) Pre-Processing, including the Voxel generation and 2D multiple views generation.
  2. Deep network construction which can estimate class label, depth map and 2D bbox in one single stage.
  3. 3D bounding box estimation.

Planning and Optimization

Published:

Implemented two of important robotics required algorithms.

  1. Polynomial Trajectory Planning and Generation: design and generate an optimal polynomial smooth trajectory for the Quad rotor given a specified 3D Map. source code
  2. Hand-handled Camera Calibration: aimed to estimate the intrinsic (K) and extrinsic (R, t) matrices for the camera using a planer world space(e.g. checkerboard) and multiple views, and optimized all variables using the reprojection errors. source code

EKF-Based Visual Inertial Odometry

Published:

source code
demo 1 / demo 2

  1. Within this project, we implemented the Visual Inertial Odometry to estimate the states of a Quadrotor, including its global position [x, y, z], pose [roll, pitch, yaw] and linear velocity with respected to the world [vx, vy, vz].
  2. We achieved both simulation on Linux/ROS and lab environment test on a real Quadrotor tasks.

IPM-based Parking Space Detection

Published:

Project Owner · Camera IPM-based parking space detection that emits 3D coordinates for downstream parking planning.

Perception Model Evaluation Toolboxes

Published:

Core Developer · Evaluation toolboxes covering classification, keypoint detection, keypoint regression, and image-based 3D-box tasks.

AGV Multi-Sensor Fusion

Published:

Project Owner · AGV-level multi-sensor fusion module for static and dynamic obstacle state estimation.

Parking Space Semantic Mapping

Published:

Project Owner · Camera-based semantic mapping for parking environments using line and marker features.

Multi-Device Anchor Map Sharing

Published:

Core Developer · Real-time sharing and localization of anchor maps across multiple devices, enabling multi-player MR games.

teaching

Teaching experience 1

Undergraduate course, University 1, Department, 2014

This is a description of a teaching experience. You can use markdown like any other post.

Teaching experience 2

Workshop, University 1, Department, 2015

This is a description of a teaching experience. You can use markdown like any other post.