PERSEUS – personal help for blind users

Description

Considerable part of the population has any sensory or perceptual impairment. Thanks to massive progress of multimedia and permanent increasing of power of small or miniature computer an appropriate assistive technology is accessible for almost everyone who need assistance for his life. We present framework to development of assistive devices mainly using combination of human operator, video stream from the user’s device and acoustic beacon to navigate user.

Research activities

The main goal of the PERSEUS (PERSEUS stands for Personal Help for Blind User) project is to help blind people in serious troubles. Most of the existing assistance devices allow users to recognize object in peri-personal space, but not to have summary about overall environmental context (and solve the problem of loosing orientation in known environment mentioned in the introduction). The major part of available systems are focused to local (i.e. automatic in device, machine based) processing of context information and not deal with the problem of global context primarily. Automatic interpretation of context information however can be incomplete.

Fig. 1: Arrangement of the whole system

PERSEUS is designed as combination of several principles. The main part of the user’s device are acrylate protective glasses, used as a holder for two cameras (see Fig. 3). In the case of emergency user send signal to the navigation centre, where operator can watch stereoscopic video simulating vision of sighted people. Stereoscopic video stream is transmitted to the operator with public Wi-Fi network. Operator see context of environment in 3D (using NVIDIA 3D vision system) so can estimate distance of obstacles. For navigation of user acoustic beacon is used. Operator has the joystick and through the use of the simple protocol is handling choice of audio sample in the memory of user wearable PC. Artificially generated audio samples simulate virtual source of  audio signal, which navigate direction of user’s walk.

In the very first prototype are used miniature security cameras with analog video output (black and white composite video) as an image sensors.  Although it is very cheap technical solution, it brings couple of disadvantages: it is necessary to use video grabber in the system  and overall characteristics (i.e. noise parameters, sensitivity and mechanic design) are matching the price range indeed. Since it is real low-cost and for example CMOS image sensors are careless fixed inside case, applicability of this kind of cameras is very poor especially in applications where accuracy of imaging system is necessary.

Since the cameras are not equipped with any lens, simple pinhole is used instead it. The benefits are wide field of view and overall simplicity (low weight of the system an inconsiderable parameter indeed), disadvantage is huge distortion of the image, so the calibration is required. After calibration part of field of view is lost, so previously mentioned benefit is less outstanding.

Fig. 2: Arrangement of user’s device

For the transmission of the stereoscopic video stream were developed custom DirectShow filter. This filter is virtual capture source with double width of captured image: left side of image corresponds to image from the left camera and right side of image corresponds to image from right camera (see Fig. 4). It means that on one moment is known where is left and where is right and no additional synchronization is not needed. The frame rate of this solution is limited by USB bandwidth – is between 12 and 15 frames per second while size of images is 640×480 pixels in true colors (it means 24 bits per pixel). JPEG compression with high quality (low compression) is used in the basic setup.

For the cameras readout is used OpenCV, the open-source and multiplatform library of C/C++ of Python functions and templates for the real time image processing and computer vision. This library is used for the calibration of cameras also and for other image processing as well.

Fig. 3: User’s device: glasses with cameras

People

Stanislav Vítek, Miloš Klíma