Case Watchmen

Road Safety Tool

Doubletapp
3 min readAug 21, 2023

About the Project

In most car-sharing services, what happens inside the rented car is beyond any control. Therefore, a registered user can allow another person, who may not have a valid driver’s license, may be intoxicated, or underage, to drive the car, turning the city into a dangerous racing track, posing a threat to the lives and well-being of all its residents. Such incidents have already been recorded, and to prevent their recurrence, it was necessary to develop a tool capable of monitoring drivers’ behavior in car-sharing vehicles.

Task

The Watchmen project was conceived and created in collaboration with DANNIE, which was responsible for the hardware part of the solution, while the Doubletapp team developed the software part of the device.

What does it look like? There is a device with two cameras that is installed on the windshield. It is placed inside the car, with one camera facing the interior and the other facing the exterior; the device operates on Android.

One of the client’s requirements was to have on-device functionality, as streaming video from multiple cars would be costly, and processing would require an increase in server capacities. This challenge was solved — all heavy computations are performed directly on the device, following the Edge AI approach.

Result

Watchmen has the following functionality:

  • Detection of faces inside the car.
  • Decision-making about who the driver is.
  • Matching the driver’s face with the photo uploaded by the car-sharing service user.
  • Using computer vision (without the need for additional sensors other than the camera) to detect smoking inside the car.
  • Transferring video from the camera to the server when system operation needs to be checked.
  • A web interface for demo mode — to promote the solution to car-sharing services. It provides real-time streaming from the device’s camera, and real-time detection of smoking and driver substitution.
  • Backend and administrative panel for working with drivers’ data.
  • All the described detection and recognition tasks are performed offline on the device.

In terms of development, the most interesting task was the implementation of cigarette detection. For this purpose, we collected a dataset of thousands of photos of people smoking in cars, as well as performing actions similar to smoking, such as scratching the nose or coughing into a fist. Even the company’s employees contributed by filming themselves smoking in their cars. Based on this dataset, a neural network was trained and subsequently ported to the device for offline operation.

Used technologies

ML: Tensorflow
Backend: Django Python
Android: Kotlin

Project team

Android: Anton Riabykh, Kirill Shcherbakov
Frontend: Stepan Panov
Backend: Maxim Shirokov
ML: Daniil Semenov

Customer feedback

At DANNIE, we often outsource tasks related to software apps since we specialize more in HW development and programming at a more hardware-oriented level. In the Watchmen project, we had tight deadlines with changing requirements from the client. However, the Doubletapp specialists found an optimal format of interaction and helped us meet the client’s requirements within the deadline.

--

--