Case Watchmen
Road Safety Tool
About the Project
In most car-sharing services, what happens inside the rented car is beyond any control. Therefore, a registered user can allow another person, who may not have a valid driver’s license, may be intoxicated, or underage, to drive the car, turning the city into a dangerous racing track, posing a threat to the lives and well-being of all its residents. Such incidents have already been recorded, and to prevent their recurrence, it was necessary to develop a tool capable of monitoring drivers’ behavior in car-sharing vehicles.
Task
The Watchmen project was conceived and created in collaboration with DANNIE, which was responsible for the hardware part of the solution, while the Doubletapp team developed the software part of the device.
What does it look like? There is a device with two cameras that is installed on the windshield. It is placed inside the car, with one camera facing the interior and the other facing the exterior; the device operates on Android.
One of the client’s requirements was to have on-device functionality, as streaming video from multiple cars would be costly, and processing would require an increase in server capacities. This challenge was solved — all heavy computations are performed directly on the device, following the Edge AI approach.
Result
Watchmen has the following functionality:
- Detection of faces inside the car.
- Decision-making about who the driver is.
- Matching the driver’s face with the photo uploaded by the car-sharing service user.
- Using computer vision (without the need for additional sensors other than the camera) to detect smoking inside the car.
- Transferring video from the camera to the server when system operation needs to be checked.
- A web interface for demo mode — to promote the solution to car-sharing services. It provides real-time streaming from the device’s camera, and real-time detection of smoking and driver substitution.
- Backend and administrative panel for working with drivers’ data.
- All the described detection and recognition tasks are performed offline on the device.
In terms of development, the most interesting task was the implementation of cigarette detection. For this purpose, we collected a dataset of thousands of photos of people smoking in cars, as well as performing actions similar to smoking, such as scratching the nose or coughing into a fist. Even the company’s employees contributed by filming themselves smoking in their cars. Based on this dataset, a neural network was trained and subsequently ported to the device for offline operation.
Used technologies
ML: Tensorflow
Backend: Django Python
Android: Kotlin
Project team
Android: Anton Riabykh, Kirill Shcherbakov
Frontend: Stepan Panov
Backend: Maxim Shirokov
ML: Daniil Semenov
Customer feedback
At DANNIE, we often outsource tasks related to software apps since we specialize more in HW development and programming at a more hardware-oriented level. In the Watchmen project, we had tight deadlines with changing requirements from the client. However, the Doubletapp specialists found an optimal format of interaction and helped us meet the client’s requirements within the deadline.
Read our other stories:
- Case Adventure Aide
- Neural Network Optimization: Ocean in a Drop
- Case Elixir Gallery
- Forgive us, John Connor, or How We Taught a Neural Network to Accurately Recognize Gunshots
- Case Bus Factor
- CI/CD for iOS-projects: device or cloud? What’s better, Doubletapp’s take
- How to set up Gitlab CI/CD with Fastlane for iOS-project on a Mac mini
- The story about the contest from telegram
- What should be done before starting the site? Checklist from Doubletapp
- How to Generate PDF Documents in Python
- How to manage Gradle dependencies in an Android project properly
- The History of the Doubletapp Company
