Navigate back to the homepage

This app got us first place at a Microsoft hackathon

The application would leverage latest of Azure and Machine Learning to help solve a $6.4 billion infrastructure issue.
Brian Ruiz
· 3 min read
0

The Challenge

The main objective of the competition revolved around developing innovative solutions for improving the infrastructure of cities in the US. The 2019 Smart Infrastructure Hackathon was hosted at The Cannon (Houston), in partnership with Microsoft Azure. Thankfully, we had a team of really bright cloud solution architects for guidance.

Technology we’re using

The technology we opted for the project revolved heavily around Python. For the phone data we used a publicly available mobile app that allowed exporting the data to CSV files. We then leverage Python’s scripting ability to analyze the data from the phone, to use for the road pothole detection algorithm. Python also allowed us to create —matplotlib— visualizations of our data which we used to tweak the algorithm. All of this was tied togther with the Microsoft Azure platform. From storing the data (CSV’s and images) in Azure, to using the Azure Machine Learning platform to train the model.

technology used in project

Our idea and proposal

After juggling several ideas we settled with the issue of road potholes. We discovered that potholes cost American drivers $6.4 billion dollars per year, that is in repair and insurance costs. A single pothole may cause are up to $300/year per vehicle (2016), while the average price to repair a pothole is on average $30 - $50 per pothole. Thus the effort to aiding this issue can be indeed cost-effective.

Looking at our challenge from a high level we need the following components or stages, in order to be able to solve the problem. Also, we need to design for a 2-layer confirmation of a road pothole. The first being the active dashboard camera, and secondly using the user’s phone telemetry data, such as GPS and accelerometer modules.

  • Data collection and analysis

  • Algorithm to predict the probability of a pothole

  • Minimal Interface

  • Data visualization / mapping

1. Analyzing mobile telemetry data

Every modern smartphone is equipped with very accurate telemetry systems. For instance the GPS and accelerometer, which we leveraged for location and to track any abrupt movement indicating a pothole on the road.

1import pandas as pd
2import matplotlib.pyplot as plt
3
4df = pd.read_csv('/home/br/Downloads/testdata.csv')
5df['Time'] = pd.to_datetime(df.Time, format='%Y-%m-%d %H:%M:%S:%f')
6
7columns = [
8 'ACCELEROMETER X (m/s²)',
9 'ACCELEROMETER Y (m/s²)',
10 'ACCELEROMETER Z (m/s²)',
11 'Time'
12]
13
14df[columns].plot(x='Time')

One of the difficulties with this approach was that logging every high acceleration movement may result in false positives of a road pothole. Below is also a time-series data visualization of the accelerometer data, generated with matplotlib.

data

2. Live computer vision

This part was more difficult to implement, but this was our second layer for prediction of potholes, implementation done with some guidance from the Azure cloud architects.

We used Azure Logic Apps to import our road image frames to the cloud, and with OpenCV where able to register a model to start analyzing our content with the conveniently named Azure microservice Computer Vision.

dashcam
Dashcam Footage Example

3. The Interface

From the start the idea was to keep the interface as intuitive as possible to minimize driving distractions. Therefore, we intended to create an interface that would show a dialog box only when our system predicted there to be a pothole, from there the user would confirm if so.

mockup
Mobile Interface Mockup

4. Geocoding / Mapping

Finally we needed to actually plot our GPS data so the city could take action. Our collected data included latitude, longitude, altitude and location accuracy. Location accuracy is important to pinpoint multiple occurrences of the same pothole from different users for clustering the locations.

1import folium
2import pandas as pd
3
4# Initialize map on San Francisco
5SF_COORDINATES = (37.76, -122.45)
6map = folium.Map(location=SF_COORDINATES, zoom_start=12)
7
8# add a marker and cluster nodes
9for each in df.iterrows():
10 map.simple_marker(
11 location = [each[1]['Y'],each[1]['X']],
12 clustered_marker = True
13 )
14
15display(map)
folium map
Geocoding example with Folium

The final result

By the time the hackathon countdown ended we were able to design and implement a viable solution that would achieve our initial idea. Nevertheless, this was achieved thanks to our our strong team collaboration and efforts, and some sprinkled guidance of the MS architecture team.

IMG_20191214_151949

The bumpit team from UHD

After presenting our minimum-viable-product, the judges called a draw for first place. Having won the competition granted us credits to some of Microsoft Azure cloud services like storage and prcessing. And, the company which hosted the competition granted us seats at their co-working space, which we could use to continue development of our application and idea.

ProjectGitHub

More articles from BR Portfolio

Innovating real estate investment & management

Our developments are crucial for managing the company's $80 Billion in assets under management

· 3 min read

How to create a COVID dashboard web application with Python

And, how we attracted thousands of users, and received contributions from developers around the world.

· 4 min read
Link to $https://github.com/BrianRuizyLink to $https://www.linkedin.com/in/brianruizy/Link to $https://www.instagram.com/brianruizy/Link to $mailto:brianruiz@protonmail.com