The problem with location history storage
6 min read
There are a ton of applications that constantly monitors and stores its users' location history. This is not an invasion of privacy, but instead part of the core business model. For example, consider an application like Glovo or Uber. Glovo needs to store the location of its glovers and Uber must store the history of the driver and maybe your position too during a trip to protect its users and to even give you that nice page in your app where you can see your previous trips with the cost you paid for each trip.
The thing that most developers might have missed is the complexity that lies underneath a simple but very much needed feature like the location storage. What's so special about location storage you say? Well, the data size.
Let's start with a small reference. Let's consider Shopify, a very successful business. After 15+ years of operation (since 2006), how many merchants, customers, orders, and products do you expect it to have stored in its database? 1,000,000 (1 million) merchants and 218,000,000 (218 million) customers at the time of writing and since there aren't a sure source on the total number of orders and products, we are going to make a wild guess by saying that there are 5 times the number of orders as there are customers and every merchant would have 50 products on average. That said we now have 50,000,000 (50 million) products and around 1,100,000,000 (1.1 billion) orders as a rough estimate for simplicity. This all adds up to 1,369,000,000 (1.369 billion) records. Again, a rough estimate.
Let's now take a look at what happens if you have a fleet of 100 cars operating under your supervision (For reference, Uber has 3.9 million drivers). A regular working day is 8 hours. Therefore, if one driver works 8 hours a day, 24 days every month, and we would like to log his location history every 3 seconds then every month this driver has (60/3) second each minute x 60 minutes in an hour x 8 hours daily x 24 days of the month = 230,400 tracking points Multiply that by 100 cars? 23,040,000 That means that you with only 100 devices, you will reach the same number of records in your database as Shopify in less than 60 months. 60 months might seem like a long time, but your business will not stay steady with only 100 devices. Your company will probably grow every day and expand month-over-month as your users grow with you therefore this number will quickly go down. Not to mention that you still have other data to store like your orders, users, invoices, and a ton of other gears that work this big machine called your company.
At this point in time many question may start flooding your thoughts about scalability. But don't worry, there are a couple of ways around that. The first approach is to build the system your own way, in-house, and delegate that problem to your engineers.
- You have full control over the system
- You fine-tune the solution to exactly your needs
- You lose development time building the system
- Your team has to optimize and maintain the software and make sure it can handle the load
The second approach is to use a third-party which will be responsible for maintaining your data. This is were we come in. Entracked can store your location tracking data and provide you with an API to store, query, and analyze this data. Pros:
- Integration takes hours instead of weeks or months
- No maintenance. Your engineers can focus on your core business needs.
- You need to contact us if we don't support all your needs
Entracked is currently in open beta.
Location tracking data is a technical debt that felt more and more as your business grows. There are multiple ways to solve that problem. At Entracked, we help take the burden off of your team and we always seek to provide you with the best solution. We would love to see your business grow every day.
By Omar Waleed
Founder of Entracked