Alex Xu V2 - 2 Nearby Friends


Clarifying questions I have before seeing the problem

What’s the max radius that friends can see other friends? Should it notify them if a friend is nearby? The user can open the app and see their friends?

Can people send friend requests to other people?

Can they see the exact location of their friends?

Step 1 – Clarify

How many users does the app have?

Can we assume distance is calculated in a straight line between two users? Like if there’s a river, that could technically add travel distance time.

Do we need to store location history?

Inactive friends for 10 minutes will disappear? Or do we display last known locations?

Step 2 - Requirements

Functional Requirements

Non-Functional Requirements

Low latency - Reliable - reliable overall but Occasional data point loss is okay Eventual Consistency (Not Strong) - Seconds delay from receiving location data from replicas is okay

Back of envelope estimations

1 B users 10% use the feature 100M users for this feature

100M * Query = 100M Q per day 100,000 seconds in a day 100,000,000 / 100,000 -> 100 100 Queries per second


100M people’s locations Long / Lat -> 8 Bytes 2,000 different locations on average?

100M * 8 * 200? -> 800M bytes * 2,000

Step 2 -> High Level Design

Problem requires efficient message passing

![[Screenshot 2024-02-09 at 7.20.48 AM.png]]

Receive Updates Find all active friends and fan out Is distance for friend is too high, don’t fan out to them

Cons - Doesn’t work at high volume

10M users that are updated every 30 seconds 334k QPS At 400 friends (10% of them active) that’s 14M Updates per second 14M writes to a database every second is a lot

Proposed Design

Mobile users use web socket and http

Load balancer

Send into Web Socket Servers (Bidirectional location info)

Send to API servers (Auth, User Management, Friend Management)

![[Screenshot 2024-02-09 at 7.36.09 AM.png]] ![[Pasted image 20240503125447.png]]

How well does each component scale?

API Servers
WebSocket servers
Client Initialization
  1. Update location in cache
  2. Save location for WS
  3. load all user’s friends from database based on user_id
  4. Makes batched request to the location to fetch X friends
  5. Computes distance and see if it’s at the maximum
  6. Send to the client if it’s good
  7. Subscribes user_id to the friend’s channel
  8. Sends current location to user’s channel in Redis Pub/Sub server
User Database

Location Cache

How many Redis Pub/Sub Servers do we need?

At 100M users, each with 20B for 100 friends

CPU Load

How do we Autoscale these servers

Whale Users with lots of friends

Spread them out over multiple web socket connections

Follow up -> Show Random People who opt-in to sharing location

Add pool of Pub/Sub channels by geohash. Anyone who is located in the geohash will subscribe to the random people