How Bumble scaled their iOS app, Rippling interview questions and More
🎬 S2:E02 Systems, Scales, Solutions
"The only way to do great work is to love what you do. If you haven't found it yet, keep looking. Don't settle." —Steve Jobs.
Hey, TechFlixers!
Did you know that one of the first computer viruses was created in 1983, named "Elk Cloner"? It targeted Apple II systems, showcasing early concerns about software security and integrity. Fast forward to today, where we explore the complexities of scaling iOS and data exploration, reminding us of the ever-evolving challenges and innovations in tech.
🔦 Spotlight
Scaling iOS at Bumble
Bumble faces significant challenges in scaling its iOS application due to the large size of its development team, the complexity of its codebase, and the limitations of Xcode for managing large projects. These challenges include modularity, build times, and collaboration issues. Here are some of the ways Bumble optimized its iOS workflows.
Swift Package Manager (SPM): Adopt SPM for dependency management to improve modularity and reduce the overhead of managing external libraries and frameworks. SPM integrates well with Xcode, making it easier to manage project dependencies.
Tuist: Use Tuist to generate Xcode projects and workspaces on the fly, simplifying the configuration and maintenance of large Xcode projects. Tuist helps in modularizing the codebase, reducing build times, and improving team collaboration by automating project configuration.
Bazel: Implement Bazel for its advanced build caching and parallel execution features to enhance build speeds significantly. Bazel can handle large-scale projects by efficiently determining which parts of the codebase need to be rebuilt, reducing unnecessary compilation.
Data exploration at Grab
Grab faces challenges in exploring and analyzing streaming data effectively due to the richness and volume of online data it collects. Quick and efficient data exploration is crucial for developing real-time data-driven decision-making. Here’s how Grab tries to democratize data access:
Apache Zeppelin: Leverage Apache Zeppelin for its web-based notebook environment that facilitates data exploration through SQL queries on streaming data. Zeppelin's interactive notebooks allow for real-time data analysis and visualization, making it easier to derive insights from Kafka streams.
Dynamic Schema Derivation: Develop custom tooling to derive table schema for Kafka streams dynamically, simplifying the exploration process by automatically adjusting to changes in the data structure.
Security Measures: Implement security measures using Strimzi for Kafka stream management and Open Policy Agent (OPA) to define and enforce policies. This ensures secure and authorized data access, maintaining data integrity and compliance with privacy regulations.
☕️ Tea Time
News that you might have missed
Google has announced the Season of Docs 2024. It’s similar to GSOC but for technical writing. Check it out if you are interested in writing.
Highlights from Mckinsey’s report on the US job landscape
Generative AI is expected to transform the job landscape by potentially automating 30% of current work hours by 2030.
Future jobs will demand higher levels of education or skills, particularly in social-emotional and digital areas.
There's a growing need for reskilling and upskilling to meet the evolving demands of the job market.
Who’s hiring?
In a recent podcast, Rippling’s CEO, Parker Conrad, mentioned they plan to hire for 100+ positions at the Bangalore office. Rippling has listed 39 open roles for India locations on their careers page (when writing this).
As of 2023, Rippling offered a base salary of 36 LPA (with PF) + 8L ESOPs for freshers. The base can vary from 40-60 LPA for SDE2 and up to ~80 LPA for SDE3. Verified from independent sources and publicly available information.
🚀 Power Up
As Rippling is hiring, let’s discuss some system design questions/concepts spotted in their interview process. Do note that the designs presented are just one of the many possibilities of how it can be approached.
Instagram Feed Service
To design a service that generates a personalized Instagram (or any social media) feed, showing posts from users they follow, including photos, videos, and stories, in a performant and scalable manner.
Data Model: A hybrid of relational (MySQL, Postgres) (for structured data such as user profiles) and NoSQL databases (MongoDB) (for unstructured data like posts and stories) to manage diverse data types efficiently.
Feed Generation Algorithm: Implement a combination of push and pull mechanisms.
Push Model: Server automatically initiates data transfer to the client when new data is available. Ideal for real-time updates.
Pull Model: Client requests data from the server when it needs updates. More suitable for non-real-time applications.
For active users, pre-generate feeds using a background service that pushes updates to their feed in real time (push model).
Using WebSockets: A persistent connection is established between the client and server, allowing the server to send data whenever it's available.
Using Server-Sent Events: A one-way channel from server to client to send data whenever new data becomes available.
For less active users, generate feeds on-demand (pull model) to conserve resources.
Using Polling: The client periodically sends requests to the server at regular intervals, asking for any new data.
Long Polling: The client sends a request to the server, but the server holds the request open until new data is available. Once the data is sent to the client, the client immediately sends another request, and the cycle continues.
Caching: Distributed caching to store hot data, such as frequently accessed feeds, to reduce database load and improve response times. Any key-value store like Redis can be used.
Content Delivery Network (CDN) (e.g., Cloudflare, Akamai, AWS CloudFront): To cache and serve media content (photos, videos) closer to the user's location, minimizing latency.
Microservices Architecture: Adopt microservices for different functionalities (e.g., user management, content upload, feed generation) to enhance scalability and maintainability.
Load Balancing: Distribute incoming requests across multiple servers to ensure even load distribution and high availability.
Data Partitioning/Sharding: Dividing a database into smaller, more manageable pieces, where each shard can be stored on a different database server. Shard databases based on user ID or geographical location to manage the dataset's growth and maintain performance.
Real-time Analytical System
To design a real-time analytical system that collects data from various services, processes this data continuously, and generates performance metrics to monitor the health and efficiency of these services in real time.
Data Ingestion: Use high-throughput, scalable messaging systems like Kafka to ingest data streams from various sources.
Deploy a Kafka cluster to serve as the central messaging system.
Implement producers (of data) in the source systems to send data to Kafka topics.
Configure Kafka topics for each data stream, considering partitioning and replication to balance load and ensure high availability.
Stream Processing: Involves processing data as it flows through the system. Frameworks like Apache Flink and Spark Streaming can be used to process data in real-time, applying transformations and aggregations to generate insights.
Time-Series Database: A time-series database (e.g., InfluxDB) is specialized for storing and querying time-stamped data. It's optimized for fast writes and queries over temporal data, making it ideal for storing metrics and performance data. Store processed metrics in such a database.
Microservices Architecture: Divide the system into microservices, each responsible for a part of the data processing pipeline, to enhance scalability and fault isolation.
Data Ingestion Service
Data Processing Service
Data Storage
Data Visualization Service
Monitoring and Alerting: Integrate monitoring tools to track system health and performance with alerting mechanisms to notify stakeholders of anomalies or performance issues.
Use monitoring tools (e.g., Prometheus) to collect metrics from each microservice.
Set up alerting rules based on thresholds for performance metrics to automatically notify stakeholders via email, SMS, or other channels when anomalies are detected.
Data Visualization: Provide dashboards using visualization tools to display real-time metrics and insights in an accessible format for stakeholders.
Integrate a visualization tool like Grafana with the time-series database to create real-time dashboards.
📨 Post Credits
Whether it's scaling iOS applications, exploring data, or diving into system design, the underlying theme is perpetual advancement and adapting to changing times. Stay curious, stay inspired, and never stop learning.
Fin.