A practical explanation of blocking queues using Go channels: what they are, how blocking producers/consumers coordinate work, and common patterns (producer-consumer, worker pools, pipelines, delay queues). The article includes Go code for blocking, timeout and non-blocking operations, discusses bounded vs unbounded queues, shutdown strategies, real-world uses (web servers, batch jobs, logging), common pitfalls (goroutine leaks, capacity sizing), and performance considerations like batching and monitoring queue depth.
Arpit Bhayani
Every week, I document and articulate my thoughts and learnings on Career Growth, Database Internals, and Engineering Explorations. Here are all blogs I wrote to date.
A technical guide to heartbeat mechanisms in distributed systems: what heartbeats are, sender/monitor roles, interval and timeout trade-offs, push vs pull models, failure detectors (fixed timeout and phi accrual), gossip-based membership for scalable failure detection, implementation considerations (transport, topology, resource management), handling partitions/quorums, and real-world examples like Kubernetes and Cassandra.
A technical deep-dive into Apache Cassandra’s write path: how a coordinator routes client writes using partition keys and the token ring, how replicas are chosen (replication strategies), and what each replica does (CommitLog -> MemTable -> SSTable). It explains consistency levels, conflict resolution via timestamps, hinted handoff and read repair, compaction strategies, multi-datacenter behavior, common failure scenarios, performance characteristics (throughput/latency), and operational best practices.
Technical deep-dive into why Redis uses push-based replication: explains the PSYNC/full-resync protocol, the replication backlog, immediate streaming of write commands to replicas, benefits (lower latency, simpler replica consistency, network efficiency), trade-offs (master resource use, reduced replica autonomy), tuning knobs (repl-backlog-size, min-replicas-to-write), monitoring metrics, and diskless replication optimization.
Practical advice for dealing with arrogant colleagues: remain professional, pick battles, ask clarifying questions, build allies, document persistent issues and escalate when necessary, protect your mental health, and lead by example to foster respect and collaboration.
A technical deep-dive into how CDNs replicate and manage content worldwide. It explains push vs pull (and hybrid) replication, hierarchical cache tiers and request coalescing, content-addressed storage with hashes/ETags, eventual-consistency trade-offs, optimized inter-node transfer techniques (chunking, delta encoding, peer fetch), and operational best practices (versioned URLs, TTLs, stale-while-revalidate, monitoring replication lag).
Advice for new hires to avoid rushing to 'fix' everything: ask why, listen, build trust, pick one battle, and prove ideas with small wins so you gain influence to make larger changes over time.
Discusses why emotional outbursts happen at work, their negative effects on team morale and productivity, practical steps individuals can take to manage them, and leadership actions to prevent and channel frustration constructively.
Explains why gRPC was built on HTTP/2 by detailing HTTP/2 features (binary framing, multiplexing, HPACK header compression, flow control) that enable gRPC's bidirectional streaming, efficient header handling, reduced connection overhead, and improved latency/throughput in microservices; also covers trade-offs like proxy compatibility, memory costs, debugging complexity, and protobuf synergy.
Advice to always include a clear agenda with meeting invites so participants can prepare, discussions stay focused, outcomes are produced, and people can decide whether to attend; suggests enforcing "no agenda, no meeting" when appropriate.
A short guide for software engineers on focused growth: ask clear questions to define motivation, niche, the 20% of work that yields 80% of results, set a timeboxed plan, and measure progress. Use AI for planning, emulate select traits from role models, produce artifacts for milestones, and prioritize consistent execution over trying to do everything or over-scheduling.
The author argues that while frequent job switches can boost short-term pay, staying longer in roles—especially at peak levels—often yields greater long-term wealth and opportunities by enabling deep ownership and career growth. Move on if a role stagnates, but prioritize longevity and thinking in decades over constant hopping.
Career advice for senior engineers and leaders: higher pay can reduce switching options but doesn’t eliminate them if you can demonstrate measurable impact (revenue saved, time reduced, team scaled, systems built). Be selective about moves, build a strong track record, double down on strengths, and choose environments where you can thrive rather than just survive.
The article explains why consensus algorithms like Raft are necessary for distributed systems: it describes common failure modes (network partitions, node crashes, asynchronous communication, clock skew), defines the safety and liveness guarantees consensus provides, states formal consensus properties, and gives real-world examples where consensus is critical (distributed databases, leader election, transactions, service discovery, and Kubernetes/etcd).
A technical guide explaining what database deadlocks are, common causes (lock ordering, long transactions, query-plan differences, lock escalation, foreign keys), how databases detect and resolve them (wait-for graphs, victim selection, DB-specific behaviors in PostgreSQL and MySQL), prevention tactics (consistent lock ordering, short transactions, retries, query/index optimization), and monitoring/logging practices to surface and mitigate deadlocks.
Explains CPU cache hierarchy, cache lines, and temporal/spatial locality; demonstrates how memory access patterns (e.g., row-major vs column-major matrix access) and data layout can cause large speed differences. Covers measuring cache behavior with perf, gives the Redis eviction-pool as a real-world example of optimizing for locality, and lists optimizations such as loop tiling, cache-oblivious algorithms, prefetching, and false-sharing mitigation.
The article explains why eventual consistency is the dominant model for large-scale distributed systems: network latency and inevitable partitions make availability and low latency more practical than global strong consistency. It covers the CAP theorem, real-world failure scenarios, implementation patterns (read-your-own-writes, vector clocks, CRDTs), case studies (DynamoDB, DNS, social platforms), cost and scaling implications, and notes when strong consistency remains necessary (e.g., financial transactions).
Explains why DNS uses both UDP and TCP: UDP is preferred for speed and low resource use, while TCP is used for large/truncated responses (EDNS0 limits, DNSSEC, big TXT/A records) and zone transfers. Covers UDP→TCP fallback logic, performance/throughput tradeoffs with numeric estimates, implementation tips (dnspython example), and operational guidance (monitoring metrics, firewall/load balancer configuration).
A candid, practical guide on whether to pursue a master’s degree: pursue one only if you have a clear reason (career reset, geographic move, or academic research), rigorously evaluate hiring prospects, field longevity, opportunity cost and loan repayment, and align the decision with a 5–7 year plan. The author shares a positive personal ROI from an IIIT master’s but emphasizes that in tech, impact often matters more than degrees.
Argues that empathy and product thinking distinguish great engineers from mere coders; being empathetic leads to better system design, clearer communication with stakeholders, stronger reputation, and more impactful career opportunities.