Ranking Systems
Instagram Feed Ranking
How thousands of possible posts become the handful you actually scroll, assembled fresh every time you pull to refresh.
How thousands of possible posts become the handful you actually scroll, assembled fresh every time you pull to refresh.
Pull to refresh and a feed appears in a fraction of a second. Behind it, the system considered a large pool of possible posts, scored them against a model of what you personally engage with, and assembled an ordered page — all before the animation finished.
The core tension is between doing that work in advance and doing it on demand. Precomputing every user's feed is fast to read but enormously wasteful, since most feeds are never opened. Computing on demand is efficient but slow at read time, which is exactly when latency is visible.
Real systems refuse to choose. Ordinary accounts push their posts into followers' feeds ahead of time; accounts with millions of followers do not, and are merged in at read time instead.
Think of it like this: it is a newspaper printed individually for every reader at the moment they pick it up — mostly typeset in advance, but with the front-page stories slotted in as you reach for it.
Feed construction has to reconcile several conflicting requirements:
A feed must appear almost instantly, because a visible delay on the most-used screen is unacceptable.
A single post by a popular account can require writing into millions of individual feeds.
Something posted moments ago should be eligible immediately, not after a batch job runs.
Chronological ordering buries good content from close friends under high-volume posters.
Most precomputed feeds are never read, so eager computation spends resources on nobody's behalf.
The pipeline mirrors the recommender shape — gather candidates cheaply, rank them expensively, then assemble a page — but with the added complication of deciding when the gathering happens at all.
The durable record of every post, its author, media references and metadata, queried by author rather than by viewer.
In shortThe permanent library of everything anyone has posted.
Who follows whom. Traversing this determines whose posts are even eligible for a given viewer's feed.
In shortThe map of who is connected to whom, which decides whose posts you could see.
A per-user list of candidate post references, populated by fan-out on write for ordinary accounts and kept deliberately bounded in length.
In shortA pre-built shortlist of posts waiting for each user.
Merges the precomputed index with pulled-at-read-time posts from very large accounts, plus recommended content from outside the follow graph.
In shortCombines the ready-made list with fresh additions fetched on the spot.
Predicts several engagement probabilities per post — like, comment, share, dwell, hide — and combines them into a single ordering score.
In shortGuesses how you will react to each post, in several different ways at once.
Applies diversity rules, integrity filters and business policy before returning the final ordered page.
In shortFinal tidy-up so the feed is varied, safe and sensible before you see it.
A new post is written to the post store and an event is emitted for downstream processing.
In shortSomeone posts, and the system records it and announces it internally.
If the author has a manageable follower count, the post reference is pushed into each follower's feed index. Very large accounts skip this entirely.
In shortNormal accounts have their post pushed to followers; huge accounts do not, or it would never finish.
The precomputed index is read, then merged with fresh posts pulled directly from any large accounts the viewer follows.
In shortYour ready-made list is loaded, then the big accounts are checked live.
Already-seen posts, blocked authors and policy-violating content are removed before any scoring work is done.
In shortAnything you have already seen or should not see is dropped first.
The ranking model scores each remaining candidate on multiple predicted engagement types, which are weighted into one value.
In shortEach post is graded on how likely you are to react to it.
Consecutive posts from the same author are spaced out, ads and recommendations are interleaved, and the page is returned.
In shortIt shuffles things so you do not get six posts from one person in a row.
This is the defining architectural decision of any feed system, and the reason a purely elegant answer does not survive contact with real follower distributions.
When a post is created, it is immediately pushed into the feed index of every follower. Reads become a simple list lookup and are extremely fast.
In shortDeliver the post to everyone's inbox the moment it is written, so reading is instant.
Nothing is precomputed. When a user opens the app, the system queries everyone they follow and merges the results on the spot.
In shortDo nothing until asked, then go and collect the posts right then.
An account with fifty million followers would require fifty million writes for one post. Fan-out on write is impossible at that scale, and the resulting delay would make the post stale before it landed.
In shortOne post by a superstar would mean millions of deliveries, which is simply not workable.
Ordinary accounts fan out on write; accounts above a follower threshold are pulled at read time and merged. Most users get fast reads, and no single post triggers an unbounded write storm.
In shortNormal accounts get pushed, famous ones get pulled, and the two are stitched together when you look.
The original design was simply reverse chronological, which is transparent, predictable and steadily worse as the number of accounts you follow grows:
In short: ranking is not one prediction but several — the chance you will like it, comment, share, linger or hide it — blended into a single number. Choosing those weights is a product decision about what the feed is for, not a technical one.
A feed ordered purely by predicted engagement is not a feed anyone enjoys for long. Several corrections are applied after ranking.
Consecutive posts from the same account are spaced apart, since a prolific poster would otherwise dominate the top of the feed by sheer volume.
In shortOne person's posts get spread out instead of stacking up together.
Hides, mutes, reports and rapid scroll-past are weighted heavily, because they are far more informative about dissatisfaction than the absence of a like.
In shortSkipping past something quickly tells the system as much as tapping like does.
Policy enforcement runs after ranking, so a highly engaging post that violates policy is removed rather than promoted — engagement and acceptability are separate questions.
In shortSomething can be very clickable and still not be allowed, and the check happens last.
Ranking on immediate engagement alone tends toward sensational content, so longer-horizon signals such as returning the next day are weighted in.
In shortIt tries to predict whether you will come back tomorrow, not just whether you will tap now.
The read path is one of the highest-volume operations on the internet, so nearly everything is bounded, cached or precomputed:
| Layer | Common Choices |
|---|---|
| Post Storage | Sharded key-value or wide-column stores partitioned by author |
| Social Graph | Purpose-built graph or adjacency storage with heavy caching |
| Feed Index | Redis lists or sorted sets holding bounded per-user candidate references |
| Fan-out Workers | Queue-driven asynchronous workers with follower-count thresholds |
| Ranking | Low-latency model serving with strict per-request time budgets |
| Media Delivery | CDN-backed image and video delivery, independent of feed assembly |
| Experimentation | A/B infrastructure with long-horizon guardrail metrics |
Feed ranking combines the retrieve-then-rank funnel of a recommender with a hard distribution problem: deciding what to compute eagerly and what to compute on demand. Any system serving personalised content to a population with wildly uneven activity levels ends up making the same hybrid compromise.
A quick, no-nonsense translation of the technical terms used above.
Pushing a new post into every follower's feed at the moment it is created.
Building the feed by querying followed accounts when the user opens the app.
A per-user list of candidate post references, usually capped in length.
The difficulty that accounts with enormous follower counts break fan-out-on-write entirely.
Scoring on several predicted outcomes at once, then combining them into one ordering value.
How long a viewer lingers on a post, used as an engagement signal that does not require a tap.
Continuing a list from a stable marker rather than a numeric offset, so shifting data does not cause duplicates.
When one logical action causes a disproportionately large number of underlying writes.