Here's how a website can become completely untraceable — and why Google only ever sees a fraction of the internet.
Scroll through the actual mechanism: crawling, the three layers of the web, and exactly how Tor moves your traffic through relays, circuits, and layers of encryption — peeled apart step by step, with no jargon left undefined.
How search engines even find anything
Two mechanics explain everything that follows. Get these right and "deep web" and "dark web" stop being spooky terms — they're just answers to one plain question: could a crawler get in here or not?
Crawling
An automated program — a crawler or spider — visits a webpage and reads through everything on it. Think of a librarian walking down every aisle of a library, opening every book, and reading every page.
Indexing
Organizing what was crawled by keyword into a searchable catalog. That same librarian writes down which books mention which topics, so next time you ask, she can point you straight to the right shelf — instantly.
Surface web, deep web, dark web
Same underlying internet. The difference is purely whether a crawler could get in, and how deliberately something is hidden. Tap a layer.
Surface web
~4–10% of the internetFully indexed by search engines and freely accessible to anyone — no login required. This is what you touch every time you Google something. Sources disagree on the exact share, so treat any single fixed percentage you see elsewhere with suspicion — the honest answer is "a small slice, roughly 4–10%."
Deep web
the majority of the internetNot fully indexed, OR requires a login — either condition on its own qualifies a page as deep web. It's usually unindexed for boring technical reasons: crawlers can't get past a login wall, can't follow a link that's generated on the fly, are told to stay out by a robots.txt file, or simply have no other page linking in to them.
Everyday examples: your online banking dashboard, a paywalled academic journal, a company's internal intranet, your email inbox.
Dark web
a hidden slice of the deep webA deliberately hidden subset of the deep web — you need special anonymizing software to even reach it. Tor is the most common route in, but it isn't the only one; I2P and Freenet are others.
The concealment here is different in kind, not just degree — it works at the network level. .onion addresses can't be looked up through normal DNS the way "google.com" can, and hidden services use a "meeting in the middle" system (more on that later) so that neither the visitor nor the website ever learns the other's real IP address.
Dark web ≠ automatically illegal. It's an anonymity technology, not a content category. Legitimate uses include SecureDrop whistleblower drop-boxes, and .onion mirrors of outlets like the BBC and the New York Times used by journalists and activists under censorship. It also hosts illicit markets. Both of those things are simultaneously true.
Relays, circuits, and the fog between neighbors
Before we watch data move, four small words need to be nailed down — the rest of this page leans on them constantly.
Relay — a server that's part of the Tor network. Anyone can volunteer to run one.
Circuit — a connection between two relays. Every circuit has its own Circuit ID.
Circuit ID — a purely local label, only meaningful on one specific link between two neighboring relays. It is not one global ID that follows your traffic the whole way through. Client↔Entry has one Circuit ID, Entry↔Middle has a completely different one, Middle↔Exit has yet another. No single relay ever sees one ID spanning the full journey.
Circuit state — the private bookkeeping table each relay keeps: "traffic on Circuit ID X, coming in from my left neighbor, maps to Circuit ID Y going out to my right neighbor — and here are the keys for this hop." That table is the mapping itself, distinct from a Circuit ID, which is just a label stamped on a packet.
Three different Circuit IDs, three separate local labels. Each relay only ever knows its own two immediate neighbors — never the full path.
| In: neighbor | In: Circuit ID | Out: neighbor | Out: Circuit ID |
|---|---|---|---|
| You (browser) | 41 | Middle relay | 9 |
How the tunnel and all six keys get built
This is the part people usually get wrong: the entire three-relay tunnel — and every key it needs — is fully negotiated before your actual request is ever sent. Nothing is built on the fly, mid-journey.
Browser picks three relays
One entry, one middle, one exit — chosen for this tunnel only.
Private handshake, directly with entry
The browser negotiates two keys with the entry relay: a forward key (Kf) that encrypts client→relay traffic, and a backward key (Kb) for relay→client traffic. The entry relay never learns any other relay's keys.
Tunneling through entry, a separate handshake with middle
Entry just relays the sealed handshake message without understanding it — it can't read the contents. Two more keys result, known only to the browser and the middle relay.
Tunneling through both, a separate handshake with exit
Same pattern, one hop further. Two more keys result.
Only now — all six keys in hand — does the real request go out
Wrapped in three layers of encryption, one per relay.
This tunnel typically stays up and gets reused for multiple site visits in one session, then rotates — roughly every 10 minutes. It isn't rebuilt before every single click.
Full data flow to an ordinary website
Same tunnel from the last section, now carrying real traffic. Step through it — watch the envelope pick up layers, then lose them one at a time.
Full data flow to a hidden service
Same building blocks — relays, circuits, layered encryption — arranged differently. The one thing to hold onto: your browser never learns the hidden service's real IP address, at any point. That's the entire point of a hidden service.
This is infrastructure knowledge, not folklore
Understanding how relays, circuits, and hidden services actually work isn't just satisfying — it's directly useful for threat intelligence and pentesting work, where knowing what Tor can and can't hide (and where the actual weak points are) matters far more than treating it as a mysterious black box.
I'm Sarath — a cybersecurity trainer and consultant with 9+ years in the field. I've trained teams at organizations including: