How EveSynapse Syncs
EVESYNAPSE

How EveSynapse stays fast without waiting on EVE's servers

Two versions of the same explanation: site copy written for pilots, then the engineering breakdown with diagrams, taken from the code.

For the site

Plain English version

TYPICAL TOOL You click Ask EVE's servers rate limited, sometimes slow Page you wait still waiting EVESYNAPSE You click Local copy of your data already fresh Page, now kept fresh in the background, every minute
The difference in one picture: EveSynapse moves the wait off your screen and into a background process.
Long version

Why EveSynapse feels instant

Most EVE tools ask CCP's servers for your data at the moment you click. EVE's API has strict rate limits and some answers take a while, so you sit and wait for the page.

EveSynapse works the other way around. A background process keeps a fresh copy of your characters, skills, assets, markets and corporation data, and refreshes each piece as soon as CCP says it has changed. When you open a page, it is built from that copy, so it loads right away.

It also looks ahead. The data closest to you, like the pilots you trade and talk with and the items in your hangars and industry jobs, is fetched before you ever click on it. As you type in search, the likely matches start loading before you pick one.

If you open something the copy does not have yet, like an item or a pilot nobody has looked up before, the page still loads. That one section shows as syncing, and EveSynapse moves it to the front of the line. It usually fills in on its own within a few seconds, without a refresh.

All of this runs on a fixed fetch budget that stays inside CCP's limits. If CCP's servers push back, EveSynapse slows down instead of retrying harder, and keeps showing you the last good copy in the meantime.

The result is pages that open instantly, data that stays current, and a tool that is a good citizen on CCP's servers.

Short version

EveSynapse builds every page from a live local copy of your EVE data, kept fresh in the background. Pages open instantly, and anything new jumps the queue and fills in within seconds.

Technical breakdown

Architecture: renders never call ESI

The core rule is that a page render reads only from Postgres and in-process caches. Two background loops are the only code that talks to CCP's ESI API, and both spend from a shared, bounded budget. Anything a render could not resolve becomes a queue row (a "want") instead of a network call.

Browser polls fragments every 1.5 s Page server pages + fragments reads DB only Postgres snapshots, name caches, want queues Minute worker every 60 s, 120 fetches Urgent drain every 5 s, small caps CCP ESI GET HTML now read rows note wants due snapshots store wants, viewed first fetch + store never during a render
Only the two loops on the right ever reach ESI. The page server's whole job is to read stored rows and note what is missing, so its latency is a database query, not a network round trip to CCP.

  1. Snapshots. The minute worker walks every linked character, refreshing each ESI dataset whose cached_until has passed, stored as raw JSON keyed by character and kind. Most-overdue characters go first; fresh logins and Sync-page requests jump ahead.

  2. Two-tier names. Item, place and pilot names render from the SDE tables and local caches only. A name still missing shows as Type #1234 until a warm-up pass fills it.

  3. Wants. When a render meets something unresolved, notePageWant writes a queue row at "viewed" priority and registers it against the page, which drives the sync indicator in the top banner. Handlers only write queue rows; they never fetch.

  4. Live regions. Sections rendered while pending carry a poll URL. The browser polls a fragment endpoint that also reads only stored rows, and swaps the section in once its state leaves pending.

A page load, step by step

Here is what happens when a pilot opens a market item nobody has viewed before, so its price history is not cached yet.

Browser Page server Postgres Urgent drain ESI 0 s open item read cached rows note want: history page, history pending +1.5 s poll fragment still pending ≤ 5 s take viewed wants fetch data store rows next poll poll fragment filled, swap in
The pilot sees a complete page at 0 s. Only the missing section waits, and it waits for a background fetch, not for the request that drew the page. The page server never appears on the ESI lifeline.

Most of the time this sequence never runs: proactive coverage (market coverage tiers, the pilot "orbit" around your characters) has usually fetched the data before anyone asks. The urgent path exists for the long tail, such as a pilot ID typed into search.

The constellation: closer data goes first

Every fetch competes for the same small budget, so the order matters more than the speed of any single call. EveSynapse ranks work by how close it is to the pilot: what is on screen, what they are about to open, what their own data touches, and only then everything else.

On screen Typing Your orbit One hop out Popular EXAMPLES, CLOSEST FIRST 1 On screen item page with no history yet 2 Typing top 3 search suggestions 3 Your orbit counterparties, hangar items 4 One hop out their corp, then alliance 5 Popular most-traded items in Jita ONE WORKER CYCLE: 120 FETCHES spent in ring order, center first Every ring draws from the same budget, so closeness decides what gets fetched first.
Priority rises toward the center. The rings never widen the budget; they only decide the order it is spent in.

Rate limits: urgency reorders work, it never widens it

Viewed wants jump the queue, but every fetch still passes the same gates. A busy page cannot make EveSynapse spend more of ESI's error budget; it can only change what gets spent first.

Work item due snapshot or want Shared lock loops never collide Budget 120 per cycle, small caps Freshness gate honors Expires ESI 420 / 429 error limit stop the cycle; hold urgent drain 2 min 403, role missing record it, retry in 6 h; page shows "needs role" Other failure keep serving the stale snapshot; retry next cycle
The three failure paths each degrade to a visible, honest state instead of a retry storm or an error page.
One deliberate exception

The interactive market search may make a single name lookup for an item nobody has cached yet, so a typed search can resolve. Every other render stays cache-only, and the Briefing module's test suite asserts zero outbound calls.

The numbers

Constants as they stand in the current source.

SettingValueWhere
Background worker cycle60 sworker.go
ESI fetches per worker cycle120maxFetchesPerCycle
Urgent drain tick5 surgent.go
Urgent caps per tick: pilots / history / corp + alliance names / item details3 / 3 / 8 / 2urgent.go
Urgent drain hold after an ESI error limit2 minurgentErrorBackoff
Retry after a missing in-game role (403)6 hroleMissingBackoff
Snapshot freshness when ESI sends no Expires header5 minREADME
Fragment polling, then slowing to1.5 s → 5 sapp.js
Sync indicator stops counting a want after3 minpageWantTTL
Killmail details warmed per character per cycle10README
War details warmed per cycle50README