The first time your production app goes down because an indexer is having a bad day, you feel betrayed. You built on a blockchain. The blockchain is fine. The blockchain has been producing blocks this whole time. But your users are staring at a loading spinner because some company's API decided now was a good time to return 503 errors.

You didn't sign up for this. But here you are.

What follows isn't a critique of any particular company. The providers are doing their best with a problem that probably shouldn't exist in the first place. This is about the system itself, the accumulated friction that every blockchain developer pays in time and sanity. You've been paying it so long you might have forgotten it's not normal.

The Onboarding Gauntlet

Your first clue that something is wrong comes during onboarding. You need blockchain data, so you sign up for an indexer. Simple enough. Then you realize your application needs data from two chains, and your first provider only supports one of them. So you sign up for another provider. Now you have two dashboards, two billing accounts, two API keys to rotate, two sets of documentation to reference.

Then someone asks about supporting a third chain.

The API key management alone becomes a chore. You're storing secrets for multiple providers, each with their own rotation policies and security recommendations. Your environment variables file starts looking like a password manager had a nervous breakdown. Someone on the team inevitably commits a key to git and you spend an afternoon rotating credentials while pretending you've never done the same thing.

Rate limits add another layer. Free tiers exist for development, but they're calibrated to stop working right around the time you have actual users. So you graduate to paid plans. One provider charges per compute unit. Another charges per request. A third has some formula involving "request weight" that nobody on your team fully understands but you pay anyway because what's the alternative?

You start to realize that "supporting multiple chains" doesn't mean writing code for multiple chains. It means maintaining relationships with multiple vendors, each of whom sends you emails about their exciting new enterprise tier.

The Integration Tax

Once you've survived onboarding, you get to the actual integration work. This is where the real friction lives.

Each provider has their own query language. Some use GraphQL. Others expose REST endpoints. A few have proprietary query syntax that exists nowhere else in the industry. You write wrapper code to normalize the differences. Then you discover edge cases where the wrappers don't quite work. Then you write special-case handling for those edge cases. Then you discover more edge cases. Your abstraction layer develops its own abstraction layer.

The data formats are worse. You'd think that token transfer events would look roughly similar across providers. They don't. Field names differ. Timestamp formats differ. The way addresses are checksummed differs. One provider returns token amounts as strings, another as numbers, a third as hex. Numerical precision handling differs in ways that surface as bugs in production and nowhere else, usually involving someone's money.

So you write transformation layers. Your codebase accumulates adapters for each provider, each adapter papering over the differences between what you get and what your application actually needs. This code adds no value to your product. It exists purely because the infrastructure underneath you couldn't agree on what a transaction looks like.

The industry has collectively spent thousands of hours on this translation problem. Every team writes their own version. None of them are quite compatible with each other.

Works in Dev, Breaks in Prod

Your local environment works perfectly. Your staging environment works fine. You ship to production with confidence.

That confidence lasts about a week.

Rate limits hit first. Development traffic is light, so you never saw the ceiling. Real users generate real load, and real load finds limits fast. Your application starts returning errors during peak hours. Features that worked in testing now fail in front of actual customers, but only sometimes, which makes it harder to debug and easier to gaslight yourself into thinking it's not that bad.

You can upgrade your plan, but that only buys you a higher ceiling. Growth means you'll hit the new one eventually. You've signed up for a game where the rules change based on how well you're doing.

Caching is subtler. Providers cache aggressively to handle load, which means your application sometimes displays stale data without knowing it. You query for a user's token balance, get a response from thirty seconds ago, and show them information that's already wrong. They complain their transaction isn't showing up. You tell them to refresh. They do. Still stale. You tell them to wait. They wait. It works. Nobody apologizes because nobody knows whose fault it was.

Schema changes arrive without warning. You wake up to alerts because an endpoint started returning different JSON overnight. The provider's changelog mentions it somewhere, buried in a release notes page you didn't know you were supposed to monitor. Your integration is broken in production. It's 7am and you're pushing a hotfix because that's when you happened to check your phone.

The Debugging Nightmare

Something is wrong with your application. Users are seeing incorrect data. You need to figure out whether the bug is in your code, in the indexer's pipeline, or somewhere in between.

You query your indexer. You query the chain directly. The results don't match. Your indexer shows a balance of X. The chain shows Y. Which one is right? Probably the chain, but you can't be certain without understanding exactly what block your indexer has processed to. That information isn't always easy to get. Sometimes it's not available at all.

Reorg handling makes this worse. The chain reorganized, and your indexer was in the middle of processing a block that no longer exists. For a while, your indexed data reflects a state that never actually happened on the canonical chain. Is this a bug? An expected behavior? An inevitable consequence of distributed systems? All of the above, probably. That doesn't help you explain to a user why their NFT briefly showed up in someone else's wallet.

You file a support ticket. Maybe you get a response in hours, maybe days. The response asks for information you already provided. You respond with the same information, formatted differently. Silence. Eventually the issue resolves itself, or you work around it, or you just accept that this particular query sometimes lies to you.

The fundamental problem is that you can't verify any of this. You're trusting that the indexer processed every block accurately, applied every transformation correctly, and served you the genuine result. When something looks wrong, there's no cryptographic proof to check. No way to independently verify. Just faith and a support ticket.

The Infrastructure You Didn't Want to Build

Open any production blockchain application's codebase. Scroll past the business logic. Find the infrastructure code.

There's retry logic, because requests fail for no apparent reason. There's fallback provider configuration, because your primary goes down more than you'd like to admit. There's a circuit breaker implementation you copied from a blog post and modified until it worked. There are health checks running on a cron job. There's stale data detection that mostly works. There's timeout handling tuned through trial and error.

This is infrastructure that exists because the infrastructure you're paying for isn't reliable enough to use directly.

The absurdity is hard to overstate. You're paying for a managed service specifically so you don't have to run your own infrastructure. Then you build infrastructure anyway because you can't trust the managed service. The "reliable" endpoint requires redundancy. The "simple" integration requires a thousand lines of defensive code. You've essentially built a worse version of what you were trying to avoid building.

None of this makes your product better. Your users don't care about your circuit breaker. They care whether the app works. You've spent weeks on code that, in a saner world, wouldn't need to exist. But here you are, proud owner of a retry mechanism with exponential backoff and jitter.

The Multi-Chain Multiplier

Everything above scales with the number of chains you support. Every pain point, every integration, every source of bugs. Your three-chain application has relationships with four providers because no single provider covers everything adequately. Each has different uptime characteristics. Different caching behavior. Different ways of failing. Different rate limit resets.

Your fallback infrastructure, already complex, now needs chain-specific logic. A provider outage on Arbitrum shouldn't affect your Ethereum features, except somehow it does, and now you're debugging connection pool exhaustion at midnight.

Cross-chain features are where the pain really compounds. You want to show a user their total portfolio across chains. This requires querying multiple indexers, normalizing responses that don't agree on how to represent the same data, handling partial failures gracefully, dealing with inconsistent update latencies, and presenting something coherent. The user just wants to see a number. They have no idea what you went through to display it. Probably better that way.

Adding a new chain means weeks of work. Not because the chain is complicated, but because you need to find a provider, learn their API, add their quirks to your adapter layer, extend your fallback logic, test everything, and pray the new integration doesn't break the old ones. You'll do this dance every time someone asks "can we support [new chain]?" with the casual tone of someone asking you to add a feature.

The Upgrade Treadmill

The infrastructure underneath you never stops moving.

API versions get deprecated. You have six months to migrate, which sounds generous until you realize you're also shipping features, fixing bugs, and dealing with two other migrations. The documentation assumes a fresh integration, not the pile of workarounds you've accumulated. Halfway through you discover an undocumented behavior you were depending on. It doesn't exist in the new version. You spend a day finding an alternative that's almost as good.

Services sunset entirely. The provider you've built on announces they're discontinuing your tier, or your endpoint, or the whole product. They point you toward alternatives that aren't quite equivalent. Every migration is an opportunity for new bugs. You've gotten good at migrations. You wish you hadn't needed the practice.

If you're using subgraphs, you're maintaining code in two places. The subgraph needs to deploy and sync before you can use new data. Syncing takes time. Sometimes it fails partway and you start over. Sometimes it completes but produces wrong data, and you only discover this after deploying application code that depends on it. The feedback loop is measured in hours.

You're not building on stable ground. The surface shifts constantly, and keeping up with the shifts is part of your job now. Time you could spend on your product goes to chasing targets that won't stop moving.

What We've Accepted

Ask any blockchain developer about indexer pain. They'll have stories. The outage during a launch. The schema change that broke production on a Friday. The rate limit that kicked in right after a popular account tweeted about their app. The support ticket that went nowhere. They tell these stories with the gallows humor of people who've adapted to dysfunction.

This is just how it is, they'll say. You work around it. You build defensive infrastructure. You develop opinions about retry strategies. You move on.

But none of this is inherent to blockchain development. It's a consequence of how the data layer evolved: as an afterthought, built by companies solving their own problems in isolation. The result is what you'd expect from an industry that grew faster than it could coordinate. Redundant effort everywhere. Inconsistent interfaces. Reliability problems that everyone routes around individually because there's no collective solution.

It doesn't have to stay this way.

Verifiable data that doesn't require trusting a company's word is an engineering problem, not a fantasy. Unified access across chains is an engineering problem. Infrastructure that doesn't require you to build backup infrastructure is an engineering problem. These problems have solutions.

All that defensive code you've written? It shouldn't need to exist.

Join the community building the read layer that lets you focus on your product.
X · Telegram · Discord · GitHub · shinzo.network