🚧

Early Access

Unfortunately, we're not yet available to the general public. If you're interested and have a project that needs real-time web3 data, sign up at https://goldsky.io and we'll get back to you!

Goldsky is a platform for reading from blockchain in real-time. Instead of reading directly from RPC nodes or pre-defined APIs, we aim to give everyone the tools they need to build reliable, real-time apps without having to do data massaging in their front-ends or back-ends.

Defining a source gives you a data stream you can manipulate using SQL and other tools. You can layer multiple transformed streams on top of each other to create alerts, new views, merge data and more. The results of your transformations can be accessed through a GraphQL API, or directly in your own backend through real-time ETL.

📘

With Goldsky, you can create a reliable, real-time view into the blockchain that minimizes the number of queries you need to make from your code.

Ingesting Blockchain Data


Subgraph Indexing

Goldsky supports The Graph compatible subgraphs for ingestion.

Subgraph indexing is made reliable through:

  • A network node layer we monitor 24/7. This:
    • uses a combination of third-party node providers and on-premise nodes for speed.
    • reduces block processing latency by grabbing the latest block out of the pool.
    • isolates node downtime through a load balancer.
  • Subgraph data consistency alerts and recovery, to help prevent corruption from reorgs or other issues.
  • An on-call team that monitors your subgraph for issues.

Since we use dedicated indexers for each customer, you are guaranteed stable performance for indexing and can see near-realtime indexing latencies.

Direct Indexing

We will be releasing support for direct indexing of blockchain events and transactions soon.

Transforming Data


Goldsky allows you to create views on top of your raw indexed data. This is the recommended layer for business logic that converts raw on-chain events to formats that are meaningful to your app's operation.

A defined view is always kept up to date with extremely low latencies, allowing you to use them as a source of truth for your app. They are also easy to iterate on, with very low re-indexing times and the ability to do previews.

Views are described through Streaming SQL (docs coming soon), and support for custom lambdas is coming soon. If you use a database like PostgreSQL, Goldsky views are much like PostgreSQL materialized views, but with fewer restrictions.

  • Views can be accessed as GraphQL entities
  • Views are historical - any new view can be applied to past data as well in minutes.
  • Views are cached by default, allowing for instantaneous querying.
  • Views are kept up to date in real-time as new blocks come in from various networks.

You can use views to:

  • Join subgraphs or streams running in multiple networks together and query them with one query.
  • Do complex filtering. For example:
    • getting all limit orders that do not have a corresponding trade for a DeFi platform.
    • getting NFTs that have been traded more than 5 times.
  • Create a view for suspicious data for fraud/data corruption detection. Combined with webhooks, this can be an alert system for many use-cases.
  • Aggregate a stream of events into abstract objects. For example:
    • Forming an ‘Account’ object form a series of user actions.
    • Summing up trades into a live-updated balance, with the ability to change anytime.

📘

Views are versioned and immutable. A typical workflow will have you creating new views and deleting old ones.

Querying


We offer two options, the Legacy Subgraph API and the Goldsky GraphQL.

Legacy Subgraph API

We support a drop-in compatible API for any application that already uses The Graph. This API automatically scales per load and multi-zone for reliability. Each user gets their own dedicated autoscaling instances to increase reliability.

This API can only serve straight subgraph data - views won’t be reflected.

Goldsky GraphQL

This GraphQL API also autoscales and deploys across multiple zones. Compared to the Subgraph API it has more powerful aggregations and filters, but as a result, the schema is different.

  • Supports all transformation results (views) as GraphQL entities
  • Enables aggregations (sum, min, max, etc) on GraphQL entities. Example:
// Query to get a count of big trades.
query GetBigTradeNum {
  views_account_aggregate(where: {collateral_volume: {_gte: "10000000"}}) {
    aggregate {
      sum {
        num_trades
      }
    }
  }
}
// Response:
{
  "data": {
      "views_account_aggregate": {
      "aggregate": {
        "sum": {
          "num_trades": 1321402
        }
      }
    }
  }
}
  • Faster query times with more responsive auto-scaling.

Webhooks


Any view you can access through Goldsky GraphQL can also be emitted to webhooks. These webhooks will be emitted whenever there is an insert, update, or delete to the entity.

Exports


We support ETL to almost anything you can think of, including S3, data warehouses, even directly to your transactional database. Since there can be some edge cases when dealing with blockchain inconsistencies like reorgs, our team will work with you to find the best ETL solution for your goal. If you're an early access customer, contact your specialist for details.


Did this page help you?