Skip to content

Datastream Service

Overview

Datastream is a TypeScript/NodeJS-based application designed for streaming external data:

  • index price via Chainlink WS Direct Streams API
  • SVI IV model parameters via Block Scholes WS API
  • interest rate via Block Scholes WS API

Technical Specifications

Core Technology Stack

  • Runtime: NodeJS >= 22
  • Primary Frameworks:
  • ExpressJS 4.21.2
  • Service Type: Streaming Service
  • Version: 0.0.1

Dependencies

Getting Started

Prerequisites

  • Python 3.9 - 3.10
  • Poetry package manager
  • Redis instance

Build

yarn nx run datastream:build

Development

# Run tests
yarn nx run datastream:it

# View test coverage
yarn nx run datastream:it --coverage

Service Architecture

Components

  1. Main Application app.ts

  2. Primary service entry point

  3. API endpoint definitions

  4. Starting WS clients

  5. Configs config.ts and config.ts

  6. Environment configuration for data providers

  7. Chainlink Client indexPrice.ts

  8. Chainlink index price feeds connection management

  9. Redis data ingestion

  10. Block Scholes Client optionsMarketData.ts

  11. Block Scholes index price feeds connection management

  12. Option expirations management (re-subscribing to new feeds)
  13. Redis data ingestion

Infrastructure

Deployment

  • Containerized deployment (Dockerfile provided)

Key Features

  • Real-time data ingestion
  • Reconnection management
  • WS sharding (per base currency)
  • Health check endpoints
  • Metrics collection (automated via GCP)

Monitoring and Health

Testing and Coverage

  • Comprehensive test coverage setup
  • Mocked Redis and light-weight integration test runner, check jest.setup.ts
  • Reports location:
  • Coverage: ../../coverage/apps/datastream/
  • Test results: ../../reports/apps/datastream/integration/

Monitoring

  • Health check endpoint: /health
  • Expiration endpoint: /expiration
  • Logging configuration
  • Error tracking integration

Integration Points

External Connections

  • Chainlink Direct Stream WS API
  • Block Scholes interest rate and SVI model parameters API
  • GCP IAP-secured endpoints

Internal Dependencies

  • Redis
  • Redis Pub/Sub

Configuration

Service configuration is managed through environment variables and can be overridden using a .env file:

ENV=test
BLOCK_SCHOLES_API_KEY=
CHAINLINK_CLIENT_ID=
CHAINLINK_CLIENT_ID

Redis connection env is .env.example and must be defined separately. Check readme.md

License

Proprietary - All rights reserved

This documentation is generated from the service README. For the most up-to-date information, refer to the original README