Skip to content

Poetry Primer

What is Poetry?

Poetry is a dependency management and packaging tool for Python. It allows you to declare the libraries your project depends on, and it will manage (install/update) them for you. In the v4-monorepo, Poetry is used to manage dependencies for Python projects like funding, poseidon, and Python libraries like commons and gleam.

Key Concepts

Project Structure

A Poetry project typically includes:

  • pyproject.toml: Defines project metadata, dependencies, and build settings
  • poetry.lock: Locks dependencies to specific versions
  • [tool.poetry.dependencies] section: Lists project dependencies
  • [tool.poetry.group.dev.dependencies] section: Lists development-only dependencies

Virtual Environments

Poetry automatically creates and manages virtual environments for projects, isolating dependencies from the system Python.

Dependency Resolution

When you add or update dependencies, Poetry resolves all dependencies to find compatible versions, updating the poetry.lock file.

How Poetry is Used in This Repository

In the v4-monorepo, Python projects are managed with Poetry integrated with NX:

  1. NX Integration:

  2. NX provides special executors for Poetry operations

  3. Poetry commands are wrapped in NX targets

  4. Robust poetry script is used for reliability

  5. Project Configuration:

  6. Each Python project has its own pyproject.toml

  7. Python projects are in apps/ (applications) and libs/ (libraries)

  8. Local dependencies reference other Python libraries in the monorepo

  9. Dependency Management:

  10. Dependencies are specified in pyproject.toml
  11. Versions are locked in poetry.lock
  12. Local dependencies use the path directive

Common Poetry Commands in NX

Basic Commands

# Install dependencies for a Python project
nx run funding:install

# Add a dependency
nx run funding:add --name fastapi

# Add a local dependency
nx run funding:add --name commons --local

# Lock dependencies without installing
nx run funding:lock

Advanced Commands

# Update dependencies
nx run funding:update

# Remove a dependency
nx run funding:remove --name fastapi

# Build a Python package
nx run funding:build

Project Configuration

pyproject.toml

The pyproject.toml file configures a Python project, including:

  • Project metadata (name, version, description)
  • Dependencies
  • Development dependencies
  • Build settings
  • Tool-specific configurations

Example from apps/funding/pyproject.toml:

[project]
name = "funding"
version = "0.1.0"
description = "The funding rate service for calculating and settling perpetual funding rates."
authors = []
license = "Proprietary"
readme = "README.md"

[[tool.poetry.packages]]
include = "funding"

[tool.poetry.dependencies]
python = ">=3.9,<3.11"
fastapi = "^0.110.1"
httpx = "^0.27.0"
pydantic-settings = "^2.2.1"
pydantic = { version = "^2.8.2", extras = ["email"] }
uvicorn = { version = "^0.29.0", extras = ["standard"] }

[tool.poetry.dependencies.commons]
path = "../../libs/commons"
develop = true

[tool.poetry.group.dev.dependencies]
pyright = "^1.1.352"
deptry = "^0.14.2"
ruff = "^0.4.4"
black = "^24.3.0"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

Local Dependencies

To use a local Python package from the monorepo:

[tool.poetry.dependencies.commons]
path = "../../libs/commons"
develop = true

This tells Poetry to install the package in development mode (changes in the source reflect immediately).

Robust Poetry Script

The repository includes a robust Poetry script at scripts/python/robust-poetry.sh that:

  • Handles common Poetry failures
  • Implements retry logic
  • Ensures consistent behavior across environments

This script is used by NX targets instead of calling Poetry directly:

"lock": {
  "executor": "nx:run-commands",
  "options": {
    "commands": [
      "bash scripts/python/robust-poetry.sh lock {projectRoot}"
    ],
    "cwd": "{workspaceRoot}"
  }
}

Type Checking with Pyright

Python projects in the repository use Pyright for type checking, which is configured in pyrightconfig.json:

# Run type checking
nx run funding:typecheck

Troubleshooting Poetry

Common Issues

Issue Solution
"Failed to lock dependencies" Run nx run <project>:lock to regenerate lock file
"Cannot resolve dependencies" Check for version conflicts between packages
"Path dependency not found" Ensure path to local package is correct
"Incompatible Python version" Verify Python version constraints in pyproject.toml
"Poetry command failed" Use robust-poetry.sh for added reliability

Poetry Cache

If you're experiencing odd behavior, you can clear Poetry's cache:

poetry cache clear

Learn More