Back to Mana

Pipelines

Read as MarkdownAgent documentation indexOpenAPI JSON

Pipelines are recurring workflows that run automatically. Each pipeline pairs a trigger, which decides what makes it run, with a workflow, which defines what runs.

What Pipelines Offer

Orchestrate data tables

Combine data from sources like SEC filings, news, prices, and company metadata. Filter, join, and add LLM-generated classification columns, then output rows into persistent tables called LiveArtifacts that accumulate over time.

Build live dashboards

Visualize a pipeline's accumulated output as charts/tables on a dashboard so trends are visible at a glance.

Monitor websites

Describe in plain English what to watch, such as FDA press releases or a competitor's pricing page, and have a workflow run when new matching content appears.

Triggers

  • Data - fires when new rows match your filters in a source table
  • Time - runs on a schedule, such as every weekday at 9am
  • Web - fires when monitored sites publish new matching content
  • Manual - only runs when you start it

Example Pipelines

  • "Daily 9am summary of overnight earnings reports for my watchlist" (Time trigger)
  • "Score every new 8-K filing from S&P 500 companies for materiality" (Data trigger -> LiveArtifact -> dashboard)
  • "Notify me when the FDA announces a new drug approval" (Web trigger)

Use via the API

Pipelines are created and edited through the Pipeline Builder agent. Create a conversation with POST /api/v1/conversations and set agentType to pipeline-builder:

{
  "prompt": "Build a pipeline that runs every weekday at 9am and summarizes overnight earnings reports for AAPL, MSFT, and NVDA.",
  "agentType": "pipeline-builder"
}

To modify an existing pipeline, including its dashboards, workflows, or LiveArtifacts, start a new pipeline-builder conversation with pipelineId set to that pipeline and describe the change you want. The agent has the tools to edit the pipeline's components in place.

You can also read and run pipelines directly:

  • GET /api/v1/pipelines lists the pipelines you created. Read one with GET /api/v1/pipelines/{pipeline_id}. Rename it with PATCH, and archive or restore it with POST .../archive and POST .../unarchive.
  • POST /api/v1/pipelines/{pipeline_id}/runs runs the whole pipeline once, from the entry points of its active version that have work, and returns the started workflowRunIds (none when its sources have no new event). POST /api/v1/pipelines/{pipeline_id}/triggers/pause and .../triggers/resume pause and resume its automatic triggers, as Pause and Resume do in the app.
  • GET /api/v1/pipelines/{pipeline_id}/workflows lists the workflows in the pipeline's active version. POST /api/v1/workflows/{workflow_id}/runs starts a run; pass entryPoint to start from a specific trigger. Follow the run with GET /api/v1/workflow-runs/{workflow_run_id}.
  • GET /api/v1/pipelines/{pipeline_id}/live-artifacts lists the pipeline's tables. Read rows with GET /api/v1/live-artifacts/{live_artifact_id}/data (limit up to 10,000, paged with offset). GET /api/v1/live-artifacts/{live_artifact_id}/stream sends Server-Sent Events when the table changes: each live_artifact event carries the table's metadata, such as updated_at and row_count, not its rows, so read new rows from /data when it arrives.
  • GET /api/v1/pipelines/{pipeline_id}/artifacts and GET /api/v1/pipelines/{pipeline_id}/dashboards list its reports, files, and dashboards.