# Pipelines Pipelines are recurring workflows that run automatically. Each pipeline pairs a trigger, which decides what makes it run, with a workflow, which defines what runs. ## What Pipelines Offer ### Orchestrate data tables Combine data from sources like SEC filings, news, prices, and company metadata. Filter, join, and add LLM-generated classification columns, then output rows into persistent tables called LiveArtifacts that accumulate over time. ### Build live dashboards Visualize a pipeline's accumulated output as charts/tables on a dashboard so trends are visible at a glance. ### Monitor websites Describe in plain English what to watch, such as FDA press releases or a competitor's pricing page, and have a workflow run when new matching content appears. ## Triggers - **Data** - fires when new rows match your filters in a source table - **Time** - runs on a schedule, such as every weekday at 9am - **Web** - fires when monitored sites publish new matching content - **Manual** - only runs when you start it ## Example Pipelines - "Daily 9am summary of overnight earnings reports for my watchlist" (Time trigger) - "Score every new 8-K filing from S&P 500 companies for materiality" (Data trigger -> LiveArtifact -> dashboard) - "Notify me when the FDA announces a new drug approval" (Web trigger) ## Use via the API Pipelines are created and edited through the Pipeline Builder agent. Create a conversation with `POST /api/v1/conversations` and set `agentType` to `pipeline-builder`: ```json { "prompt": "Build a pipeline that runs every weekday at 9am and summarizes overnight earnings reports for AAPL, MSFT, and NVDA.", "agentType": "pipeline-builder" } ``` To modify an existing pipeline, including its dashboards, workflows, or LiveArtifacts, start a new `pipeline-builder` conversation with `pipelineId` set to that pipeline and describe the change you want. The agent has the tools to edit the pipeline's components in place. You can also read and run pipelines directly: - `GET /api/v1/pipelines` lists the pipelines you created. Read one with `GET /api/v1/pipelines/{pipeline_id}`. Rename it with `PATCH`, and archive or restore it with `POST .../archive` and `POST .../unarchive`. - `POST /api/v1/pipelines/{pipeline_id}/runs` runs the whole pipeline once, from the entry points of its active version that have work, and returns the started `workflowRunIds` (none when its sources have no new event). `POST /api/v1/pipelines/{pipeline_id}/triggers/pause` and `.../triggers/resume` pause and resume its automatic triggers, as Pause and Resume do in the app. - `GET /api/v1/pipelines/{pipeline_id}/workflows` lists the workflows in the pipeline's active version. `POST /api/v1/workflows/{workflow_id}/runs` starts a run; pass `entryPoint` to start from a specific trigger. Follow the run with `GET /api/v1/workflow-runs/{workflow_run_id}`. - `GET /api/v1/pipelines/{pipeline_id}/live-artifacts` lists the pipeline's tables. Read rows with `GET /api/v1/live-artifacts/{live_artifact_id}/data` (`limit` up to 10,000, paged with `offset`). `GET /api/v1/live-artifacts/{live_artifact_id}/stream` sends Server-Sent Events when the table changes: each `live_artifact` event carries the table's metadata, such as `updated_at` and `row_count`, not its rows, so read new rows from `/data` when it arrives. - `GET /api/v1/pipelines/{pipeline_id}/artifacts` and `GET /api/v1/pipelines/{pipeline_id}/dashboards` list its reports, files, and dashboards.