> ## Documentation Index
> Fetch the complete documentation index at: https://bulkgrid.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Source Management

> Create, configure, and operate sources so Bulkgrid ingestion stays useful, safe, and maintainable.

Source management is one of the highest-leverage parts of a Bulkgrid integration. Poor source boundaries create noisy retrieval, wasted crawl work, and harder operational debugging.

## What a source is in Bulkgrid

A source is a first-class product object, not just a URL passed into a one-off crawl.

The current source API supports domain-type sources with configuration such as:

* `identifier`
* `label`
* `visibility`
* `source_mode`
* `crawl_config`
* `crawl_interval`
* `custom_interval_minutes`

## Source modes

Current source modes include:

* `discover`
* `selected_pages`

These modes affect how recrawl behaves and how customers should think about source scope.

## Source lifecycle surfaces

For a source, the product supports:

* source details
* source status
* source folders
* source documents
* source changes
* source runs
* manual recrawl

The public API exposes these as dedicated endpoints, not as one overloaded source response.

## What to decide before you crawl

<CardGroup cols={2}>
  <Card title="Domain Scope" icon="database">
    Decide which domains are in scope and which are never allowed.
  </Card>

  <Card title="Path Rules" icon="layers">
    Define which paths are included and which should always be excluded.
  </Card>

  <Card title="Document Links" icon="database">
    Decide whether linked documents belong in the same ingestion flow.
  </Card>

  <Card title="Knowledge Boundaries" icon="layers">
    Keep support, marketing, and internal knowledge separated when needed.
  </Card>
</CardGroup>

## Practical recommendation

Start with a small, high-value source boundary. Expand only after you validate retrieval quality and operational behavior.

For domain sources, Bulkgrid normalizes the identifier to the URL origin when it creates the source. That means `https://docs.example.com/foo` and `https://docs.example.com/bar` are treated as the same source root.

## Related docs

* [Source Discovery](/guides/source-discovery)
* [Source Recrawl and Monitoring](/guides/source-recrawl-and-monitoring)
* [Source Documents and Changes](/guides/source-documents-and-changes)
* [Collections and Access](/guides/collections-and-access)