Migrate from ChromaDB to OSYRA Memory (OME)

This guide maps ChromaDB's collection model onto OSYRA Memory (OME) claims. ChromaDB is a vector-store ID-preserving migration, the same family as the Pinecone guide; read that guide first for the full reference, then read this one for the ChromaDB-specific differences.

| Aspect | Value | |---|---| | Migration pattern | Vector-store ID-preserving (id → external_id) | | Source scope | One ChromaDB collection per run | | Source-side write access required | No — client.get_collection is non-mutating | | Fidelity guarantees | Bit-stable round-trip of id, document text (NFC-normalized), source timestamp (if present in metadata) | | Embedding policy | Source vectors discarded; OME re-embeds | | Rollback | Per-claim tombstone (see §8) |

§1 — Prerequisites

Source side (ChromaDB)

ChromaDB ships in three deployment modes, each with different prerequisites.

In all three modes you also need:

• The collection name to migrate. ChromaDB has no namespace hierarchy beyond collections.
• For metadata-rich collections: the list of metadata fields you want preserved (see §3).

OSYRA side (OME)

Identical to the Pinecone prerequisites — workspace ID, a dedicated migration signer DID (rotation-eligible), the osyra:ome:claim:create and osyra:ome:bundle:import OPA scopes, and a tag plan.

Credential handling

Supply credentials through environment variables sourced from a secret store, never as command-line flags:

export CHROMA_SERVER_AUTH_TOKEN="$(aws secretsmanager get-secret-value --secret-id chromadb/migrate --query SecretString --output text)"

§2 — Source-system schema (ChromaDB)

ChromaDB organizes data as collection → item. An item has:

| Source field | Source type | Treatment | |---|---|---| | ids | string (customer-assigned, unique per collection) | Preserved verbatim as claim.external_id | | documents | string (the text; can be null if the customer pre-embedded) | The body field — typically used directly as claim.body | | embeddings | float[N] | Discarded — OME re-embeds | | metadatas | object (string → string/int/float/bool) | Field-mapped (passthrough by default) | | uris (optional, multimodal) | string (URI) | Passthrough into metadata.source_uri; not fetched |

ChromaDB-specific concepts that are not preserved, and why:

| Source concept | Why not preserved | Implication | |---|---|---| | Embedding-function identity | OME has its own embedder | Re-validate retrieval (see §6) | | HNSW index params | OME has its own ANN config | None | | Collection-level metadata (collection.metadata) | Logged to the migration record, not promoted to OME workspace metadata | If you set collection.metadata = {"team": "support"}, it survives only as a migration-record annotation, not on individual claims | | Multimodal blob content (uris dereferences) | URI content is not fetched | If your collection points to S3/HTTP URIs for binary content, the URIs are preserved as strings but the content is not migrated |

Schema discovery:

collection = client.get_collection("my-collection")
print(collection.count())            # total item count
sample = collection.peek(limit=10)   # sample 10 items to see metadata shape

The CLI does this in --dry-run mode and prints the count plus the metadata-key inventory across a sample of items.

§3 — OSYRA target mapping

The mapping follows the same shape as the Pinecone mapping. ChromaDB-specific deltas:

Top-level mapping

| Source field | Source type | OME claim field | OME field type | Transformation | Required/Optional | Notes | |---|---|---|---|---|---|---| | ids[i] | string (customer-assigned) | external_id | string (max 256 bytes UTF-8) | Verbatim | Required | | | documents[i] | string (nullable) | body | string (max 64 KB UTF-8, NFC-normalized) | NFC normalize; reject if > 64 KB | Required if present | Override with --body-field to source the body from a metadata field instead | | metadatas[i].<timestamp_field> | string (ISO-8601) / number (epoch s) | observed_at | int64 (epoch ms UTC) | Parse and convert; fall back to migration-time | Optional | Default field timestamp | | metadatas[i].<type_field> | string | claim.type | enum {memory.fact, memory.observation} | Lookup table; default memory.fact | Optional | | | uris[i] (if present) | string (URI) | metadata.source_uri | string | Verbatim string | Optional | URI content not fetched | | metadatas[i].* (other) | object | metadata.source_metadata.* | JSON object (max 16 KB serialized) | Coerce to string | Optional | | | (implicit) collection name | string | tag | string | Format chromadb:<collection> | Required (auto-derived) | | | embeddings[i] | float[N] | (discarded) | — | — | N/A | OME re-embeds |

Override flags

| Flag | Default | Effect | |---|---|---| | --body-field <name> | (uses documents) | Override: pull body from metadatas[i].<name> instead of documents[i] | | --timestamp-field <name> | timestamp | Which metadata field becomes observed_at | | --type-field <name> | type | Which metadata field becomes claim.type | | --on-missing-document <action> | skip | skip, fail, or body-field:<name> (fall back to a metadata field for the body) | | --mode <mode> | auto | in-process, server, cloud, or auto (detected from connection params) |

documents vs metadatas[i].text

Many ChromaDB collections store the body text in both documents and a metadatas[i].text (or similar) field. The default behavior pulls from documents. If your collection stores body text exclusively in metadata (for example, you embedded only a hash in documents), use --body-field <metadata-key>.

§4 — CLI walkthrough

export OSYRA_WORKSPACE_ID="ws_abc123"
export OSYRA_SIGNER_DID="did:osyra:ws:abc123:signer:migration"
 
osyra migrate chromadb \
  --mode in-process \
  --persist-dir ./chroma-data/ \
  --source-collection my-collection \
  --dry-run

export CHROMA_SERVER_URL="http://chroma.internal:8000"
export CHROMA_SERVER_AUTH_TOKEN="$(aws secretsmanager get-secret-value --secret-id chromadb/migrate --query SecretString --output text)"
 
osyra migrate chromadb \
  --mode server \
  --source-collection my-collection \
  --dry-run

# This collection stores body text in metadatas[i].full_text, not in documents
osyra migrate chromadb \
  --mode server \
  --source-collection legal-docs \
  --body-field full_text \
  --on-missing-document body-field:full_text \
  --batch-size 1000 \
  --output-dir ./chromadb-legal-migrate/

export CHROMA_CLOUD_API_KEY="$(aws secretsmanager get-secret-value --secret-id chromadb/cloud --query SecretString --output text)"
export CHROMA_CLOUD_URL="https://api.trychroma.com"
 
osyra migrate chromadb \
  --mode cloud \
  --source-collection prod-knowledge \
  --batch-size 1000 \
  --output-dir ./chromadb-cloud-migrate/

Resume semantics and staging strategy are identical to the Pinecone walkthrough.

§5 — Expected output

osyra-migrate chromadb
Workspace:           ws_abc123
Signer DID:          did:osyra:ws:abc123:signer:migration
Source mode:         in-process
Source persist-dir:  ./chroma-data/
Source collection:   my-collection
Source items:        4,219 (collection.count())
Body source:         documents (no override)
Batch size:          1000
Bundles planned:     5
Output dir:          ./chromadb-migrate-full/
 
[1/5] Extracting batch ... 1000/1000 items
[1/5] Building claims ... 1000/1000 claims (0 skipped)
[1/5] Signing bundle ... CID: bafy2bzaceabc...001
[1/5] Imported  bundle ... 1000/1000 claims accepted
 
...
 
[5/5] Extracting batch ... 219/219 items
[5/5] Signing bundle ... CID: bafy2bzaceabc...005
[5/5] Imported  bundle ... 219/219 claims accepted
 
Summary:
  Total claims:      4,219
  Total bundles:     5
  Total skipped:     0

§6 — Caveats

1. Embedding-function identity

ChromaDB collections are typically created with a specific embedding function (the default is all-MiniLM-L6-v2 from sentence-transformers). If your collection was embedded with a different function than your OME workspace's embedder, retrieval results will shift after migration. ChromaDB makes the embedding-function choice explicit (it is a constructor argument), so confirm which function your collection used before validating retrieval. See the Pinecone embedding caveat for the underlying rationale.

2. In-process mode requires Python compatibility

In-process ChromaDB pins a particular chromadb library version. The migration CLI ships with a tested version of the chromadb client; if your source data was written with a substantially older or newer version, the dry-run warns (and may fail to read the persistence files). Test mode-1 dry-run before scheduling a maintenance window.

3. Multimodal URIs are not dereferenced

If your collection uses uris for multimodal data (images, audio), the URIs survive in claim.metadata.source_uri but the binary content is not fetched. Applications that previously dereferenced these URIs against ChromaDB's content cache need to point at the original URI source after migration.

4. Collection-level metadata is not promoted

collection.metadata (the dict attached to the collection itself) is logged to the migration record file but not promoted to OME workspace or tag metadata. If you used this field for important annotations, capture them manually before migration.

5. Sequencing, idempotency, rate limits, GDPR

These behave identically to Pinecone — see the Pinecone caveats.

§7 — Round-trip verification

The protocol is the same as the Pinecone verification protocol, with these substitutions:

• Source id → ChromaDB ids[i].
• Pinecone metadata.description → ChromaDB documents[i] (or the field configured with --body-field).
• --tag is chromadb:<collection>.

ChromaDB-specific check

If you used --body-field <metadata-key> (sourcing the body from a metadata field instead of documents), verify that the resulting claim.body matches source.metadatas[i].<metadata-key> — not source.documents[i]. The default pulls from documents; the override pulls from metadata; conflating the two is a common verification-script mistake.

Acceptance criterion

At least 99.9% of sampled IDs round-trip with byte-equal body text and an exact external_id match.

§8 — Rollback

Identical to the Pinecone rollback. Per-claim tombstone; the 24-hour observation window applies; the source ChromaDB collection is untouched at all times.

ChromaDB-specific rollback note

If you migrated from in-process mode, the source ./chroma-data/ persistence directory is still present and usable — rollback (the OME-side tombstone) leaves your local ChromaDB ready for a cutover-revert without re-importing.

§9 — Troubleshooting

1. ChromaDB version mismatch (in-process mode)

• Symptom: --mode in-process fails with Failed to load persistence: schema version 'X' requires chromadb library Y, found Z.
• Likely cause: the CLI ships a pinned chromadb client version that does not match your source persistence schema.
• Diagnostic: pip show chromadb in the environment that wrote the data.
• Fix: upgrade the source data to a compatible version, or use server mode with a compatible-version ChromaDB server in front of your data.

2. Document field empty on N rows

• Symptom: non-zero skip count; .skipped.jsonl shows reason: "documents[i] is null".
• Likely cause: some items were added with embeddings but no documents (a pattern where the customer pre-computes embeddings and stores only metadata).
• Diagnostic: collection.peek(limit=10) and count items where documents[i] is None.
• Fix: re-run with --on-missing-document body-field:<metadata-field-with-text> to fall back to a metadata field for body text on those rows.

3. Server-mode auth fails

• Symptom: --mode server fails immediately with 401 Unauthorized.
• Likely cause: the server uses an auth provider, but CHROMA_SERVER_AUTH_TOKEN is missing or wrong.
• Diagnostic: curl -H "X-Chroma-Token: $CHROMA_SERVER_AUTH_TOKEN" $CHROMA_SERVER_URL/api/v1/heartbeat.
• Fix: set or correct the env var.

4. Signer DID not authorized

Same as the Pinecone signer-DID troubleshooting.

5. Round-trip verification fails on body diff

Same as the Pinecone body-diff troubleshooting. ChromaDB-specific note: if you used --body-field <metadata-key>, diff against source.metadatas[i].<key>, not source.documents[i]. Mis-targeting the source of truth is the most common verification-script mistake for --body-field overrides.

Related guides

• Vector-store reference: Pinecone
• Vector-store delta: Weaviate
• Agent-framework migrations: LangGraph, CrewAI, AutoGen