---
title: "MongoDB — Cross-Collection ObjectId Reference Confusion"
aliases: [mongodb-objectid-wrong-collection, mongodb-cross-collection-id, mongodb-silent-null-lookup]
tags: [mongodb, debugging, python, fastapi, gotcha, silent-failure]
sources:
  - "daily/2026-04-30.md"
created: 2026-04-30
updated: 2026-04-30
---

# MongoDB — Cross-Collection ObjectId Reference Confusion

When a document field stores a reference to collection A, using that value as a `_id` lookup in collection B silently returns `None` — MongoDB finds no matching document and returns nothing, causing the calling code to proceed with null data (silent failure, not an exception).

## Key Points

- **MongoDB returns `None` (not an error) for a valid ObjectId that doesn't exist in the queried collection** — there's no `404` or exception; `find_one` simply returns `None`
- **The failure is completely silent** if the calling code doesn't handle `None` explicitly, the function skips the operation without error
- **Common pattern:** a job stores `client_id` (reference to `db.clients`) but code accidentally looks it up in `db.users` — every lookup returns `None`, feature silently never works
- **Diagnosis:** add a `print/log` right after the `find_one` to confirm what was returned before assuming the downstream logic is the bug
- **Fix:** look up in the correct collection, or query the target collection using a field that references the ID (e.g., `db.users.find({"pm_client_ids": client_id})`)

## Details

### The Silent Failure Pattern

```python
# job document: {"_id": ObjectId("abc..."), "client_id": ObjectId("xyz..."), ...}
# "client_id" is an ObjectId from db.clients collection

# ❌ WRONG — looking up client_id in db.users where it doesn't exist
user = await db.users.find_one({"_id": client_id})
# Returns None — ObjectId "xyz..." doesn't exist in users collection
# No error raised, user is None

if user:
    send_email(user["email"])  # ← never executes — email silently never sent
```

The function exits normally (no exception), logging shows it "ran", but the email was never sent.

### The Fix: Query the Right Collection

```python
# ✅ Option 1: look up in the correct collection first
client = await db.clients.find_one({"_id": client_id})
if not client:
    logger.warning(f"Client {client_id} not found")
    return

# Then find users who belong to this client
users = await db.users.find({"pm_client_ids": client_id}).to_list(None)
```

```python
# ✅ Option 2: query users by their reference to the client
users = await db.users.find({
    "pm_client_ids": client_id,
    "role": {"$in": ["pm", "production"]}
}).to_list(None)

# Fallback if no users matched
if not users:
    users = await db.users.find({"role": "admin"}).to_list(None)
```

### Why This Happens

MongoDB ObjectIds are just 12-byte identifiers with no type information attached — they don't know which collection they "belong to". An ObjectId `xyz...` from `db.clients` is structurally identical to an ObjectId `xyz...` from `db.users`. Python code that passes one where the other is expected will not get a type error; it will get `None` from the database.

In relational databases (Postgres), a foreign key constraint would catch this at the DB level. MongoDB has no such constraint.

### Detection

Add an assertion or explicit check right after `find_one`:

```python
user = await db.users.find_one({"_id": client_id})
assert user is not None, f"BUG: expected user for {client_id}, got None — check collection"
```

Or log immediately:

```python
logger.debug(f"Looking up user by _id={client_id}: found={user is not None}")
```

## Related Concepts

- [[wiki/concepts/multitenant-fail-open-authz]] — another silent null return pattern where `None` leads to fail-open behavior
- [[wiki/concepts/pydantic-v2-alias-id-gotcha]] — related Pydantic/MongoDB ID mapping pitfalls

## Sources

- [[daily/2026-04-30.md]] — `notify.py` in video-accessibility: `db.users.find_one({"_id": client_id})` returned `None` because `client_id` references `db.clients`, not users; fix was `db.users.find({"pm_client_ids": client_id})` with fallback to admin users
