Firestore Get All Documents Where a Specific Field Exists or Does Not Exist
In Firebase’s Firestore database, you may want to query and get all the documents in a collection that either contains a specific field (or key) or does not contain a specific field (or key). By contain, I mean all the documents where a specific field either exists, or doesn’t exists.
Query and get all documents that DO NOT have a certain field
Let’s first tackle this problem. Imagine you wanted to query and retrieve all the documents from the collection
posts that do not have the
tags field. This is not possible in Firestore currently. We should understand why.
In Firestore, all queries are served off indexes for performance reasons. Most of these indexes on single columns are automatically created for us, which is why when we get started, we don’t realise it. So if we create documents in the
posts collection with fields like
title, description, comments, tags, created_by, created_at and then query on the collection with a
firestore.collection('posts').where('created_by', '==', 'xxx'), then behind the scenes, firebase uses an “automatically maintained” index which is used to serve the results. This fetching process is real fast because of the indexes.
So if a document does not have a specific field, that means it never made it to the index for that field. Hence it won’t be returned when we performed a
collection.where('field', '==', 'xxx').
Query and Retrieve all documents that DO have a specific field
This is possible, but in a way that may feel weird to some. From the Firestore documentation on order and limit data:
orderBy()clause also filters for existence of the given field. The result set will not include documents that do not contain the given field.
So if you want to query all the
posts documents that do not contain the
tags field, all you have to do is this:
firestore.collection('posts').orderBy('tags') // or firestore.collection('posts').orderBy('tags', 'desc')
You will have a single index automatically available on
tags, so it’ll just work!
In some cases like when you’re trying to query a collection group for a similar use-case, you will have to manually or explicitly create an index under the collection group scope. This is because firebase does not automatically create composite indexes and collection group scope indexes. The best way to do it is, let your code run and execute the query which will eventually cause the firebase SDK to throw an error due to a missing index. The error message will contain a link to create the index manually, so just copy paste that link in the browser and make it work.
The other option is going to the Firebase console and doing it in
Firestore Database > Indexes by yourself. Do this if you have a good understanding of single indexes, composite indexes, collection groups and index modes.
Bonus: There’s an alternate “hack” to the
orderBy solution. To check for documents that do contain a field, you could do a
not "value" check, for instance a
firestore.collection('posts').where('tags', '!=', null)
!= check excludes documents where the specified field does not exist but it also excludes documents where the specific field is not equal to the value passed. If you think about it, it poses a different meaning from what we want to achieve, but it can be used as long as you understand this and think for yourself whether it is ok for your use case or not. From the docs:
not-inqueries exclude documents where the given field does not exist.