211

I am wondering if it's possible to get multiple documents by a list of ids in one round trip (network call) to the Firestore database.

10
  • 6
    You seem to assume that the roundtrips are causing performance problems in your app. I wouldn't assume that. Firebase has a history of performing fine in such cases, since it pipelines the requests. While I haven't checked how Firestore behaves in this scenario, I'd love to see proof of a performance problem before assuming that it exists. Commented Oct 13, 2017 at 3:22
  • 5
    Let's say I need documents a, b, c to do something. I requests for all three in parallel in separate requests. a takes 100ms, b takes 150ms, and c takes 3000ms. As result, I need to wait for 3000ms to do the task. It's going to be max of them. It's going to be riskier when the number of documents to fetch is large. Depends on network status, I think this can become a problem. Commented Oct 13, 2017 at 8:26
  • 2
    Wouldn't sending them all as a single SELECT * FROM docs WHERE id IN (a,b,c) take the same amount of time though? I don't see the difference, since the connection is established once and the rest is pipelined over that. The time (after the initial establishing of the connection) is the load time of all documents + 1 round trip, same for both approaches. If it behaves different for you, can you share a sample (as in my linked question)? Commented Oct 13, 2017 at 13:19
  • 2
    Yes, I've read your answer, but it's still unclear whether there will be multiple network calls or just one. It sounds like there will be n network calls in parallel for n items rather than just one network call that does n queries at once. Commented Oct 14, 2017 at 2:20
  • 1
    They're not really in parallel, but pipelined. But the result is practically the same. Again: I haven't yet checked that is what Firestore does, but I'd do some measurements before assuming it takes N*(read_time+transfer_time+latency). If they're pipelined, it's closer to N*(transfer_time)+read_time+latency. Commented Oct 14, 2017 at 3:08

17 Answers 17

176

if you're within Node:

https://github.com/googleapis/nodejs-firestore/blob/master/dev/src/index.ts#L978

/**
* Retrieves multiple documents from Firestore.
*
* @param {...DocumentReference} documents - The document references
* to receive.
* @returns {Promise<Array.<DocumentSnapshot>>} A Promise that
* contains an array with the resulting document snapshots.
*
* @example
* let documentRef1 = firestore.doc('col/doc1');
* let documentRef2 = firestore.doc('col/doc2');
*
* firestore.getAll(documentRef1, documentRef2).then(docs => {
*   console.log(`First document: ${JSON.stringify(docs[0])}`);
*   console.log(`Second document: ${JSON.stringify(docs[1])}`);
* });
*/

This is specifically for the server SDK

UPDATE: Cloud Firestore Now Supports IN Queries!

myCollection.where(firestore.FieldPath.documentId(), 'in', ["123","456","789"])
Sign up to request clarification or add additional context in comments.

13 Comments

For anyone looking to call this method with a dynamically generated array of document references, you can do it like this: firestore.getAll(...arrayOfReferences).then()
I'm sorry @KamanaKisinga ... I haven't done any firebase stuff in almost a year and can't really help at this time (hey look, I actually posted this answer one year ago today!)
The client-side SDKs now also offer this functionality. see jeodonara's answer for an example: stackoverflow.com/a/58780369
warning: the in filter is limitted to 10 items currently. So you'll probably find out it's useless when you're about to hit production.
actually you need to use firebase.firestore.FieldPath.documentId() and not 'id'
|
68

In practise you would use firestore.getAll like this

async getUsers({userIds}) {
    const refs = userIds.map(id => this.firestore.doc(`users/${id}`))
    const users = await this.firestore.getAll(...refs)
    console.log(users.map(doc => doc.data()))
}

or with promise syntax

getUsers({userIds}) {
    const refs = userIds.map(id => this.firestore.doc(`users/${id}`))
    this.firestore.getAll(...refs).then(users => console.log(users.map(doc => doc.data())))
}

3 Comments

this should really be the selected answer because it lets you use more than 10 ids
This did work! Thanks. Where is the documentation on this? i looked for getAll and could not find it anywhere.
@TravRob this may be available in certain flavors of Firebase, like Node, but it's definitely not in the JavaScript API.
47

They have just announced this functionality, https://firebase.googleblog.com/2019/11/cloud-firestore-now-supports-in-queries.html .

Now you can use queries like, but mind that the input size can't be greater than 10.

userCollection.where('uid', 'in', ["1231","222","2131"])

5 Comments

There is a whereIn query rather than where. And I don't know how to design query for multiple documents from a list of ids of documents which belongs to specific collection. Please help.
@Compileerrorend could you try this? db.collection('users').where(firebase.firestore.FieldPath.documentId(), 'in',["123","345","111"]).get()
thank you , especially for the firebase.firestore.FieldPath.documentId()
@jeadonara in case of input array greater than 10, what should i use?
@RameshVishnoi you can use Promise.all() ( developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/…) .
32

With Firebase Version 9 (Dec, 2021 Update):

You can get multiple documents by multiple ids in one round-trip using "documentId()" and "in" with "where" clause:

import {
  query,
  collection,
  where,
  documentId,
  getDocs
} from "firebase/firestore";

const q = query(
  collection(db, "products"),
  where(documentId(), "in", 
    [
      "8AVJvG81kDtb9l6BwfCa", 
      "XOHS5e3KY9XOSV7YYMw2", 
      "Y2gkHe86tmR4nC5PTzAx"
    ]
  ),
);

const productsDocsSnap = await getDocs(q);

productsDocsSnap.forEach((doc) => {
  console.log(doc.data()); // "doc1", "doc2" and "doc3"
});

3 Comments

I get documentId is undefinted. This seems like this should be soooo easy, but I'm finding only complex workarounds... building an array of refs, etc.
@KirkRoss can't think why, it definitely exists in v9+
Keep in mind that in has a limit of 30 items.
13

If you are using flutter, you can do the following:

Firestore.instance.collection('your_collection_name')
  .where(FieldPath.documentId, whereIn:["list", "of", "document", "ids"])
  .getDocuments();

This will return a Future containing List<DocumentSnapshot> which you can iterate as you feel fit.

3 Comments

'list containing multiple document IDs' can be maximum 10 items, correct?
@KrishnaShetty I am wondering the same thing. For the other answers in this same question it looks like if you use "where-in" to filter them, this limitation disappears (?)
@KrishnaShetty It looks like it is correct: stackoverflow.com/a/64603480/7927429
12

You could use a function like this:

function getById (path, ids) {
  return firestore.getAll(
    [].concat(ids).map(id => firestore.doc(`${path}/${id}`))
  )
}

It can be called with a single ID:

getById('collection', 'some_id')

or an array of IDs:

getById('collection', ['some_id', 'some_other_id'])

Comments

11

No, right now there is no way to batch multiple read requests using the Cloud Firestore SDK and therefore no way to guarantee that you can read all of the data at once.

However as Frank van Puffelen has said in the comments above this does not mean that fetching 3 documents will be 3x as slow as fetching one document. It is best to perform your own measurements before reaching a conclusion here.

12 Comments

The thing is that I want to know theoretical limits to Firestore's performance before migrating to Firestore. I don't want to migrate and then realize that it's not good enough for my use case.
Hi, there is also a considaration of cose here. Let say I have stored list of all my friend's IDs and the number is 500. I can get the list in 1 read cost but in order to display their Name and photoURL, it will cost me 500 reads.
If you're trying to read 500 documents, it takes 500 reads. If you combine the information that you need from all 500 documents into a single extra document, it only takes one read. That's called sort of data duplication is quite normal in most NoSQL database, including Cloud Firestore.
@FrankvanPuffelen For instance, in mongoDb, you can use ObjectId like this stackoverflow.com/a/32264630/648851.
Like @FrankvanPuffelen said, data duplication is pretty common in NoSQL database. Here you have to ask yourself how often these data are required to be read, and how up-to-date they need to be. If you do store 500 of users information, let's say their name + photo + id, you can get them in one read. But if you need them up-to-date, you'll probably have to use a cloud function to update these references each time a user update their name / photo, therefore running a cloud function + doing some write operations. There is no "right" / "better" implementation, it just depends on your use case.
|
5

Surely the best way to do this is by implementing the actual query of Firestore in a Cloud Function? There would then only be a single round trip call from the client to Firebase, which seems to be what you're asking for.

You really want to be keeping all of your data access logic like this server side anyway.

Internally there will likely be the same number of calls to Firebase itself, but they would all be across Google's super-fast interconnects, rather than the external network, and combined with the pipelining which Frank van Puffelen has explained, you should get excellent performance from this approach.

2 Comments

Storing the implementation in a Cloud Function is the right decision in some cases where you have complex logic, but probably not in the case where you just want to merge a list with multiple id's. What you lose is the client side caching and standardized return formatting from regular calls. This caused more performance issues than it solved in some cases in my apps when I used the approach.
If you don't call the function regularly (for example about every 30 seconds) the Firebase Function will cold start causing it to take at least 4-8 seconds to return a simple set of data from the Firestore. This is definitly NOT what you want.
4

You can perform an IN query with the document IDs (up to ten):

import {
    query,
    collection,
    where,
    getDocs,
    documentId,
} from 'firebase/firestore';

export async function fetchAccounts(
    ids: string[]
) {
    // use lodash _.chunk, for example
    const result = await Promise.all(
        chunk(ids, 10).map(async (chunkIds) => {
            const accounts = await getDocs(
                query(
                    collection(firestore, 'accounts'),
                    where(documentId(), 'in', chunkIds)
                ));
            return accounts.docs.filter(doc => doc.exists()).map(doc => doc.data());
        })
    );
    return result.flat(1);
}

Comments

3

Here's how you would do something like this in Kotlin with the Android SDK.
May not necessarily be in one round trip, but it does effectively group the result and avoid many nested callbacks.

val userIds = listOf("123", "456")
val userTasks = userIds.map { firestore.document("users/${it!!}").get() }

Tasks.whenAllSuccess<DocumentSnapshot>(userTasks).addOnSuccessListener { documentList ->
    //Do what you need to with the document list
}

Note that fetching specific documents is much better than fetching all documents and filtering the result. This is because Firestore charges you for the query result set.

1 Comment

Works nicely, exactly what I was looking for!
3

For some who are stucked in same problem here is a sample code:

List<String> documentsIds = {your document ids};

FirebaseFirestore.getInstance().collection("collection_name")
.whereIn(FieldPath.documentId(), documentsIds).get().addOnCompleteListener(new OnCompleteListener<QuerySnapshot>() {
            @Override
            public void onComplete(@NonNull Task<QuerySnapshot> task) {
                if (task.isSuccessful()) {
                     for (DocumentSnapshot document : Objects.requireNonNull(task.getResult())) {
                        YourClass object = document.toObject(YourClass.class);
                        // add to your custom list
                    }   
                }
                
            }
        }).addOnFailureListener(new OnFailureListener() {
            @Override
            public void onFailure(@NonNull Exception e) {
                e.printStackTrace();
            }
        });

Comments

3

if you are using the python firebase admin sdk this is how you query for multiple documents using their uids

from firebase_admin import firestore
import firebase_admin
from google.cloud.firestore_v1.field_path import FieldPath

app = firebase_admin.initialize_app(cred)
client = firestore.client(app)

collection_ref = client.collection('collection_name')
query = collection_ref.where(FieldPath.document_id(), 'in', listOfIds)
docs = query.get()

for doc in docs:
   print(doc.id, doc.to_dict())

Instead of importing FieldPath you can also simply use the string __name__. Now your query will be collection_ref.where('__name__', 'in', listOfIds)

Comments

2

I hope this helps you, it works for me.

getCartGoodsData(id) {

    const goodsIDs: string[] = [];

    return new Promise((resolve) => {
      this.fs.firestore.collection(`users/${id}/cart`).get()
        .then(querySnapshot => {
          querySnapshot.forEach(doc => {
            goodsIDs.push(doc.id);
          });

          const getDocs = goodsIDs.map((id: string) => {
            return this.fs.firestore.collection('goods').doc(id).get()
              .then((docData) => {
                return docData.data();
              });
          });

          Promise.all(getDocs).then((goods: Goods[]) => {
            resolve(goods);
          });
        });
    });
  }

Comments

2

For the ones who want to do it using Angular, here is an example:

First some library imports are needed: (must be preinstalled)

import * as firebase from 'firebase/app'
import { AngularFirestore, AngularFirestoreCollection } from '@angular/fire/firestore'

Some configuration for the collection:

yourCollection: AngularFirestoreCollection;

constructor(
    private _db : AngularFirestore,
) { 
    // this is your firestore collection
    this.yourCollection = this._db.collection('collectionName');
}

Here is the method to do the query: ('products_id' is an Array of ids)

getProducts(products_ids) {
    var queryId = firebase.firestore.FieldPath.documentId();
    this.yourCollection.ref.where(queryId, 'in', products_ids).get()
        .then(({ docs }) => {
            console.log(docs.map(doc => doc.data()))
        })
}

1 Comment

You're the only one who's actually wrote down what goes after .get().then(). { docs } works for me but docs alone does not!
1

Yes, it is possible. Sample in .NET SDK for Firestore:

/*List of document references, for example:
    FirestoreDb.Collection(ROOT_LEVEL_COLLECTION).Document(DOCUMENT_ID);*/
    List<DocumentReference> docRefList = YOUR_DOCUMENT_REFERENCE_LIST;
    
    // Required fields of documents, not necessary while fetching entire documents
    FieldMask fieldMask = new FieldMask(FIELD-1, FIELD-2, ...);
    
    // With field mask
    List<DocumentSnapshot> documentSnapshotsMasked = await FirestoreDb.GetAllSnapshotsAsync(docRefList, fieldMask);
    
    // Without field mask
    List<DocumentSnapshot>documentSnapshots = await FirestoreDb.GetAllSnapshotsAsync(docRefList);

Documentation in .NET:

  1. Get all snapshots

  2. Field mask

3 Comments

While this link may answer the question, it is better to include the essential parts of the answer here and provide the link for reference. Link-only answers can become invalid if the linked page changes. - From Review
Yes, this totally answers the question. I have also added an example code snippet along with the reference links.
How do I build this list of document references, if I had a list of document id's?
0

This doesn't seem to be possible in Firestore at the moment. I don't understand why Alexander's answer is accepted, the solution he proposes just returns all the documents in the "users" collection.

Depending on what you need to do, you should look into duplicating the relevant data you need to display and only request a full document when needed.

Comments

-1

The best you can do is not use Promise.all as your client then must wait for .all the reads before proceeding.

Iterate the reads and let them resolve independently. On the client side, this probably boils down to the UI having several progress loader images resolve to values independently. However, this is better than freezing the whole client until .all the reads resolve.

Therefore, dump all the synchronous results to the view immediately, then let the asynchronous results come in as they resolve, individually. This may seem like petty distinction, but if your client has poor Internet connectivity (like I currently have at this coffee shop), freezing the whole client experience for several seconds will likely result in a 'this app sucks' experience.

3 Comments

It's asynchronous, there are plenty of use cases for using Promise.all... it doesn't necessarily have to "freeze" anything – you might need to wait for all the data before you're able to do something meaningful
There are several use cases when you do need to load all of your data, therefore the wait (like a spinner with an appropriate message, no need to "freeze" any UI like you say) can be totally needed by Promise.all... It just really depends on what kind of products you are building here. These kind of comments are to my own opinion very irrelevant and there shouldn't be any "best" words in it. It really depends on every different use cases one can face and what your app is doing for the user.
This depends on what you need

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.