Querying multiple collections of firestore using array-contains-any

Question

I am trying query a number of documents in different collections with array-contains-any, by using Promise.all() to get all the documents at once.

I've checked whether the path is correct, whether the documents exist, whether the key exists in the document and everything is ok.

The querySnapshots and snapshots have no data retrieved. At some point of the log says: "_size":0,"_materializedDocs":null.

  let promises = []
  depIds.forEach(id => {
    const prodIds = groupedProducts[id].reduce((acc, val) => [...acc, val.prodId], []);
    console.log("All prodIds: ", prodIds, "; also id is: ", id);
    promise = admin.firestore()
      .collection('Products')
      .doc('Departments')
      .collection(id)
      .where('key', 'array-contains-any', prodIds)
      .get();
    promises.push(promise)

  })

  const querySnapshots = await Promise.all(promises);
  const snapshots = querySnapshots.map(doc => {
    console.log("docs: ", JSON.stringify(doc))
    return doc;
  });

So my questions are:

Is it possible to query as above?
How to get the actual data after the Promise.all() command?

I appreciate any help!

samthecodingman · Accepted Answer · 2021-04-29 08:13:55Z

If the key field in your document is a string, you should be using the in operator.

The array-contains-any operator checks if any of the values you have given are in the array of the named field. As key is a string, this operator will always return no results.

To get all documents where key matches a given ID, while also ensuring that you can fetch more than 10 documents at a time, you can use:

/** splits array `arr` into chunks of max size `n` */
function chunkArr(arr, n) {
  if (n <= 0) throw new Error("n must be greater than 0");
  return Array
    .from({length: Math.ceil(arr.length/n)})
    .map((_, i) => arr.slice(n*i, n*(i+1)))
}

/** Fetch all given product IDs (if they exist) for the given department */
fetchDepartmentProducts(depId, prodIdList) {
  const prodIdListInBatches = chunkArr(prodIdList, 10);
  const departmentCollectionRef = admin.firestore()
    .collection('Products')
    .doc('Departments')
    .collection(depId);
  
  const promises = prodIdListInBatches.map((prodIdListBatch) => {
    return departmentCollectionRef
      .where('key', 'in', prodIdListBatch)
      .get();
  });
  
  return Promise.all(promises) // waits for all get requests
    .then((allQuerySnapshots) => {
      // flatten the found documents of the query snapshots into one array
      const allDocSnapshots = [];
      allQuerySnapshots.forEach((querySnapshot) =>
        allFoundDocSnapshots.push(...querySnapshot.docs)
      );
      return allDocSnapshots;
    });
}

Working this into your code, gives:

const promises = depIds.map((id) => {
  const prodIds = groupedProducts[id].map((product) => product.prodId);
  return fetchDepartmentProducts(id, prodIds);
}

const productsByDepartment = await Promise.all(promises);

productsByDepartment.forEach((docsInDeparment, i) => {
  console.log(`Found ${docsInDeparment.length} products in department #${depId[i]}.`);
});

Thank you very much! It worked! Split the array into chunks of 10 made all the difference.

Renaud Tarnec · Accepted Answer · 2021-04-29 07:58:11Z

1

Is it possible to query as above

Yes, it is totally possible since the get() method returns a Promise that will be resolved with the results of the Query (i.e. a QuerySnapshot)

How to get the actual data after the Promise.all() command?

await Promise.all(promises); returns a single Promise that "resolves to an array of the results of the input promises" (MDN source), i.e. in your case to an Array of QuerySnapshots.

So to read the results you need to loop on the Array and for each element loop on the QuerySnapshot. For example as follows:

  const promises = [];
  depIds.forEach((id) => {
    const prodIds = groupedProducts[id].reduce(
      (acc, val) => [...acc, val.prodId],
      []
    );
    console.log('All prodIds: ', prodIds, '; also id is: ', id);
    promise = admin
      .firestore()
      .collection('Products')
      .doc('Departments')
      .collection(id)
      .where('key', 'array-contains-any', prodIds)
      .get();
    promises.push(promise);
  });

  const querySnapshots = await Promise.all(promises);

  const docs = [];
  
  querySnapshots.forEach(querySnapshot => {
      querySnapshot.forEach(doc => {
          docs.push(doc.data());
      });
  });

I've used forEach to loop over the Array of QuerySnapshots but you may adapt it with any other way of looping over/mapping an Array

answered Apr 29, 2021 at 7:58

Renaud Tarnec

83.4k10 gold badges99 silver badges130 bronze badges

1 Comment

Aliton Oliveira Over a year ago

As @samthecodingman pointed out, my mistake was to search an array, when in fact I should be searching in a field in the document.

DIGI Byte · Accepted Answer · 2021-04-29 04:11:29Z

0

To query, the index must first exist. Check your Firebase console or your app console to make sure there are no index not created warnings.

Documentation: https://firebase.google.com/docs/firestore/query-data/indexing

Firestore also has an inherent spam limit before it blocks requests, make sure that you batch your methods to be no more than 50 concurrent pending.

answered Apr 29, 2021 at 4:11

DIGI Byte

4,1731 gold badge16 silver badges27 bronze badges

1 Comment

Aliton Oliveira Over a year ago

Everything was checked beforehand. Thank you for your help!

Collectives™ on Stack Overflow

Querying multiple collections of firestore using array-contains-any

3 Answers 3

1 Comment

1 Comment

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

1 Comment

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related