How can I filter an array on an asynchronous predicate?

Question

I have an array of users and I want to grab a subset of them based on an inexpensive lookup in a redis set.

const users = [
  { _id: '1', name: 'george', age: 36 },
  { _id: '2', name: 'henry', age: 33 },
  { _id: '3', name: 'agatha', age: 28 },
  { _id: '4', name: 'janet', age: 29 },
  { _id: '5', name: 'gary', age: 21 },
  // ... 995 more users
]

const db = {/* my redis connection */}

const isInside = (db, user) => {
  return db.contains('my:set:key', user._id)
}

I have tried Array.prototype.filter but it doesn't seem to work

users.filter(user => isInside(db, user))
// => always gives me back all the users even when I see they are not in the set

I know something is wrong here. How do I filter out users using isInside?

DedaDev · Accepted Answer · 2020-05-25 02:49:28Z

4

The problem is that filter is always synchronous and your DB calls are asynchronous, filter function always returns true because its db.contains is a running promise, so it converts to true.

One of the solutions could be, to create an array of promises, wait for all of them, and then filter out.

const users = [
  { _id: '1', name: 'george', age: 36 },
  { _id: '2', name: 'henry', age: 33 },
  { _id: '3', name: 'agatha', age: 28 },
  { _id: '4', name: 'janet', age: 29 },
  { _id: '5', name: 'gary', age: 21 },
  // ... 995 more users
]

const dbCheck = users.map(user => isInside(db, user))

Promise.all(dbCheck).then((values) => {
  // here you have array of bools [true, false ...]

  const filteredUsers = users.filter((_, index) => values[index]))
});

edited May 25, 2020 at 2:49

answered May 25, 2020 at 2:43

DedaDev

5,2792 gold badges26 silver badges32 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Roamer-1888 · Accepted Answer · 2020-05-25 04:26:44Z

You will probably be OK with Promise.all(users.map(user => isInside(db, user))) but there's a danger of hitting the database too hard with multiple simultaneous requests, particularly with some 3rd-party cloud services.

If so, then you can orchestrate an asynchronous filter in which db queries are performed sequentially, based on Array.prototype.reduce.

It's a bit of a palaver, but not too bad:

const users = [
	  { _id: '1', name: 'george', age: 36 },
	  { _id: '2', name: 'henry', age: 33 },
	  { _id: '3', name: 'agatha', age: 28 },
	  { _id: '4', name: 'janet', age: 29 },
	  { _id: '5', name: 'gary', age: 21 },
	  // ... 995 more users
]

users.reduce(function(promise, user) {
	return promise
	.then(arr => {
		return isInside(null, user) // <<< the asynchronous call
		.then(bool => { // isInside delivers Boolean
			if(bool) arr.push(user); // act, depending on asynchronously derived Boolean
			return arr; // deliver arr to next iteration of the reduction
		});
	});
}, Promise.resolve([])) // starter promise, resolved to empty array
.then(filtered => {
	console.log(filtered); // Yay! a filtered array
});

// dummy isInside() function
function isInside(db, user) {
	return Promise.resolve(Math.random() < 0.5); // 50% probability
}

Of course, this will be slower than a .map() solution but if .map() doesn't work ....

Can quite eacily be rephrased as a method on Array.prototype.

Collectives™ on Stack Overflow

How can I filter an array on an asynchronous predicate?

2 Answers 2

Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related