0

There has been a lot of confusion so I have rewritten the question to be more clear. I will describe the entire process of how the requests are dealt with, however this question is only related to the first part of the process.

I have an object called requests that is initialised empty. Other modules within my node.js server add request objects that are in the format as below:

requests["name of request"] = {
    requestType: "type of request",
    parameters: {
        "request parameters here"
    },
    callback: function(data){//Do stuff with the data}
}

Here is how requests like this are processed.

All requests MUST make a call to an external JSON API that is rate limited. I can group individual requests up into groups of size 1-40, regardless of the size of the group it will only count as 1 request against my rate limit. Therefore to maximise my use of the rate limit I should try to make groups that are as large as possible while not delaying the processing of individual requests.

Since the maximum group size is 40 I need to get 40 requests from the requests object and copy then into a second object called inProgressRequests leaving any remaining objects untouched. I then want to delete them to prevent the requests being processed more than once.

Once this has been done I can make the call to the external API, processing all 40 requests at once. I receive the data in bulk and then I must split the data into 40 sections. I then need to return the right data section to the right request's callback function as the data parameter.

If I were to use an array then for each chunk of data (of which there are 40) I would have to loop through the array to find the correct request and then return its chunk of data to it. This would require me to loop through the array a total of 40 times, which is inefficient. Therefore I have decided to use an object, as then I can access each request more efficiently like so: requests[requestName]

Therefore using arrays is not an option unless there is a more efficient way to search for request objects in an unordered array that I have not though of.

My question is what is the most time and resource efficient way for me to perform the first part of processing the request? That is to say what is the most effective way for me to get the first 40 individual request objects from the requests object and then delete them from the requests object?

Improve this question
9
  • "I'm asking if there is a more efficient way to do this" - Yeah. Use an array. Seriously though, how do you know what the name of the property will be? Commented May 5, 2016 at 13:12
  • @nnnnnn That would be more inefficient. As I said once the requests have been processed I must return unique data to the callback of each request. To do this in an array would require me looping through each element of the array and checking if the requestName matches. If there are 40 requests at a time like in my example that would require me to loop through the array 40 times and that would be very time consuming Commented May 5, 2016 at 13:16
  • You wouldn't have to loop through the whole array, because you said requests must be processed in the order they are received. So you know the 40 requests will be the first 40 items, and you could just remove all 40 at once with one call to .splice(), which would automatically create an array of in progress requests because .splice() returns an array of the deleted items. Commented May 5, 2016 at 13:22
  • @Adam0410 Sorry but you are contradicting yourself. On one hand you are saying delete first 40 items. On the other hand you are saying you will loop through entire array 40 times. Why would you loop through entire array if you are simply deleting 40 items? Commented May 5, 2016 at 13:23
  • @nnnnnn No, My program has to process the requests in the order they were received but this part of my server processes requests that require data from an external JSON API. This API can process requests in bulk and returns the result of each request at the same time. I must then split this data up and pass the relevant section of the data to the correct request's callback function. Commented May 5, 2016 at 13:26

1 Answer 1

Reset to default
1

Yes you can delete the request from the object by doing

delete requests[requestName];

This will remove the request from this request object.

If you want to copy-first and the delete based on requestNames from array

requestNames.forEach( function( requestName ){
   secondRequests[ requestName ] = requests[ requestName ];
   delete requests[ requestName ];
});
Improve this answer
Sign up to request clarification or add additional context in comments.

12 Comments

and before that, if you want to transfer it to another object: otherRequests[requestName] = requests[requestName];
Yes I already knew this, the real question is how can I efficiently copy x sub-objects to a new object and then delete all the copied ones. With your solution I'd still have to loop through the object x number of times and copy each sub-object then delete it individually which is not very efficient.
@Adam0410 with what frequency do you delete items and take backup?
@gurvinder372 your updated solution still would not work. I need to copy only a certain number of the elements from the object (e.g. 40) not them all. If I was simply looking to move them all I could clone the entire requests object. The frequency I do this at is every 250ms or so
I need to copy only a certain number of the elements from the object, not them all. Can you add the example of same in your question?
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.