Goal

I would like to be able to separate all my data file readers from the main thread and balance the load over multiple frames. The idea being that while the data is being read/parsed/filtered I can continue playing the game at a decent FPS and show a loading progress bar while the results of the file importing pop in one at a time when they are ready.

Information

The data files contain things like transform information for objects, points to be used in runtime mesh generation, and terrain data. They all have to be parsed, filtered, and/or modified to be ready to be used in the scene.

I already have import code for the project I just need to transplant it from blocking my main thread and being done in one frame to a multi-threaded and multi-frame approach.

Thoughts

I have little experience in multi-threading so this is also a learning project for me. I would appreciate all your comments, input, code samples, and pointers.

From my perspective I need to create a queue of tasks. Each task being a file to load, read, parse, and filter. Then if a CPU is open a task is taken from the queue and is given a thread to be executed on that CPU. The task completes and gives the resulting data back to Unity3D for it to handle.

Questions

1. Does a heavy load on other CPUs freeze the game like it would if it were on the main thread, and thus require some sort of system where you yield for one frame if execution time is taking too long?
2. How do I receive progress updates from the tasks to be used in a interface progress bar via some sort of co-routine?
3. If a user only has one CPU how would this system handle the situation? Is that even a realistic scenario?
4. What sort of C# classes already exist for multi-threading and task queuing that can save me time?

Goal

I would like to be able to separate all my data file readers from the main thread and balance the load over multiple frames. The idea being that while the data is being read/parsed/filtered I can continue playing the game at a decent FPS and show a loading progress bar while the results of the file importing pop in one at a time when they are ready.

Information

The data files contain things like transform information for objects, points to be used in runtime mesh generation, and terrain data. They all have to be parsed, filtered, and/or modified to be ready to be used in the scene.

I already have import code for the project I just need to transplant it from blocking my main thread and being done in one frame to a multi-threaded and multi-frame approach.

Thoughts

I have little experience in multi-threading so this is also a learning project for me. I would appreciate all your comments, input, code samples, and pointers.

From my perspective I need to create a queue of tasks. Each task being a file to load, read, parse, and filter. Then if a CPU is open a task is taken from the queue and is given a thread to be executed on that CPU. The task completes and gives the resulting data back to Unity3D for it to handle.

Questions

1. Does a heavy load on other CPUs freeze the game like it would if it were on the main thread, and thus require some sort of system where you yield for one frame if execution time is taking too long?
2. How do I receive progress updates from the tasks to be used in a interface progress bar via some sort of co-routine?
3. If a user only has one CPU how would this system handle the situation? Is that even a realistic scenario?
4. What sort of C# classes (usable in Mono's subset of the .NET runtime) already exist for multi-threading and task queuing that can save me time?

Goal

I would like to be able to separate all my data file readers from the main thread and balance the load over multiple frames. The idea being that while the data is being read/parsed/filtered I can continue playing the game at a decent FPS and show a loading progress bar while the results of the file importing pop in one at a time when they are ready.

Information

The data files contain things like transform information for objects, points to be used in runtime mesh generation, and terrain data. They all have to be parsed, filtered, and/or modified to be ready to be used in the scene.

I already have import code for the project I just need to transplant it from blocking my main thread and being done in one frame to a multi-threaded and multi-frame approach.

Thoughts

I have little experience in multi-threading so this is also a learning project for me. I would appreciate all your comments, input, code samples, and pointers.

From my perspective I need to create a queue of tasks. Each task being a file to load, read, parse, and filter. Then if a CPU is open a task is taken from the queue and is given a thread to be executed on that CPU. The task completes and gives the resulting data back to Unity3D for it to handle.

Questions

1. Does a heavy load on other CPUs freeze the game like it would if it were on the main thread, and thus require some sort of system where you yield for one frame if execution time is taking too long?
2. How do I receive progress updates from the tasks to be used in a interface progress bar via some sort of co-routine?
3. If a user only has one CPU how would this system handle the situation? Is that even a realistic scenario?

Goal

I would like to be able to separate all my data file readers from the main thread and balance the load over multiple frames. The idea being that while the data is being read/parsed/filtered I can continue playing the game at a decent FPS and show a loading progress bar while the results of the file importing pop in one at a time when they are ready.

Information

The data files contain things like transform information for objects, points to be used in runtime mesh generation, and terrain data. They all have to be parsed, filtered, and/or modified to be ready to be used in the scene.

I already have import code for the project I just need to transplant it from blocking my main thread and being done in one frame to a multi-threaded and multi-frame approach.

Thoughts

I have little experience in multi-threading so this is also a learning project for me. I would appreciate all your comments, input, code samples, and pointers.

From my perspective I need to create a queue of tasks. Each task being a file to load, read, parse, and filter. Then if a CPU is open a task is taken from the queue and is given a thread to be executed on that CPU. The task completes and gives the resulting data back to Unity3D for it to handle.

Questions

1. Does a heavy load on other CPUs freeze the game like it would if it were on the main thread, and thus require some sort of system where you yield for one frame if execution time is taking too long?
2. How do I receive progress updates from the tasks to be used in a interface progress bar via some sort of co-routine?
3. If a user only has one CPU how would this system handle the situation? Is that even a realistic scenario?
4. What sort of C# classes already exist for multi-threading and task queuing that can save me time?