0

I have a script with neural networks here is example of init

import torch
from torchvision.models.detection import fasterrcnn_resnet50_fpn
from torchvision.models.detection import FasterRCNN_ResNet50_FPN_Weights

class Detector():
....
def _initialize_detector(self):
        """Initialize object detection model."""
        model = fasterrcnn_resnet50_fpn(weights=FasterRCNN_ResNet50_FPN_Weights.DEFAULT)
        model.to(device).eval()
        return model

it is working fine, but when I create celery task

@celery_app.task
def video_detect_task(
    video_file: str,
    bvh_filename: str,
):
    """Celery task for video_detect_task module.

    :param video_path: path to input video file
    :param bvh_filename: path to save bvh file
    """
    task_id = current_task.request.id
    try:
        model = Detector.load(Config())
        frames_num = get_video_duration(video_file)
        logger.info('frames: ',extra = {'number ':frames_num})
        skeleton_name, _ = os.path.splitext(os.path.basename(bvh_filename))
        skeleton_full_path = os.path.dirname(bvh_filename) + "/" + skeleton_name
        save_character_path = ".".join([skeleton_full_path, 'bvh'])

        status = model.process(
            video_file=video_file,
            bvh_file_name=save_character_path
        )

then it code not working it cant even load the model by logs I see this:

22.5%-12-22 00:10:27,496: WARNING/ForkPoolWorker-2] 
22.5%-12-22 00:10:27,496: WARNING/ForkPoolWorker-2] 
22.6%-12-22 00:10:27,497: WARNING/ForkPoolWorker-2] 
22.7%-12-22 00:10:27,497: WARNING/ForkPoolWorker-2] 
22.8%-12-22 00:10:27,497: WARNING/ForkPoolWorker-2]

Thank you!

BUT when I do

torch.set_num_threads(1)

celery task starts works also okay only long. All computetaion oc CPU.

I think it is some problem with Celery and Troech multiprocessing

I run celery with this command

command: ["celery", "-A", "video_detect_task.tasks.celery_app.celery_app", "worker", "-Q", "video_detect_task_queue", "--loglevel=info"]

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.