1

I am trying to read data from an input file, and for each line perform a task in a while loop. Problem is that when I create the first process - its loop is executing and not returning control to the above for loop. Bottom line there is no parallelism. What am I doing wrong?

Here is the relevant code:

from multiprocessing import Process


def work_line(list1Line,jobId):
    while True:
        print list1Line
        tenant = list1Line[0]
        module = list1Line[1]
        endTime = int(time.time())
        startTime = endTime - startTimeDelta
        generate(jobId, startTime, endTime, tenantServiceAddress, tenant, module)
        print ("tenant {} will sleep for {} seconds").format(tenant,sleepBetweenLoops)
        time.sleep(sleepBetweenLoops)


def openFiles():
    file = open(CLOUD_INPUT_FILE, 'r')
    lines = file.readlines()
    file.close()
    linesLen = len(lines)
    processes = []

    for linesIndex in range(0, linesLen):
        jobId = GenerateRandomID()
        line = lines[linesIndex]
        list1Line = line.split()

        p = Process(target=work_line(list1Line,jobId))
        p.start()
        processes.append(p)
        print processes

    for p in processes:
        p.join()


if __name__ == '__main__':
    CLOUD_INPUT_FILE = r'C:\CF\input_file.txt'
    tenantServiceAddress = 'address.address'
    startTimeDelta = 300
    sleepBetweenLoops = 1800
    print multiprocessing.cpu_count()
    openFiles()

1 Answer 1

2

You are actually calling the function. Change to

p = Process(target=work_line, args=(list1Line,jobId))
Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.