0

I have a ssh dask.distributed cluster with a main computer containing all modules for my script and another one with only a few, including dask itself of course.

Is it possible to change the syspath of the other computer so that it also looks for modules in the main one? Of course, I could simply upload them via sftp but since I keep making a lot of modules that would be very annoying to do repeatedly.

Improve this question

1 Answer 1

Reset to default
1

Such things are possible with networking solutions such as NFS or SSH remote mounts, but that's a pretty big field and beyond the scope of Dask itself. If you are lucky, other answers will appear here, others have solved similar problems, but more likely copying is the simpler solution.

Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

I guess it's worth a try. Do you know any tutorial for me to get started?
sshfs has been around a long time, but the specifics will depend on your networks (i.e., you'll need to distribute keys).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.