Copy files recursively from source to target based on wildcard using Python

Question

I have a source directory that has several xml files in nested location. I am trying to write a python script to copy files recursively based on a pattern (say *1.2.3.xml) to a target location.

source
├── master.1.2.3.xml
    └── directory
           └── fileA.1.2.3.xml
           ├── fileA.1.2.5.xml
           ├── fileB.1.2.3.xml

Expected outcome:

target
├── master.1.2.3.xml
    └── directory
           └── fileA.1.2.3.xml
           ├── fileB.1.2.3.xml

The following script doesn't do the filtering.

from shutil import copytree
def ignored_files(adir,filenames):
    return [filename for filename in filenames if not filename.endswith('1.2.3.xml')]
copytree(source, target, ignore=ignored_files)

What am I missing here?

Can you give us the exact source and target you're using? And what is the outcome? Is there an error message? — Saravana Kumar
– Saravana Kumar, Commented Feb 22, 2018 at 4:47

pd321 · Accepted Answer · 2018-02-22 05:13:19Z

3

What is happening here is that the copytree function will work recursively. First, it will descend into source and give ignored_files() two items for filenames arg - [master.1.2.3.xml, directory]

The ignored_files will return [directory] as it does not match with the pattern and thus copytree will ignore the entire directory itself.

You will have to add an additional check for directories in your condition in ignored_files() something like os.path.isdir().

answered Feb 22, 2018 at 5:13

pd321

6496 silver badges14 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Aditi · Accepted Answer · 2018-02-22 04:50:15Z

1

Can you please try this?

import glob
import shutil
dest_dir = "path/to/dir"
for file in glob.glob(r'/path/to/dir/*1.2.3.xml'):
    print(file)
    shutil.copy(file, dest_dir)

answered Feb 22, 2018 at 4:50

Aditi

82612 silver badges28 bronze badges

Comments

dux2 · Accepted Answer · 2019-04-01 09:55:27Z

0

Try this instead:

def copy_tree_with_wildcards(src: str, dest: str, pattern: str):
src = os.path.abspath(src)
for filename in glob.iglob(src + '/**/' + pattern, recursive=True):
    src_file = os.path.join(src, filename)
    dest_file = dest + filename[len(src):]
    dest_dir = os.path.dirname(dest_file)
    os.makedirs(dest_dir, exist_ok=True)
    if os.path.isfile(src_file):
        shutil.copy(src_file, dest_file)

edited Apr 1, 2019 at 9:55

answered Apr 1, 2019 at 9:48

dux2

1,9101 gold badge24 silver badges29 bronze badges

Collectives™ on Stack Overflow

Copy files recursively from source to target based on wildcard using Python

3 Answers 3

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related