How to import data to mongodb container and creating an image

Question

I spent a lot of time on a task which seems a simple one. I want to run a monogodb container. then copy csv file in to the container or execute a command from docker file to import the csv file into the mongodb database. I tried using RUN, script.sh but nothing is working. Please help. I want to automate the task and when I copy the data, I will make a new image for my automated task.

Here is the Dockerfile I have.

FROM mongo
RUN mkdir -p /app/data
WORKDIR /app/data
COPY gene.csv /app/data
COPY script.sh /app/data
EXPOSE 27017
# CMD ["/app/data/script.sh"]

The script file is

#! /bin/sh 
mongoimport -d Gene -c genes --type csv --file gene.csv --headerline

Why @DanGreen-Leipciger? I can't see any possible reason to choose node for such a ridiculously lightweight task, but I have no expertise in it so genuine question. — johnharris85
– johnharris85, Commented Apr 23, 2017 at 19:20
It's just quick and easy, if you already use it. Runs by default in cli, $node myscript.js. It also takes no configuration and runs on any machine. — Dan Green-Leipciger
– Dan Green-Leipciger, Commented Apr 23, 2017 at 19:22
Also, MongoDB has an amazing node interface and all of the said commands are available. Plus, the OP would have access to all of the node tools available on NPM should the needs grow more complex. — Dan Green-Leipciger
– Dan Green-Leipciger, Commented Apr 23, 2017 at 19:24
This is so common thing to do, I wonder why its not already on the net. What actually I am doing is: I am creating a web app and I have written a docker-compose.yml file. If I am able to enter the data and make an image, all the users will do is docker-compose up and browse to localhost:3000/genes to see the data. — drmaa
– drmaa, Commented Apr 23, 2017 at 19:30

ishak O. · Accepted Answer · 2019-03-17 21:51:06Z

2

Too late, maybe help to others.

    FROM mongo
    RUN mkdir -p /app/data
    WORKDIR /app/data
    COPY gene.csv /app/data
    COPY script.sh /app/data
    CMD ["mongod", "&&", "mongoimport", "-d", "Gene", "-c", "Genes", "--file", "gene.csv", "--headerline"]

answered Mar 17, 2019 at 21:51

ishak O.

1901 gold badge3 silver badges15 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

drmaa Over a year ago

I will check it out. Thanks for the reply

johnharris85 · Accepted Answer · 2017-04-23 20:09:41Z

1

Edit: This answer solves a problem, but not the problem, as apparently mongo has to be running for any of this to work. In that case a custom entrypoint is probably the way to go.

Using RUN is the right way, as then when you use CMD you can actually run the Mongo process. However your problem is with your COPY instructions.

COPY gene.csv /app/data
COPY script.sh /app/data

These lines need a / on the end, as right now they say 'copy this file from my host, to be this file in my container'. Whereas you're trying to say 'copy this file from my host, into this folder in my container. They should read as below (but can be simplified even further, see final Dockerfile at the end):

COPY gene.csv /app/data/
COPY script.sh /app/data/

Your script will then run if you put it in a RUN instruction and that layer will be committed into the image, then you can have a CMD at the end to run mongo (or just leave it blank and inherit the default ENTRYPOINT / CMD from the parent (mongo) image. So the full Dockerfile becomes:

FROM mongo
WORKDIR /app/data        # This will be created if it doesn't exist
COPY gene.csv .
COPY script.sh .
RUN chmod +x script.sh && sync && ./script.sh

edited Apr 23, 2017 at 20:09

answered Apr 23, 2017 at 19:18

johnharris85

19.3k5 gold badges52 silver badges54 bronze badges

9 Comments

drmaa Over a year ago

Thanks for help. For the above docker file I got the following error Step 5/5 : RUN ./script.sh ---> Running in 1d385c7df879 /bin/sh: 1: ./script.sh: Permission denied The command '/bin/sh -c ./script.sh' returned a non-zero code: 126

drmaa Over a year ago

I got this error now Step 5/5 : RUN chmod +x script.sh && sync && ./script.sh ---> Running in 6de14ed40581 2017-04-23T19:45:30.083+0000 [#.......................] Gene.genes 4.00KB/57.6KB (6.9%) 2017-04-23T19:45:30.598+0000 [#.......................] Gene.genes 4.00KB/57.6KB (6.9%) 2017-04-23T19:45:30.598+0000 Failed: error connecting to db server: no reachable servers 2017-04-23T19:45:30.598+0000 imported 0 documents The command '/bin/sh -c chmod +x script.sh && sync && ./script.sh' returned a non-zero code: 1

drmaa Over a year ago

I read somewhere an hour ago that you have to restart db or something like that. Dont know why they made it so complicated.

johnharris85 Over a year ago

I'm not that familiar with mongo. Does mongo need to be running for this to work? If so then we'll need a different solution.

drmaa Over a year ago

If we give FROM mongo CMD ["usr/bin/mongod", "--smallfiles"] before executing the script. Same error,

|

Community · Accepted Answer · 2017-05-23 12:25:58Z

0

I was able to do that but through another container. Whose job is to insert the data in to the mongodb database container and exit. For more details check the accepted answer here. I changed the JSON file with my csv file and the command which import it, although I found that mongodb works better with json.

edited May 23, 2017 at 12:25

CommunityBot

11 silver badge

answered Apr 28, 2017 at 15:51

drmaa

3,74411 gold badges53 silver badges78 bronze badges

Collectives™ on Stack Overflow

How to import data to mongodb container and creating an image

3 Answers 3

1 Comment

9 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

9 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related