Create test data in MongoDB

Question

I want to generate test data for MongoDB. The size should be 200 Mb. I tried this code:

@Test
    public void testMongoDBTestDataGenerate()
    {
        MongoClient mongoClient = new MongoClient("localhost", 27017);

        DB db = mongoClient.getDB("development");
        DBCollection collection = db.getCollection("ssv");

        for (int i = 0; i < 100; i++)
        {
            BasicDBObject document = new BasicDBObject();
            document.put("database", "test");
            document.put("table", "hosting");

            BasicDBObject documentDetail = new BasicDBObject();
            documentDetail.put("records", 99);
            documentDetail.put("index", "vps_index1");
            documentDetail.put("active", "true");

            document.put("detail", documentDetail);

            collection.insert(document);
        }

        mongoClient.close();

    }

How I can generate data exactly with this size?

Vladimir Vagaytsev · Accepted Answer · 2016-07-13 19:20:48Z

4

I am not getting what are you trying to achieve by setting size 200 Mb.

You can add logical checks.

db.testCollection.stats() - You can check size of the collection before every insert.
Object.bsonsize(..) - Also you can check document size before inserting to make it exactly 200 MB.

And also you can make capped collection where you can notify number of documents or size of collection.

Hope this helps.

edited Jul 13, 2016 at 19:20

Vladimir Vagaytsev

2,8919 gold badges36 silver badges38 bronze badges

answered Jul 7, 2016 at 11:40

Somnath Muluk

58.3k39 gold badges225 silver badges230 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Somnath Muluk Over a year ago

db.testCollection.stats() Object.bsonsize() Given some pointers. Help yourself. @PeterPenzov

airudah · Accepted Answer · 2016-07-07 12:27:32Z

2

What I would probably do is create a capped collection with size 200MB (209715200 bytes):

db.createCollection( "ssv", { capped: true, size: 209715200 } )

Then insert records as you are doing. Then at intervals inside the for loop, check if the collection if full (or almost full).

So in your code, maybe (totally pseudo code):

if(i % 10 == 0) {
   if(db.ssv.stats().size >= 209715100){ //Or an arbitrary value closer to 200MB
     break;
   }
}

edited Jul 7, 2016 at 12:27

answered Jul 7, 2016 at 12:20

airudah

1,17912 silver badges19 bronze badges

Comments

Chaitanya Bapat · Accepted Answer · 2016-07-14 07:17:02Z

2

+50

Why would you copy the same data 100 times to make 200mb worth test data? Instead

1.append a counter to the value so that you can generate data sequentially

OR

Use random function to generate random data

@Test
    public void testMongoDBTestDataGenerate()
    {
        MongoClient mongoClient = new MongoClient("localhost", 27017);

        DB db = mongoClient.getDB("development");
        DBCollection collection = db.getCollection("ssv");
        int counter=0;
        for (int i = 0; i < 873813; i++)
        {
            BasicDBObject document = new BasicDBObject();
            document.put("database", "test");
            document.put("table", "hosting");

            BasicDBObject documentDetail = new BasicDBObject();
            documentDetail.put("counter0", counter++);
            documentDetail.put("counter1", counter++);
            documentDetail.put("counter2", counter++);
            documentDetail.put("counter3", counter++);
            documentDetail.put("counter4", counter++);
            documentDetail.put("counter5", counter++);
            documentDetail.put("counter6", counter++);
            documentDetail.put("counter7", counter++);
            documentDetail.put("counter8", counter++);
            documentDetail.put("counter9", counter++);

            document.put("detail", documentDetail);

            collection.insert(document);
        }

        mongoClient.close();

    }
}

10 eight two-bytes char strings and 10 eight-bytes numbers => 240B

240B * 873813 = 200MB

answered Jul 14, 2016 at 7:17

Chaitanya Bapat

4,1199 gold badges42 silver badges67 bronze badges

1 Comment

Chaitanya Bapat Over a year ago

Yes sure. Assuming that you are using Java and hence can utilize Random class object. Values we are setting to the counter variable can rather be replaced by the object of Random class. E.g. Random counter = new Random(); documentDetail.put("counter0",counter.nextInt(100)); Here - nextInt(100) generates random number from 0-100. Can be changed as per the need/requirement

chf · Accepted Answer · 2016-07-07 12:34:49Z

1

A quick dirty solution based on Robert Udah and Somnath Muluk suggestions:

answered Jul 7, 2016 at 12:34

chf

7634 silver badges14 bronze badges

2 Comments

Peter Penzov Over a year ago

Ok, what is the best solution?

chf Over a year ago

It's the same suggestion :)

SdSaati · Accepted Answer · 2016-07-14 05:56:32Z

1

create a field with type nvarchar then generate a .txt file in your for example desktop with 200MB data. then put this string into that field. that's it.

answered Jul 14, 2016 at 5:56

SdSaati

9222 gold badges10 silver badges20 bronze badges

Collectives™ on Stack Overflow

Create test data in MongoDB

5 Answers 5

1 Comment

Comments

1 Comment

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

1 Comment

Comments

1 Comment

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related