adding documents to an existing index in lucene

Question

I would like to ask of how to add new documents to an existing lucene index. in the source code below, I just change the paramater of IndexWriter into false.

IndexWriter indexWriter = new IndexWriter(
            FSDirectory.open(indexDir),
            new SimpleAnalyzer(),
            false,
            IndexWriter.MaxFieldLength.LIMITED);

because false means that the index will still be open and not close. also to add new document I should use

indexWriter.addDocument(doc)

but my question is how exactly can I add new documents to an existing lucene index. I am a bit loss in finding out where to put a new path directory containing new Documents in lucene class so that lucene can index those new documents and add it into existing indexes. any help would be appreciated though. thanks.

import org.apache.lucene.analysis.SimpleAnalyzer;
import org.apache.lucene.document.Document;
import org.apache.lucene.document.Field;
import org.apache.lucene.index.IndexWriter;
import org.apache.lucene.store.FSDirectory;
import java.io.File;
import java.io.FileReader;
import java.io.IOException;

public class testlucene1 {
public static void main(String[] args) throws Exception {
    File indexDir = new File("C:/Users/Raden/Documents/lucene/LuceneHibernate/adi");
    File dataDir = new File("C:/Users/Raden/Documents/lucene/LuceneHibernate/adi");
    String suffix = "txt";
    testlucene1 indexer = new testlucene1();
    int numIndex = indexer.index(indexDir, dataDir, suffix);
    System.out.println("Total files indexed " + numIndex);
}

private int index(File indexDir, File dataDir, String suffix) throws Exception {
    IndexWriter indexWriter = new IndexWriter(
            FSDirectory.open(indexDir),
            new SimpleAnalyzer(),
            false,
            IndexWriter.MaxFieldLength.LIMITED);
    indexWriter.setUseCompoundFile(false);
    indexDirectory(indexWriter, dataDir, suffix);
    int numIndexed = indexWriter.maxDoc();
    indexWriter.optimize();
    indexWriter.close();
    return numIndexed;
}

   private void indexDirectory(IndexWriter indexWriter, File dataDir, String suffix)    throws IOException {
    File[] files = dataDir.listFiles();
    for (int i = 0; i < files.length; i++) {
        File f = files[i];
        if (f.isDirectory()) {
            indexDirectory(indexWriter, f, suffix);
        } else {
            indexFileWithIndexWriter(indexWriter, f, suffix);
        }
    }
}

private void indexFileWithIndexWriter(IndexWriter indexWriter, File f, String suffix) throws IOException {
    if (f.isHidden() || f.isDirectory() || !f.canRead() || !f.exists()) {
        return;
    }
    if (suffix != null && !f.getName().endsWith(suffix)) {
        return;
    }
    System.out.println("Indexing file " + f.getCanonicalPath());
    Document doc = new Document();
    doc.add(new Field("contents", new FileReader(f)));
    doc.add(new Field("filename", f.getCanonicalPath(), Field.Store.YES,  Field.Index.ANALYZED));
    indexWriter.addDocument(doc);
}
}

XL Zheng · Accepted Answer · 2017-02-10 19:28:15Z

3

Based on Lucene API, when you construction the IndexWriter, the constructor allow you specify the IndexWriterConfig.

IndexWriter(Directory d, IndexWriterConfig conf)

IndexWriterConfig allows you specify the open mode:

IndexWriterConfig conf = new IndexWriterConfig(analyzer);
conf.setOpenMode(IndexWriterConfig.OpenMode.APPEND);

And you have 3 options:

IndexWriterConfig.OpenMode.APPEND
IndexWriterConfig.OpenMode.CREATE
IndexWriterConfig.OpenMode.CREATE_OR_APPEND

answered Feb 10, 2017 at 19:28

XL Zheng

3631 gold badge6 silver badges14 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

recursive9 · Accepted Answer · 2010-10-28 18:17:00Z

1

also to add new document I should use .... but my question is how exactly can I add new documents to an existing lucene index

can you please clarify what you mean? you know how to add documents to an index, as you stated, but then you ask how to... add new documents?

answered Oct 28, 2010 at 18:17

recursive9

3602 silver badges12 bronze badges

2 Comments

jacobian Over a year ago

allright,it was my mistake.I didn't fully understand the source code.but upon reading your comment I just realize it.thanks for the tips then. :-)

Brain Over a year ago

The question was how to add a new document to an existing index, not how to simply add a new document during initial indexing.

Xodarap · Accepted Answer · 2010-10-27 14:23:47Z

1

When you instantiate a new IndexWriter, you will not create a new index (unless you explicitly tell lucene to force a new one). So your code will work, regardless of whether the index already exists.

answered Oct 27, 2010 at 14:23

Xodarap

11.9k12 gold badges64 silver badges99 bronze badges

2 Comments

jacobian Over a year ago

yes I know.but I was trying to add new documents to an existing index.what do you reckon I should do to achieve this? :-)

Xodarap Over a year ago

I don't understand your question then. You create an indexwriter that looks at an existing index in the exact same way that you create an indexwriter which makes a new index. So your code will work regardless of whether indexDir has stuff in it or not.

Collectives™ on Stack Overflow

adding documents to an existing index in lucene

3 Answers 3

Comments

2 Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

2 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related