4

I want to use mongo and ElasticSearch in my projects, and I also like adopt Spring Data Mongo and Spring Data ElasticSearch, but both has their Repository and model specs, how to use them together?

There are some options:

  1. Use the same model class for Mongo and ElasticSearch?

    @Document//from Spring Data Mongo
    @Document// from Spring Data ElasticSearch
    public class Book{
        @Id//Spring Data Commons
        private String id;
    }
    

    But there are some mismatch in Spring Data Mongo and Spring Data ElasticSearch, such as Geo field type.

  2. Define different model for Mongo and ElasticSearch, and copy data state from Mongo model and create index when create a new model.

Any suggestion here?

I would like use option 2 in projects.

  1. Save mongo document as normal.
  2. Fire a event via JMS/AMQP/Reactor to sync the data to Elasticsearch, and choose the index strategy for every field in the ElasticSearch Document.
  3. All search operations are based on ElasticSearch.

Updated on 5/15/2016

I have created a sample to demo this approach.

Sample codes

I used Spring builtin ApplicationEvent to demo this approach.

  1. The event publisher side, Mongo saved the post and publish an event.

    @Component
    public class Publisher implements ApplicationEventPublisherAware {
    
        private static final Logger LOG = LoggerFactory.getLogger(Publisher.class);
    
        @Autowired
        PostRepository repository;
    
        private ApplicationEventPublisher publisher;
    
        public Publisher() {
        }
    
    
        public void savePost(Post post) {
            Post saved = repository.save(post);
            this.publisher.publishEvent(saved);
    
            LOG.debug("saved post data in mongo@" + saved);
        }
    
        @Override
        public void setApplicationEventPublisher(ApplicationEventPublisher publisher) {
            this.publisher = publisher;
        }
    
    }
    
  2. The event receiver side, received the data and sync it into ElasticSearch storage.

        @Component
    public class Receiver {
    
        private static final Logger LOG = LoggerFactory.getLogger(Receiver.class);
    
        @Autowired
        ESPostRepository repository;
    
        @EventListener
        public void onPostSaved(Post savedPost) {
            LOG.debug("=================received post data============== @\r\n"+ savedPost);
    
            ESPost doc=new ESPost();
            doc.setId("1");
            doc.setTitle(savedPost.getTitle());
            doc.setContent(savedPost.getContent());
            repository.save(doc);
        }
    
    }
    

In production environment, the publisher and receiver can be placed in different applications via JMA/AMQP instead of builtin ApplicationEvent.

The mongo is used as main storage and ElasticSearch as index/search server.

5
  • Did you solve it somehow? Having the same problem here ... Commented Oct 2, 2015 at 6:38
  • @DavidMarko I have used the second approach in the above list, use different models in ElasticSearch and MongoDB. Generally we could only needs part of data be index, which data will be stored in ElasticSearch via AMQP or Reactive Stream asynchronously(e.g. Reactor project). and the keyword query/search will shake hands with ElasitcSearch other than Mongo, but for details of the line will be fetched from Mongo. Commented Oct 3, 2015 at 7:38
  • @Hantsy , if possible could you share us the working implemented sample code Commented Apr 21, 2016 at 13:56
  • @rajadilipkolli I have updated my post, and created a working demo to explain my thought. Commented May 15, 2016 at 15:50
  • @Hantsy Thanks for the update. Commented May 16, 2016 at 5:57

1 Answer 1

0

can you just use the fully qualified domain name for the different Document annotations?

thats what we are trying todo here.

  @Document(collection = "SPECTRUM")
  @org.springframework.data.elasticsearch.annotations.Document(indexName = "spectrum", `type` = "spectra", shards = 1, replicas = 0, refreshInterval = "-1")
  case class Spectrum(
                  ...
  )
Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.