We need to improve the reindex or the documentation on how to reindex in a production environment where entity changes/additions/deletions may occur during reindexation. We now get exceptions such as:

 [java] 16:58:57,122 ERROR LogErrorHandler:82 - Exception occurred org.apache.lucene.index.MergePolicy$MergeException: org.apache.lucene.index.CorruptIndexException: doc counts differ for segment _2: fieldsReader shows 1 but segmentInfo shows 0
 [java] 
 [java] org.apache.lucene.index.MergePolicy$MergeException: org.apache.lucene.index.CorruptIndexException: doc counts differ for segment _2: fieldsReader shows 1 but segmentInfo shows 0
 [java]     at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:517)
 [java]     at org.hibernate.search.backend.impl.lucene.overrides.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:49)
 [java]     at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:482)
 [java] Caused by: org.apache.lucene.index.CorruptIndexException: doc counts differ for segment _2: fieldsReader shows 1 but segmentInfo shows 0
 [java]     at org.apache.lucene.index.SegmentReader$CoreReaders.openDocStores(SegmentReader.java:295)
 [java]     at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:580)
 [java]     at org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:684)
 [java]     at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3918)
 [java]     at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3614)
 [java]     at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:388)
 [java]     at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:456)

This is probably because of concurrent modifications to the index, because reindex runs in a seperate process/vm.

Submitted by Elmer van Chastelet on 14 July 2012 at 17:01