Having at least 500 GB of data for inverted index (integer keys and large List<String> records) with 1 million keys at maximum, what would be the best (memory efficient and fast look-up) strategy for distributing the index into X parts?
  1. X engines and 1 table per engine (X threads and one thread per engine)
  2. 1 engine and X tables and X threads communicating with the engine
  3. 1 engine and X tables and 1 thread
  4. 1 engine and 1 table (no distribution at all)
Thanks