multithreading - How to implement several threads in Java for downloading a single table data?

Question

How can I implement several threads with multiple/same connection(s), so that a single large table data can be downloaded in quick time.

Actually in my application, I am downloading a table having 12 lacs (1 lac = 100,000) records which takes atleast 4 hrs to download in normal connection speed and more hrs with slow connection.

So there is a need to implement several threads in Java for downloading a single table data with multiple/same connection(s) object. But no idea how to do this.

How to position a record pointer in several threads then how to add all thread records into a single large file??

Thanks in Advance

Answer 1

First of all, is it not advisable to fetch and download such a huge data onto the client. If you need the data for display purposes then you dont need more records that fit into your screen. You can paginate the data and fetch one page at a time. If you are fetching it and processsing in your memory then you sure would run out of memory on your client.

If at all you need to do this irrespective of the suggestion, then you can spawn multiple threads with separate connections to the database where each thread will pull a fraction of data (1 to many pages). If you have say 100K records and 100 threads available then each thread can pull 1K of records. It is again not advisable to have 100 threads with 100 open connections to the DB. This is just an example. Limit the no number of threads to some optimal value and also limit the number of records each thread is pulling. You can limit the number of records pulled from the DB on the basis of rownum.