1

This is my first post so please let me know if there is anything I should add to the post if needed.

I have a program that grabs urls from my MySQL database and sends them to a class that will download the images to my desktop.

I have 80,000 images I need to get, and I well I want to utilize multithreading to make it quicker.

I have gotten somewhere but I am now stuck and have spent hours researching as I am brand new to multithreading.

The problem is when the program runs, it goes through a loop processing the first 5 entries over and over again. This happens to be the ExecutorFixedPoolSize.

Can someone help me please.

My Code is as follows.

    public static void main(String[] args) {

        try{

            countAllAltPicsNotSaved();
            System.out.println("Not saved: " + totalAltPicsNotSaved);

            Class.forName("com.mysql.jdbc.Driver");
            Connection conn =DriverManager.getConnection(dbUrl + dbName, userName, password);
            Statement s = conn.createStatement ();
            s.executeQuery ("SELECT DISTINCT id from productsaltimages WHERE saved != 1");
            ResultSet rs = s.getResultSet();
            int saveCounter = 0, altImageCount = 0;

            List<Thread> threads = new ArrayList<Thread>();



            while (rs.next()) { //Get ID
                altImageCount = 0; //Product Alt Image Counter
                String id = rs.getString("id");  //Product Table ID

                Statement news = conn.createStatement (); //New Conn for get Alt Images From ID
                news.executeQuery ("SELECT url from productsaltimages WHERE id ='"+id+"' AND saved != 5"); //Gets query from mySQL

                ResultSet newrs = news.getResultSet(); //Resultset for AltImges for ID

                ExecutorService executor = Executors.newFixedThreadPool(5);
                Runnable task = null;

                while (newrs.next()){ //Get Images
                    altImageCount++; //Increment ID Alt Image Counter
                    String url = newrs.getString("url");
                    task = new DownloadImage(url, id, altImageCount);
                    executor.execute(task); 

                }
            }

        } catch (Exception e){
        }

    }

    public static void countAllAltPicsNotSaved() throws Exception {
        try{
            totalAltPicsNotSaved=0;
            Class.forName("com.mysql.jdbc.Driver");
            Connection conn =DriverManager.getConnection(dbUrl + dbName, userName, password);
            Statement s = conn.createStatement ();
            boolean sqlExecuteok = false;
            s.executeQuery ("SELECT * from productsaltimages WHERE saved = '0'");
            ResultSet rs = s.getResultSet ();
            while (rs.next()) {
                totalAltPicsNotSaved++;
            }
            rs.close();
            s.close();
            conn.close();
        } catch (Exception e){
        }
    }


For my DownloadImage class code is: 

    public class DownloadImage implements Runnable
{
   String url, id, threadName;
   private int altImageCount = 0;
   private static String userName = "owner"; 
    private static String password = "hello123";
    private static String dbUrl = "jdbc:mysql://localhost/";
    private static String dbName = "shop"; 
    private static String dbClass = "com.mysql.jdbc.Driver";

   public DownloadImage(String url, String id, int altImageCount)
   {
      this.url = url;
      this.id = id;
      this.altImageCount = altImageCount;
   }
   public void run()
   {
      while(true)
      {

        File file=new File("D:\\FBShop_AltImages\\" + id + "\\");
        boolean exists = file.exists();
        if (!exists) {  
            // Create multiple directories
            boolean success = (new File("D:\\FBShop_AltImages\\" + id + "\\")).mkdirs();
        }

        String newFilename = "D:\\FBShop_AltImages\\" + id + "\\" + id + "_" + altImageCount + ".jpg";

        try {

            try {
                BufferedImage image = null;
                URL imgurl = new URL(url);
                URLConnection con = imgurl.openConnection();
                con.setConnectTimeout(50 * 10000);
                con.setReadTimeout(50 * 10000);
                InputStream in = con.getInputStream();
                image = ImageIO.read(in);
                ImageIO.write(image, "jpg", new File(newFilename));

            } catch (Exception e) {
                System.out.println("Error getting image: " + url);
            }  

            try {
                //UpdateTable
                Class.forName("com.mysql.jdbc.Driver");
                String updatesql = "UPDATE productsaltimages SET saved = '1' WHERE id = '"+id+"'";

                Connection conn =DriverManager.getConnection(dbUrl + dbName, userName, password);
                Statement ups = conn.createStatement ();
                int val = ups.executeUpdate(updatesql);
                System.out.println("Task Complete: " + url);
                try {
                    Thread.sleep(5000);
                } catch (Exception e) {
                }
            } catch (Exception e) {
            }
        } catch (Exception e) {
        }
         //System.out.println("Thread Finished: " + threadName);
      }
   }
}
1
  • @Eugene Lol, subtle hint :p Commented Aug 17, 2014 at 15:07

1 Answer 1

2
 while(true){
    //lots of code inside you Runnable

    Thread.sleep(5000);
    //After the Thread will sleep it will restart it's work again, and again..
 }

When will this end? Remove the while true and you should be good.

Sign up to request clarification or add additional context in comments.

2 Comments

Sorry, i don't what happened and why that was there..Because im learning i copied paste some code from the net.. i guess i learnt my lesson there not to do that!! I removed that and all went well.. Took a few hours, but at least i researched about Multithreading and have learnt alot haha.. Thank you this is what fixed it for me.
@JeffreyHolmes if you want to learn about Threading you should read a book - Concurrency In Practice - it is the BEST out there. Cheers!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.