Java too many open files exception

Question

I have a problem on my code; basically I have an array containing some key:

String[] ComputerScience = { "A", "B", "C", "D" };

And so on, containing 40 entries.

My code reads 900 pdf from 40 folder corresponding to each element of ComputerScience, manipulates the extracted text and stores the output in a file named A.txt , B.txt, ecc ...

Each folder "A", "B", ecc contains 900 pdf.

After a lot of documents, an exception "Too many open files" is thrown. I'm supposing that I am correctly closing files handler.

 static boolean writeOccurencesFile(String WORDLIST,String categoria, TreeMap<String,Integer> map) {
    File dizionario = new File(WORDLIST);
    FileReader fileReader = null;
    FileWriter fileWriter = null;

    try {

        File cat_out = new File("files/" + categoria + ".txt");
        fileWriter = new FileWriter(cat_out, true);
    } catch (IOException e) {
        e.printStackTrace();
    }

    try {
        fileReader = new FileReader(dizionario);
    } catch (FileNotFoundException e) { }

    try {
        BufferedReader bufferedReader = new BufferedReader(fileReader);

        if (dizionario.exists()) {

            StringBuffer stringBuffer = new StringBuffer();
            String parola;
            StringBuffer line = new StringBuffer();
            int contatore_index_parola = 1;

            while ((parola = bufferedReader.readLine()) != null) {

                if (map.containsKey(parola) && !parola.isEmpty()) {
                    line.append(contatore_index_parola + ":" + map.get(parola).intValue() + " ");
                    map.remove(parola);
                }
                contatore_index_parola++;
            }

            if (! line.toString().isEmpty()) {
                fileWriter.append(getCategoryID(categoria) + " " + line + "\n"); // print riga completa documento N x1:y x2:a ...
            }



        } else { System.err.println("Dictionary file not found."); }

        bufferedReader.close();
        fileReader.close();
        fileWriter.close();


    } catch (IOException e) { return false;}
    catch (NullPointerException ex ) { return false;}
    finally {
        try {
            fileReader.close();
            fileWriter.close();
        } catch (IOException e) {
            e.printStackTrace();
        }

    }

    return true;
}

But the error still comes. ( it is thrown at:)

 try {
    File cat_out = new File("files/" + categoria + ".txt");
    fileWriter = new FileWriter(cat_out, true);
} catch (IOException e) {
    e.printStackTrace();
}

Thank you.

EDIT: SOLVED I found the solution, there was, in the main function in which writeOccurencesFile is called, another function that create a RandomAccessFile and doesn't close it. The debugger sais that Exception has thrown in writeOccurencesFile but using Java Leak Detector i found out that the pdf were already opened and not close after parsing to pure text.

Thank you!

You're not supposed to catch a NullPointerException, it's a programmer error that needs to be fixed. Empty catch blocks is also a great way to make sure you don't know what's going wrong in your code. — Kayaman
– Kayaman, Commented May 24, 2018 at 11:25
Also consider using try with resources. Makes sure everything is closed. — Ben
– Ben, Commented May 24, 2018 at 11:27
@mko The code never tries to open more than two files at one time, so raising the open files limit is definitely the wrong way to go. — David Conrad
– David Conrad, Commented May 24, 2018 at 12:20
What does getCategoryID(String) do? Does it open any files to determine the category? There are a lot of problems with this code, not handling exceptions properly, not closing fileWriter if fileReader.close() fails, not checking if they're null before trying to close them, not using try-with-resources, but it doesn't look like leaving files open is one of them. — David Conrad
– David Conrad, Commented May 24, 2018 at 12:24
$getCategoryID(String) simply scan ComputerScience array for get the corresponding index-value of a category, in my example getCategoryID("B") will return 2. (Yeah i know it would be 1, but for some reasons i need to "normalize"). Now i'm trying with try-with resources — MARCO LAGALLA
– MARCO LAGALLA, Commented May 24, 2018 at 12:41

Eby Jacob · Accepted Answer · 2018-05-24 11:45:34Z

0

Try using this utility specifically designed for the purpose.

This Java agent is a utility that keeps track of where/when/who opened files in your JVM. You can have the agent trace these operations to find out about the access pattern or handle leaks, and dump the list of currently open files and where/when/who opened them.

When the exception occurs, this agent will dump the list, allowing you to find out where a large number of file descriptors are in use.

answered May 24, 2018 at 11:45

Eby Jacob

1,4681 gold badge13 silver badges29 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

MARCO LAGALLA · Accepted Answer · 2018-05-24 14:20:34Z

i have tried using try-with resources; but the problem remains. Also running in system macos built-in console print out a FileNotFound exception at the line of FileWriter fileWriter = ...

 static boolean writeOccurencesFile(String WORDLIST,String categoria, TreeMap<String,Integer> map) {
    File dizionario = new File(WORDLIST);


    try (FileWriter fileWriter = new FileWriter( "files/" + categoria + ".txt" , true)) {

        try (FileReader fileReader = new FileReader(dizionario)) {

            try (BufferedReader bufferedReader = new BufferedReader(fileReader)) {

                if (dizionario.exists()) {

                    StringBuffer stringBuffer = new StringBuffer();
                    String parola;
                    StringBuffer line = new StringBuffer();
                    int contatore_index_parola = 1;

                    while ((parola = bufferedReader.readLine()) != null) {

                        if (map.containsKey(parola) && !parola.isEmpty()) {
                            line.append(contatore_index_parola + ":" + map.get(parola).intValue() + " ");
                            map.remove(parola);
                        }
                        contatore_index_parola++;
                    }

                    if (!line.toString().isEmpty()) {
                        fileWriter.append(getCategoryID(categoria) + " " + line + "\n"); // print riga completa documento N x1:y x2:a ...
                    }


                } else {
                    System.err.println("Dictionary file not found.");
                }

            }

        }
    } catch (IOException e) {
        e.printStackTrace();
    }

    return true;
}

This is the code that i am using now, although the bad managing of Exception, why the files seem to be not closed?

Now i am making a test with File Leak Detector

If this doesn't answer your question, you should edit the question instead.

PedroAJunior · Accepted Answer · 2018-05-24 11:50:26Z

-2

Maybe your code raises another exception that you are not handling. Try add catch (Exception e) before finally block

You also can move BufferedReader declaration out the try and close it in finally

answered May 24, 2018 at 11:50

PedroAJunior

273 bronze badges

1 Comment

AxelH Over a year ago

Without a catch somewhere, an exception doesn't just dissapear, in the worst case, the JVM will get it and kill the process.

Collectives™ on Stack Overflow

Java too many open files exception

3 Answers 3

Comments

1 Comment

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

1 Comment

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related