Loading a file into memory(Java)?

You are certainly on the right track thinking you should read this into memory and access it via some sort of mapping. This will remove a lot of the bottleneck, namely being disk I/O and access time (memory is much faster).

I would recommend reading the data into a HashMap with the ID being the key and the Text being the value.

Try something like:

public Map<Integer, String> getIdMap(final String pathToFile) throws IOException {
    // we'll use this later to store our mappings
    final Map<Integer, String> map = new HashMap<Integer, String>();
    // read the file into a String
    final String rawFileContents = new String(Files.readAllBytes(Paths.get(pathToFile)));
    // assumes each line is an ID + value
    final String[] fileLines = rawFileContents.split(System.getProperty("line.separator"));
    // iterate over every line, and create a mapping for the ID to Value
    for (final String line : fileLines) {
        Integer id = null;
        try {
            // assumes the id is part 1 of a 2 part line in CSV "," format
            id = Integer.parseInt(line.split(",")[0]);
        } catch (NumberFormatException e) {
            e.printStackTrace();
        }
        // assumes the value is part 2 of a 2 part line in CSV "," format
        final String value = line.split(",")[1];
        // put the pair into our map
        map.put(id, value);
    }
    return map;
}

This will read the file into memory (in a String), then cut it up into a Map so that it's easy to retrieve the values, ex:

Map<Integer, String> map = getIdMap("/path/to/file");
final String theText = map.get(theId);
System.out.println(theText);

This sample code is untested, and makes some assumptions about your file format, namely that it's one ID and value per line, and that they ID's and Values are comma separated (CSV). Of course, if your data is structured a little differently, just tweak to taste.

UPDATED to match your file description:

public Map<String, String> getIdMap(final String pathToFile) throws IOException {
    // we'll use this later to store our mappings
    final Map<String, String> map = new HashMap<String, String>();
    // read the file into a String
    final String rawFileContents = new String(Files.readAllBytes(Paths.get(pathToFile)));
    // assumes each line is an ID + value
    final String[] fileLines = rawFileContents.split(System.getProperty("line.separator"));
    // iterate over every line, and create a mapping for the ID to Value
    for (final String line : fileLines) {
        // get the id and remove the leading '>' symbol
        final String id = line.split(" ")[0].replace(">", "").trim();
        // use the key 'transcript_biotype:' to get the 'IG_D_gene' value
        final String value = line.split("transcript_biotype:")[1].trim();
        // put the pair into our map
        map.put(id, value);
    }
    return map;
}

Agreeing with most other comments. 60 MB is not too large for today's memories. But where the time is being sucked is almost certainly in that "+=" appending each line to an increasingly monstrous single string. Make an array of lines.

Better yet, separate out the ID text and the "related text" while reading, to make the later ID searching faster. A hash table would be ideal.

If the file contains a collection of records , then you can
1.Create a class that has id and text content attributes.
2.Read each record from the file and create an object from it and add it to a HashMap.
3. Use the HashMap to retrieve objects by ID

Supposing that your VM has enough heap assigned to it, you can load the raw file into memory like so:

public byte[] loadFile(File f) throws IOException {
    long size = f.length();
    InputStream source;
    byte[] bytes;
    int nread;
    int next;

    if (size > Integer.MAX_VALUE) {
        throw new IllegalArgumentException("file to long");
    }
    bytes = new byte[(int)size];

    source = new FileInputStream(f);

    for (next = 0; next < bytes.length; next += nread) {
        nread = source.read(bytes, next, bytes.length - next);
        if (nread < 0) {
            throw new FileTruncatedWhileReadingItException();
            // or whatever ...
        }
    }
    if (source.read() != -1) {
        throw new FileExtendedWhileReadingItException(); 
        // or whatever ...
    }

    return bytes;
}

You can then process that in-memory copy instead of reading from disk by creating a ByteArrayInputStream around it -- you should be able to plug that in to your existing code with relative ease.

There may be other ways to optimize still more. For example, if processing the data necessarily involves decoding them into characters, then you could cache the results of the decoding by using a Reader to read into a char[] instead of an InputStream to read into a byte[], and then by proceeding similarly. Do note, however, that storing ASCII data in char form takes twice as much space as storing it in byte form.

If the data are suitable, then it would probably be useful to perform a full parse into some more sophisticated data structure, such as a Map, which could make the subsequent lookups extremely fast. The price, of course, is even more memory usage.

I think your problem come from the addition of string on text. You should use a StringBuffer instead. I also advice you to use a Scanner class instead of FileReader :

public String createLocalFile(String path)
{   
    StringBuffer text = new StringBuffer();
    try
    {
        Scanner sc = new Scanner( new File(path) );
        while( sc.hasNext() )
        {
            String currentLine = sc.nextLine();
            text.append(currentLine);
            System.out.println( currentLine );
        }

    }
    catch(IOException ex)
    {
        System.out.println(ex.getMessage());
    }
    return text.toString();
}

That should be much faster.

What you are working with is a FASTA file. Give BioPerl a try...there are tons of libraries to parse and work with these kinds of files. Whatever you are doing, it is most likely done already....