Faulty index numbering

One specific problem is that your index counting is incorrect. Rather than use indexes of 0 to 7, you use 0 to 8 while skipping # 6.

Alternative implementation

Here is my implementation. A bit more concise and simpler, though basically the same idea as yours. I assume the entire data file can fit in memory.

Let's define your input data as a string, for simplicity.

I corrected the last field of each book to use a COLON SPACE as delimiter rather than just COLON.

String input =
        """
        ISBN: 9781119846475
        category: Computers
        author: Andy Rathbone
        title: Windows 11 For Dummies
        language: English
        pages: 464
        price: 25
        itemsInStock: 24
                        
        ISBN: 9781118506851
        category: Music
        author: Alistair Wood
        title: Ukulele Exercises For Dummies
        language: English
        pages: 272
        price: 23
        itemsInStock: 5
        """;

Define a class as a record to hold the data parsed from each book entry.

record Book(
        String isbn ,
        String category ,
        String author ,
        String title ,
        String language ,
        int pages ,
        BigDecimal price ,
        int itemsInStock

) { }

Split the overall input string into an array of strings. Each string represents one book.

String[] bookInputs = input.split( "\n\n" );

Make a list where we will store our resulting Book objects.

List < Book > books = new ArrayList <>( bookInputs.length );

Give a name to the COLON & SPACE characters delimiting the key-value pair in each line.

final String FIELD_DELIMITER = ": ";

Loop each book entry string. Split into individual lines.

Each line is a key-value pairing, delimited by a pair of characters, COLON & SPACE. So split on that pair. We always want the second item of that pair. With annoying zero-based index counting, that means we always want index 1.

for ( String bookInput : bookInputs )
{
    String[] lines = bookInput.split( "\n" );
    Book book =
            new Book(
                    lines[ 0 ].split( FIELD_DELIMITER )[ 1 ] ,
                    lines[ 1 ].split( FIELD_DELIMITER )[ 1 ] ,
                    lines[ 2 ].split( FIELD_DELIMITER )[ 1 ] ,
                    lines[ 3 ].split( FIELD_DELIMITER )[ 1 ] ,
                    lines[ 4 ].split( FIELD_DELIMITER )[ 1 ] ,
                    Integer.parseInt( lines[ 5 ].split( FIELD_DELIMITER )[ 1 ] ) ,
                    new BigDecimal( lines[ 6 ].split( FIELD_DELIMITER )[ 1 ] ) ,
                    Integer.parseInt( lines[ 7 ].split( FIELD_DELIMITER )[ 1 ] )
            );
    books.add( book );
}

Dump to console.

System.out.println( "books = " + books );

books = [Book[isbn=9781119846475, category=Computers, author=Andy Rathbone, title=Windows 11 For Dummies, language=English, pages=464, price=25, itemsInStock=24], Book[isbn=9781118506851, category=Music, author=Alistair Wood, title=Ukulele Exercises For Dummies, language=English, pages=272, price=23, itemsInStock=5]]

This code is for demonstration only. In real work I would do a bunch of data validation.

As its name implies, reader.readLine() reads one line of text, so there is no "\n" in the line that it returns.

I would do something like this:

private static final Pattern RE = Pattern.compile("^\\s*([^\\s:]+)\\s*:\\s*(.+)$");

...

    List<BookClass> bookList = new ArrayList<>();
    String bookInfo = "books.txt";
    try (FileInputStream stream = new FileInputStream(bookInfo);
            Reader reader = new InputStreamReader(stream, "UTF-8");
                BufferedReader in = new BufferedReader(reader)) {
        Map<String,String> currentBook = new HashMap<>();
        String line;            
        while ((line = in.readLine()) != null) {
            if (line.trim().length() == 0) {
                // empty line: record separator
                if (!currentBook.isEmpty()) {
                    BookClass bookObj = makeBook(currentBook);
                    bookList.add(bookObj);
                    currentBook.clear();
                }
            } else {
                Matcher matcher = RE.matcher(line);
                if (matcher.matches()) {
                    currentBook.put(matcher.group(1), matcher.group(2));
                }
            }
        }
        if (!currentBook.isEmpty()) {
            BookClass bookObj = makeBook(currentBook);
            bookList.add(bookObj);
        }
        System.out.println(bookList);
    } catch (IOException ex) {
        Logger.getLogger(Main.class.getName()).log(Level.SEVERE, null, ex);
    }

...

private static BookClass makeBook(Map<String, String> currentBook) {
    BookClass bookObj = new BookClass(
            currentBook.get("ISBN"),
            currentBook.get("category"),
            currentBook.get("author"),
            currentBook.get("title"),
            currentBook.get("language"),
            Integer.parseInt(currentBook.get("pages")),
            Double.parseDouble(currentBook.get("price")),
            Integer.parseInt(currentBook.get("itemsInStock")));
    return bookObj;
}