Array elements and hash values are scalars, so when you are nesting arrays and hashes, you must use references. Just as $authors->[$x] is not a hash but a reference to a hash, you must set $authors->[$x]->{'books'} to a reference to the array.
$authors->[$x]->{'books'} = \@books; # reference the original array
$authors->[$x]->{'books'} = [@books]; # reference a copy
You would then access elements of the array using something like
$authors->[$x]->{'books'}->[0]
which can be abbreviated
$authors->[$x]{books}[0]
or access the whole array as
@{$authors->[$x]{books}}
Your original attempt
$authors->[$x]->{'books'} = @books;
is exactly equivalent to
$authors->[$x]->{'books'} = scalar @books;
because the left operand of the = operator is a hash value, which is a scalar, so the right operand is evaluated in scalar context to provide something that can be assigned there.
P.S.
On rereading this answer I realized it may be confusing to say "a hash value is a scalar" because of the possible interpretation of "hash value" as meaning "the value of a hash variable" i.e. "the whole hash". What I mean when I write "hash value" is an item that is stored in a hash as a value (as opposed to a key).