Set the all-visible flag on heap page before writing WAL record, not after.

If we set the all-visible flag after writing WAL record, and XLogInsert
takes a full-page image of the page, the image would not include the flag.
We will then proceed to set the VM bit, which would then be set without the
corresponding all-visible flag on the heap page.

Found by comparing page images on master and standby, after writing/replaying
each WAL record. (There is still a discrepancy: the all-visible flag won't
be set after replaying the HEAP_CLEAN record, even though it is set in the
master. However, it will be set when replaying the HEAP2_VISIBLE record and
setting the VM bit, so the all-visible flag and VM bit are always consistent
on the standby, even though they are momentarily out-of-sync with master)

Backpatch to 9.3 where this code was introduced.

Author: hlinnaka

src/backend/commands/vacuumlazy.c

@@ -1213,6 +1213,13 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
 
 	PageRepairFragmentation(page);
 
+	/*
+	 * Now that we have removed the dead tuples from the page, once again
+	 * check if the page has become all-visible.
+	 */
+	if (heap_page_is_all_visible(onerel, buffer, &visibility_cutoff_xid))
+		PageSetAllVisible(page);
+
 	/*
 	 * Mark buffer dirty before we write WAL.
 	 */
@@ -1231,14 +1238,13 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
 	}
 
 	/*
-	 * Now that we have removed the dead tuples from the page, once again
-	 * check if the page has become all-visible.
+	 * All the changes to the heap page have been done. If the all-visible
+	 * flag is now set, also set the VM bit.
 	 */
-	if (!visibilitymap_test(onerel, blkno, vmbuffer) &&
-		heap_page_is_all_visible(onerel, buffer, &visibility_cutoff_xid))
+	if (PageIsAllVisible(page) &&
+		!visibilitymap_test(onerel, blkno, vmbuffer))
 	{
 		Assert(BufferIsValid(*vmbuffer));
-		PageSetAllVisible(page);
 		visibilitymap_set(onerel, blkno, buffer, InvalidXLogRecPtr, *vmbuffer,
 						  visibility_cutoff_xid);
 	}