I have a ton of data I have to parse and insert into a mysql database.
Some of it is duplicate, so I setup a unique index on the column that needs to be unique.
Instead of running the insert queries 1 by 1, I'm doing batches of 100 to lessen the load on the DB. However if I get a duplicate entry error, it seems that entire batch is not inserted. Is there a way to structure the query to say "if there are duplicates ignore those but please insert the rest of the data"?
Thanks in advance.