mysql - Bulk native SQL inserts from Hibernate -
i need perform batch operation inserts 10000 rows in empty mysql table. these rows come file values each column, this:
(col1,col2,col3) values ('sample','es_es','other value') (col1,col2,col3) values ('john','en_us','other value') (col1,col2,col3) values ('james','es_es','other value') ...
so know table have put values in, , need fast possible.
first have tested doing one-by-one (calling sqlinsertraw in loop), , takes 20 seconds:
@transactional(propagation = propagation.requires_new) @override public void sqlinsertraw(string tabla, string valores) { getentitymanager().createnativequery("insert " + tabla + " " + valores).executeupdate(); getentitymanager().flush(); }
and secondly tested not commiting each insert, execute each query , after call .flush. takes 7 seconds:
@transactional(propagation = propagation.requires_new) @override public void sqlinsertraw(string tabla, list<string> valuelist) { entitymanager em = getentitymanager(); (string value : valuelist) { try { em.createnativequery("insert " + tabla + " " + value).executeupdate(); } catch (exception e) { throw e; } } getentitymanager().flush(); }
but found problem: there wrong rows in input file (with unescaped quotes, etc.).
(col1,col2,col3) values ('sample','es_es','other wrong'value')
if exception thrown (javax.persistence.persistenceexception: org.hibernate.exception.sqlgrammarexception
), transaction closes , loop not work... next executeupdates
throw transactionrequiredexception
.
is there way ommit exceptions or validate query before?
i have tested norollbackfor
parameter no luck.
Comments
Post a Comment