Skip to main content

BULK INSERTS EJB3

11 replies [Last post]
priyasubu
Offline
Joined: 2007-08-21

Hi

I have ar equirement to have bulk inserts
I Know in hibernate we shouldn't flush everytime during a bulk insert
ie
say for every 10 rows I want to flush I will explicitly set that how do I do the same for ejb3

Reply viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
kurti
Offline
Joined: 2007-06-12

With EJB3 and CMT, there is no way to change the transaction. You can only cause a rollback, that's all. But you can mark one bean to use BMT by annotating the class with

@TransactionManagement(TransactionManagementType.BEAN)

then let a field be injected with

@Resource
private EJBContext ctx;

The context gives you a UserTransaction on which you can do begin/commit/rollback.

kurti
Offline
Joined: 2007-06-12

flush() + clear() does not work, as at least Hibernate does not store entries to the disk if you do this.

I had success with this code:

final UserTransaction tx = ctx.getUserTransaction();
tx.begin();

for (...) {
em.persist(wa);

if (0 == ++pos % BATCH_SIZE) {
tx.commit();
tx.begin();
}
}
tx.commit();

flush() and clear() made no difference, neither concerning memory nor speed.

Witold Szczerba

2008/11/4 :
> flush() + clear() does not work, as at least Hibernate does not store entries to the disk if you do this.

Neither flush() nor clear() methods are supposed to commit
transaction, so do not expect anything to be "stored to the disk"
after invoking those methods.

Regards,
Witold Szczerba

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@glassfish.dev.java.net
For additional commands, e-mail: users-help@glassfish.dev.java.net

priyasubu
Offline
Joined: 2007-08-21

Can anyone let me know please since my container is managing the session how do I ask the container to flush say for 10 rows or something
Cheers
Priya

priyasubu
Offline
Joined: 2007-08-21

Basically how do I do this in EJB3 using CMT

Session session = sessionFactory.openSession();
Transaction tx = session.beginTransaction();

for ( int i=0; i<100000; i++ ) {
Customer customer = new Customer(.....);
session.save(customer);
if ( i % 20 == 0 ) { //20, same as the JDBC batch size
//flush a batch of inserts and release memory:
session.flush();
session.clear();
}
}

tx.commit();
session.close();

priyasubu
Offline
Joined: 2007-08-21

The only way I can think of is to use user transactions?
Any other ideas please
help
Cheers
Priya

whartung
Offline
Joined: 2003-06-13

Yea, why use JPA at all for bulk inserts? Why not simply use JDBC directly? For bulk inserts you don't have to worry about cache corruption, so JDBC works quite nicely in this case.

priyasubu
Offline
Joined: 2007-08-21

I am not sure how you say cache corruption wont happen when I say insert what I do is
I read somehting from a webservice and update my database so this can be an update if this row is there or an insert if an update wont it be corrupting cache if I use jdbc
Cheres
priya

whartung
Offline
Joined: 2003-06-13

If you insert something new in to the database, you're not corrupting the cache because there's nothing for the data row in the cache yet.

Typical cache corruption happens when you update the DB behind the back of the cache, i.e. changing a row so that the DB has different data than the cached row. But an insert doesn't have this problem, because the row doesn't exist in the cache yet. So, using JDBC for bulk inserts is no problem.

Updates are best managed through the JPA, specifically because of the caching issue.

But bulk updates are readily applied using the JPA and EQL, as EQL is cache sensitive and will "do the right thing" when you use EQL for updates.

But if you are bulk loading a lot of data, direct JDBC inserts are simply far more efficient.

Dennis Gesker

My experience has also been that JDBC is better for inserting bulk data.

However, the draw back *seems* to be that when the rest of the application
is using JPA (for instance for handing data from the database to JSF for
reporting) JPA remains unaware of the new data in the tables.

Dennis

On Tue, Nov 27, 2007 at 12:20 PM, wrote:

> If you insert something new in to the database, you're not corrupting the
> cache because there's nothing for the data row in the cache yet.
>
> Typical cache corruption happens when you update the DB behind the back of
> the cache, i.e. changing a row so that the DB has different data than the
> cached row. But an insert doesn't have this problem, because the row doesn't
> exist in the cache yet. So, using JDBC for bulk inserts is no problem.
>
> Updates are best managed through the JPA, specifically because of the
> caching issue.
>
> But bulk updates are readily applied using the JPA and EQL, as EQL is cache
> sensitive and will "do the right thing" when you use EQL for updates.
>
> But if you are bulk loading a lot of data, direct JDBC inserts are simply
> far more efficient.
> [Message sent by forum member 'whartung' (whartung)]
>
> http://forums.java.net/jive/thread.jspa?messageID=247408
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@glassfish.dev.java.net
> For additional commands, e-mail: users-help@glassfish.dev.java.net
>
>

--
Dennis R. Gesker
email: dennis@gesker.com
google talk: gesker@gmail.com
aim: dennisgesker
pgp/gpg: 0x267D3FE8
First things first, but not necessarily in that order. -- Unknown
[att1.html]

Witold Szczerba

2008/11/5 Dennis Gesker :
> My experience has also been that JDBC is better for inserting bulk data.
>
> However, the draw back seems to be that when the rest of the application is
> using JPA (for instance for handing data from the database to JSF for
> reporting) JPA remains unaware of the new data in the tables.
>
> Dennis

There are other issues as well, for example:
https://glassfish.dev.java.net/issues/show_bug.cgi?id=3866
but this one can be worked around.

Another good idea is to disable level2 cache or clear cache for
specific entities by class and/or id or entire cache after JDBC
operations (for example A owns collection of B, then after inserting B
using JDBC it is wise to clear cache of A-class instances).
It is very important not to forget to flush entity manager before
using JDBC if JPA was used before JDBC operations.

Regards,
Witold Szczerba

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@glassfish.dev.java.net
For additional commands, e-mail: users-help@glassfish.dev.java.net